We recently completed a project which involved automatically creating heatmaps containing click data from a questionnaire. In this article I’ll discuss the details of how we generated these maps. Sample code will be given in Processing, but the concepts are applicable to any programming language/API that lets you manipulate individual pixels.

Heatmaps

Clickmap (left) and corresponding heatmap

The project’s basic premise was that users would get a set of questions related to the area in which they live, questions along the lines of “What’s your favorite spot?” or “Which place do you think could use more trees?”. They answered these questions by clicking on a map. These clicks would be stored in a database and periodically evaluated by a server-side utility written in C++, which would create clickmaps (containing the locations of every click for a given question) and heatmaps (giving a rough overview of the relative click densities in different regions on the map).

The total number of clicks recorded would typically be around a couple hundred per question, decidedly less than the amount of pixels available on the map, so any approach that simply tallies up clicks on every pixel and then assigns that pixel a color was out of the question. Clicks needed to affect the region around them, otherwise the heatmaps wouldn’t look much different from the clickmaps.

Gradients, blending, color mapping

In principle, the solution we came up with is very simple. It’s composed of two steps: creating or updating an intermediate gradient image (I’ll call this the “gradient map”) and recoloring that gradient map to produce the final heatmap.

Start with a black image. For each click, blit an image containing a radial gradient from black to white (white at the center) onto the gradient map, centered at the click’s coordinates and drawn with additive blending.

Neighboring clicks on a gradient map. Each click is drawn additively as a gradient from black to white.

The image above shows how this works for two neighboring clicks. At the clicks’ centers, the densities are 100%. As you move from the centers to the fringes, the densities become 0 (i.e. the pixel colors get darker until they reach black). But where the halos of the two clicks overlap, depending on the distance between the two centers, the densities don’t go all the way down.

The second step is to produce a heatmap by taking a copy of the gradient map and recoloring it. This is pretty straight-forward: Create a gradient image from blue to red (or whatever colors you want) in Photoshop and use it to map brightness levels in your gradient map to colors in your resulting heatmap.

In our case we wanted the heatmap to not display absolute densities, but to always show the hottest parts in red, so we scaled the color mapping to the brightest pixel in our gradient map (so far this would still be white, but that will change by the end of this article).

There are two very neat things about this method:

  • You won’t have to recreate the gradient map from scratch every time you add a click. Simply blit another copy of the bitmap that stores your gradient (let’s call this your “brush”) onto the existing image and update the heatmap by redoing the color mapping.
  • You get really great control over the falloff of a click’s influence on its surroundings. Simply edit the brush in Photoshop, adjusting the brightness curve. Want a click to add a broad halo onto your heatmap? Make the gradient ease towards white. Want every click to add a sharp but small pinch, with a large but faint halo outside? Adjust accordingly.

Different gradients for the brush

Different brush images control the influence of a click on its environment. Left: linear falloff. Center: easing towards 100%. Right: small pinch in the middle, faint halo outside.

There is however one glaring problem with this method: A typical RGB image stores 8 bits per channel for 256 levels of brightness, so your gradient map will get saturated very quickly. Suppose the brush has a gradient from 0×000000 black to 0xffffff white and you evaluate two clicks that are right on top of each other: Well, the centers of the clicks are already 0xffffff white, and it doesn’t get any whiter than that. So while in the resulting heatmap these two clicks will have a different gradient than a single click would have, their centers couldn’t get any hotter than the center of a single click would be.

It might seem like having the gradient in the brush go from black to some level of grey would alleviate this issue, but that really doesn’t scale well either. Even taking the extreme case where the brush’s gradient goes from 0×000000 to 0×010101 (which isn’t really a gradient at all, just a background of one color and a blob of another), you’d only have 255 clicks with an overlapping area of influence until you hit maximum brightness.

R8G8B8 → B24

Your typical bitmap is composed of 3 channels (or 4, if we include alpha) of 8 bits of color depth, giving you 256 values of brightness. Additively blending two images simply adds the values in each image’s channels, resulting in an output image that is literally the sum of its parts. Our problem is that 256 levels of brightness really don’t give us the fidelity we need.

So far, we’ve been thinking in terms of greyscale images and brightness levels. In truth, that’s very redundant: in an RGB image, a greyscale brightness level is stored by setting the (8 bit) red, green and blue channels to the same value. We could have done all the operations in the method outlined above, using only the blue color channel for both the gradient map and the gradient stored in the brush, and leaving the green and red channels blank, without any loss of fidelity. In the process, we would have freed up two whole bytes per pixel.

What if we could put those two extra bytes to use and create an image consisting of a 24 bit blue channel (or even 32 if we use alpha as well), with 0 bits for red and green? 24 bits would allow us to additively blit an 8 bit (i.e. 0…255) gradient on top of itself 65536 times before reaching saturation, more than enough for our purposes (and 32 bits would allow more than 16 million clicks on top of each other).

As it turns out, treating a 24 bit RGB image as a 24 bit single channel image is mostly just a matter of reinterpreting the data:
Most APIs’ getPixel and setPixel methods return or take an integer in that format anyway. GetPixel implementations usually return color values as bytes packed into a single integer, in RGB or ARGB format, and it actually takes some bit-fiddling to extract the red, green and blue components. The PNG format we use to store and retrieve our images doesn’t care about whether we think of each three byte pixel as three separate color channels or as one big integer either.
All we have to do is write the drawing code to blend one image onto another by ourselves (for additive blending, this just means adding the pixels of each image), and live with the fact that our 24 bit single channel images look very weird when opened in an image viewer.

To visualize this idea of a 24 bit channel, take a look at the following illustration:

24-bit gradient

Image showing a 24-bit gradient from 0 to 767, visualized as a RGB image. When interpreted as 24-bit single channel, the int part is the actual pixel value.

The image shows a gradient starting at integer value 0 (0×000000) at the left-most pixel, going to integer value 767 (0x0002ff) at the right-most pixel.
Up to color value 255 (0x0000ff), what’s happening is very straight-forward, but things get interesting when we move from 255 to 256 (0×000100). In the RGB interpretation, what happens is that we go from RGB(0,0,255) to RGB(0,1,0). If we kept thinking in terms of RGB, the gradient would then continue with (0,1,1), (0,1,2), and so on. After (0,1,255), the next pixel would be (0,2,0). Eventually, after a long time, the gradient would hit (0,255,255), after which the next pixel would be (1,0,0). Again, this should all become a lot more intuitive if you simply think of the three separate RGB bytes as bytes in a single integer.

24-bit overlaps

Overlapping clicks in a 24-bit image.

Taking this into the second dimension, the image above shows how clicks affect the 24 bit gradient map: The two brush blits to the left are basically the same as in the RGB greyscale scenario. In the center, two clicks have been made a little closer to each other, so that their gradients added together exceed 0x0000ff. This results in the “dark” spot in the center, which in an RGB interpretation would have a green channel value of 1 (instead of 0).

Finally, the shape to the right is the result of many hundred clicks in the same area. As the blue component in the RGB interpretation wraps around 255 many times, the green component becomes higher and higher, resulting in the visible green halo in the center of the blob.

Things to watch out for

It’s important to note that you’ll need to store gradient maps in a lossless format, as lossy compression formats like JPG subtly alter color values. For a normal RGB image, JPG compression can be barely noticeable. However in a 24 bit single channel image, a JPG artifact that would normally alter the red channel of a pixel by a single bit will result in a vastly different value.

In the same vein, scaling these images up or down is problematic as well, if you’re using antialiasing.

Sample Code

You can download a sample implementation written in Processing at http://www.philippseifried.com/blog/files/misc/heatmaps.zip. (Note: I’ve updated this sample on 2014/9/9 to work with current versions of Processing)

Heatmap sample code in action

The provided heatmap sample code in action

The sketch lets you click anywhere in the top left map to update the heatmap and clickmap. The gradient map is also shown.

The images folder contains the files heatmapBrush.png and heatmapColors.png, among others. “heatmapBrush” contains the gradient used for the brush. While this is a greyscale image, only the blue channel is used. “heatmapColors” defines the color mapping from cold to hot. The only thing worth pointing out about this image is that it is interpreted as an array of pixels, so it should have a height of 1.

The sample code is MIT licensed, so feel free to build on it and use it in your own projects.

 

7 Responses to Generating heatmaps from code

  1. Jose Manuel Carmona says:

    Hi! I’m really interested in the code because looks like what I need for my school project.

    But when running the sketch, when you click the second time you get an ArrayIndexOutOfBoundsException: -32711740. I found that the problem is because the variable gmValue = gradientMap.pixels[i] is returning -1.6777216E7.

    How can I solve this?

    Best regards,
    Jose

  2. Jose Manuel Carmona says:

    Well I found the problem. You should never use the constructor new PImage(), you should always use loadImage().

    Thanks for that awesome example!

  3. leigh says:

    I’m having the same problem. Jose, can you post the fix?
    thanks!
    Leigh

  4. leigh says:

    ok, so I figured out exactly where the problem lies.

    in “mousereleased” there is a line:

    image(gradientMap, 0, backgroundImage.height);

    gradientMap.pixels is somehow affected by this. This line sets all of the elements in gradientMap.pixels to -16777216. This causes

    float gmValue = gradientMap.pixels[i];

    to be set to -16777216.

    and then

    int colIndex = (int) ((gmValue/maxValue)*(heatmapColors.pixels.length-1));

    becomes nearly the same thing.

    and then when you call

    int col= heatmapColors.pixels[colIndex];

    you end up calling an array position that is out of range.

    very weird problem… any idea what causes it?

    for now, to get it to work, I just commented out the line:

    // image(gradientMap, 0, backgroundImage.height);

  5. Philipp says:

    Hi Jose & Leigh,

    It looks like newer Processing versions broke my sample code. I’ve replaced the .zip with a version that works. Jose was right that PImages shouldn’t be created via the constructor (instead use createImage).

    You were seeing negative array indices because those PImages were created with an alpha channel (AARRGGBB), which was set to opaque. The top bit is 1 in an opaque pixel, but interpreted as an int, that top bit is responsible for the integer’s sign. Hence the negative numbers.

  6. Dan says:

    In case it’s useful to anyone, I’ve ported the code to ruby: https://github.com/dgopstein/ruby-processing-heatmaps

Leave a Reply

Your email address will not be published. Required fields are marked *

*


+ 1 = seven

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>