Lab 9a: Making Heatmaps
heatmapbanner.png

XKCD-heatmap.png
Source: http://imgs.xkcd.com/comics/heatmap.png

What is a "heat map"?

A heat map is a raster image in which the color of each pixel is based on the result of an inverse distance weighted function applied to a collection of points. Each pixel is rated on the basis of how many data points are how near. Sometimes it is referred to as density mapping. It's useful when you have a large number of point observations spread across space.

Basics

Start with a set of point data:

heatmap_points.png

And then we impose a grid on this data:

heatmap_pointsWgrid.png

And then we count the number of points in each grid square (here we've made the grid fine enough that it has at most one point in each square). We can then create a "graduated theme" on the grid, coloring squares with the lowest value (0) white and the highest value (1) red.

heatmap_level0.png

But we want to take into account some sense of spatial effects : even if you don't have a point in your square, being next to a square with a point is more "point-ish" than being next to other empty squares. But it is not the same as having your own point either. So, what we we score ourselves 1 if we have a point and then add to that 0.75 for each of our immediate neighbors that has a point and then beyond that nothing Thus, the effects would be added up like this:

0.75 0.75 0.75
0.75 1.0 0.75
0.75 0.75 0.75

Thus, a square with a dot and four neighbors with dots would be scored $1+3=4$. An empty square surrounded by dots would score $8 \times 0.75 = 6$. Again, we'll color the grid with white for the minimum value (0) and red for the maximum (as high as 7 for a dot surrounded by dots).

heatmap_level1.png

STOP and THINK: Consider the grid we looked at last week. If contribution of each nearest neighbor is 0.5 and the square itself counts 1, assign values to each of the squares in the grid. Assume squares on the outside of the grid are 0s.

sample_grid01.png

But why stop at this level? If there is some "influence" from dots that are one step away, surely there is some (if a little bit less) influence from dots that are two steps away. We could represent this by weighting the "2-neighbors" less (here we are using 0.75 for 1 neighbors and 0.3 for 2-neighbors):

0.03 0.03 0.03 0.03 0.03
0.03 0.75 0.75 0.75 0.03
0.03 0.75 1.0 0.75 0.03
0.03 0.75 0.75 0.75 0.03
0.03 0.03 0.03 0.03 0.03
heatmap_level2.png

STOP and THINK: Calculate values for the top left quadrant under first neighbors contributing 0.5 and second neighbors contributing 0.2.

sample_grid02a.png

In practice we will be able to specify how far out the influence goes and how it is attenuated by distance as well as what the grid size is. What we end up with is (shown here zoomed in on) a raster with each pixel assigned a density value and colored accordingly.

heatmap_pixel_values.png

RT%!M

Before proceeding, read through the User Manual Page on the heat map plugin.

Alternatively, you might want to view Rudi von Staden's video tutorial

Tutorial

Work through Part I of the "Making Heatmaps using QGIS" tutorial by Ujaval Gandhi (if you are using a newer version of QGIS, use the updated tutorial). Depending on your QGIS version, your screens may appear slightly different from what is shown there.

This compressed file, LAB09A_Data.zip, contains a clean version of the data. This compressed file, LAB09A_geoContext.zip, contains background boundary layers for the County of Surrey and the greater London area. There is a project file in the context directory that will set you up with an attractive background map. The data file will allow you to skip the data cleaning steps and start the old tutorial at Click on Plugins → Manage Plugins or the new one at step 6.

NOTE: If your machine has trouble digesting the 120,000 data point crime layer you may want to try using a 10% sample (about 10,0000 points) or even a 2% sample (2-3 thousand points). Or, if this step continues to get in the way, you can use this heat map raster file and proceed from the steps after that. The sample files are zip archives of shape files. Each also contains a geotiff of the heat map raster based on the data using same parameters described in the tutorial in case you still can't get it to work.

heatmap_data.png

1. Start with a point layer.

right_arrow_orange.png

2. Use heatmap plugin; here we specify 1000 meter "influence" radius around data points and each pixel in the output raster will represent 100 meters square in the real world.

heatmap_settings.png
down_arrow_orange.png

3. For more on these parameters see the User Manual Page on the heat map plugin.

heatmap_colorize.png
left_arrow_orange.png

4. Output is a grey-scale raster image with each pixel assigned a value corresponding to computed data density. Next step is to change grey scale to a color ramp.

heatmap_v1.png
down_arrow_orange.png

5. The values next to the colors represent the threshold values for each color but the color scale is varied continuously based on the pixel values.

heatmap_v2.png
right_arrow_orange.png

6. Sometimes a heat map raster will cover the entire extent with color because most areas beyond the data is mapped to 0. We can erase this by setting zero valued pixels to transparent.

heatmap_transparency.png
down_arrow_orange.png

7. And we slide the layer's transparency up so we can see the underlying map and data points.

heatmap_raster_calculator.png
left_arrow_orange.png

8. Next we dichotomize the raster, using the "Raster Calculator."

heatmap_zero_transparent.png
down_arrow_orange.png

9. This creates a new raster in which pixels have value of 1 or 0 based on the threshold expression in the calculator (here 3.5 is used based on using the I(nfo) tool to ascertain the approximate value at the edge of the blue zone in the heat map).

heatmap_hi_lo.png
right_arrow_orange.png

10. Use Raster > Conversion > Polygonize to create polygons based on pixels that have the same value.

heatmap_polygonize.png
down_arrow_orange.png

11. Since we are using the dichotomous raster as input, we'll get polygons around the blue (hot) spots and around all the rest.

heatmap_zero_polygons_selected.png
left_arrow_orange.png

12. If we examine the attribute table of this new polygon layer we will find lots of features with value 1 (these are the hotspots) and just a few with value 0 (the rest). We'll select and delete the zeroes.

heatmap_polygonized.png
down_arrow_orange.png

13. Now we have a raster heat map and we have the hot spots in a polygon layer that can be used for further analysis.

heatmaphotspots.png

Finished Products

Adjust styles and add legends and titles as appropriate. Produce two finished maps in print composers exported at PNG files.

FinishedProduct1.png
FinishedProduct2.png

Data

File nameFile typeSize
CrimeData_10pc_sample.zipZip archive data770.71 kBInfo
CrimeData_2pc_sample.zipZip archive data313.79 kBInfo
HeatMapGeoTiff.tifTIFF image data1.17 MBInfo
LAB09A_Data.zipZip archive data1.58 MBInfo
LAB09A_geoContext.zipZip archive data1.19 MBInfo

References

Data Sources

Acknowledgments, etc.

Created October 2013. This lab is based on two tutorials created by Ujaval Gandhi (this one and this one)

Total time for data cleaning, version checking, image creation, writing and editing approximately 12 hours.