Truthful Mapping

The exact same data mapped using three common classification methods.  What?  Read on...

Divide and Color
What do each of these thematic maps have in common? They are all mapping the exact same data. Then why are they so different? They each use a different classification method. One of the first things that pops up when whipping up a thematic data visualization that has discrete range brakes (commonly a choropleth map, but not necessarily) is how do I bucketize my data for color assignment?
All three are commonly used and justifiable. And picking the "right" one can have a big impact on what you ultimately communicate.

Some Options
There are lots of ways to carve datasets into discrete classes. I’ll go over three of them...

• Quantile Breaks the data into equally filled groups
• Standard Deviation Breaks the data into statistical chunks diverging from the mean
• Equal Interval Breaks the data into equally distant groups

Or you could just eyeball the data and then divide it into range breaks that look good or are easy to read.  Or-or your breaks are driven by a conceptual categorization that doesn't really have anything to do with math.  Anyways...

Distribution
When choosing a method of classifying your dataset into discrete ranges, there a couple of things to consider off the bat. First, what does the data distribution look like (if it is dynamic, what does itgenerally look like?)? Is it skewed toward one extreme or the other? Is it relatively normal (bell shaped on a histogram)? Are there outliers to consider? Applying various classification methods can create very different impressions of the data. Any interface is a manifestation of tradeoffs, let’s take a look at some examples...
Normal Data With relatively normally-distributed data, picking a classification method may not make a massive difference in your visualization. Your biggest concern is probably how many breaks to make, and what colors to use. Check out this example of average age per county -nice and normal...

Nice normal data looks relatively similar across the three classification methods.  But normal data is pretty rare.

Skewed Data
Now it gets fun. With a data set like the percent of folks who consider themselves multi-ethnic, the distribution is far from normal. In this case, there is a bulge at the lower end and a long tail that eventually pinches off around 30% multi-ethnic. What a difference the classification methods make here!
Does the "Quantile" map below tell the truth? Yes. I can clearly see the locations of higher and lower proportions of multi-ethnic US residents, even regional trends and abrupt shifts. Does the "Equal Interval" map tell the truth? Yes. I get a clear indication that most places in the US are, proportionally, pretty low in multi-ethnic residents.
The two maps look strikingly different because I’m telling the truth about two different things...

Skewed data is a trickier beast.  Different classification methods result in very different pictures.  It's time to ask yourself what is the content of your message, and proceed while honoring the phenomenon (and your audience).

Examples...

For Normal Data
Equal Interval for Normal data
Equal interval slices the data into equally distant range breaks. Some color buckets get more counties than others, but if the distribution is wide, then the visualization will be adequate.
The gist: Evenly spaced, unevenly filled buckets...

Equal Interval for normal data: Evenly spaced, though unevenly filled, buckets.

Quantile for Normal data
Quantile yields a pretty high-contrast map, that is reliably good looking. The fact that the data is normally distributed doesn’t really matter –each bucket has the exact same number of counties, but you’ll notice that in order to accommodate that, the ranges have to span varying distances.
The gist: Unevenly spaced, evenly filled buckets...

Quantile for normal data: Unevenly spaced, though evenly filled, buckets.

Standard Deviation for Normal data
Trusty old Standard Deviation. It is going to look alright in most cases, but it really shines when applied to normal datasets. You’ve got the mean in the middle and you chunk it out from there based on standard deviation distances in either direction from the mean. Beautiful. Also, don’t do what I did –you should put actual values in your legend instead of the math nerd standard deviation breaks. And while we’re at it, it’s often a good idea to pick a diverging color scheme for data that is classified by Standard Deviation. Pick a neutral color for the mean (center) range and then transition to one color on the left and another color on the right. ColorBrewer gives some nice background here along with a rocking tool to generate your own cartographic color schemes.
The gist: Evenly spaced (to a statistician), unevenly filled buckets...

Standard Deviation for normal data: evenly spaced (from a mathy point of view), though unevenly filled, buckets.

For Skewed Data
Quantile for Skewed data
Quantile to the rescue. The buckets are defined by an equal number of member counties. The result is a map that, in spite of a data set that is packed on one end of the spectrum and virtually empty through the rest of the spectrum, you still tease out the comparative terrain of a phenomenon with a map that is guaranteed to have dynamite contrast. It's a bit of statistical/visualization magic, really.

To that point, though, it could be argued that this method implies a false or misleading heterogeneity of the data. While the vast number of counties have a proportionally tiny multi-ethnic population, this method could imply a greater variance (as compared to the Equal Interval example above) than there really is.
Just remember, when reading a map, read those legends and take the range breaks for what they are worth. Quantile is a good illustration of that.
The gist: Unevenly spaced, evenly filled buckets...

Quantile for skewed data: unevenly spaced, though evenly filled, buckets.  Engineered to be handsome, but can be misleading or mask actual disparity.

Standard Deviation for Skewed data
Standard Deviation. It is still providing valuable visual breaks when applied to highly skewed data. But I can never get too cozy with it because it is so hard to explain.
The gist: Evenly spaced (to a statistician), unevenly filled buckets...

Standard Deviation for skewed data: evenly spaced (good luck explaining how), though unevenly filled, buckets.  Also handsome (so long as you don't use the color scheme I did).

How-To’s
Equal Interval

• Figure out the overall value range (highest value – lowest value) for the data population…
• Decide how many classes you want, and divide the overall range value by it, to get the distance value between classes…
• Insert the break every Nth value.

Quantile

• Decide how many classes you want, and divide the overall population by it. This will tell you how many items get lumped into each class…
• Chop up the population into buckets of that many.
• SQL Server has a nice built in method for this, called "Ntile"

Standard Deviation

• https://en.wikipedia.org/wiki/Standard_deviation…
• Good luck.

Also, check out the CartoDB section on "quantification."

Truth
It’s one thing to willfully mislead others by the categorization and representation of data (obviously not cool). It’s another to do it on accident and mislead your audience and yourself. Varying classification methods will produce very different results. In gaining a little background about various methods of classification you’ll be in a better position to…

• Create better, more effective visualizations
• More keenly understand the visualizations of others
• Read the legend
• Use what you learn for good instead of evil

In any case, the thing to keep in mind is effective and truthful communication; your visualization should enable the data to tell it’s story. Let us know if we can be of any help!

This post originated from the UX/visualization blog of IDV Solutions: uxblog.idvsolutions.com

@JohnNelsonIDV