Lies, Damn Lies, & Statistics

So I fell in love with data and charts back as an undergrad when I took my Political Science methodology class (thank you Professor Folger).  It is then that I learned how they can be used to help explain what happens in the world around us.  However, I also learned how they can be manipulated in such a way to create a false explanation or mislead someone.  That is why the quote in the title has to be one of my favorite sayings of all time.  So much so, that my friends and co-workers roll their eyes every time I say it.


Most of the time the manipulations of data are subtle and those manipulating the data can justify how they are presenting their results.  However, sometimes the manipulation is so outlandish you wonder if it was done purposefully or out of incompetence.  For instance, for those of you who follow data visualization topics or media watchdog groups, you have probably already heard about the Fox News unemployment chart.  I first read about it at FlowingData who links to MediaMatters about this story.  You should definitely check out the comments section for the article at FlowingData for a lively discussion of politics, Fox News, and data.


Here is the Fox News chart for those who missed the story...

Can you spot what is wrong with this chart?  

Maybe it is the fact that even though the unemployment rate for November is the lowest it has been in all of 2011 (8.6.), they keep it even with the point for October (9.0).  I would also like to add the fact that the chart is ugly, difficult to read, and the title is misleading.  The chart is titled "Unemployment Rate Under President Obama".  Then underneath the chart they mention that it is just for 2011.  I would argue that the average attention span of a tv viewer is such that they would look at the title, look at the line in the chart, and by the time they figured it out and the graphic is removed, they come away with the impression that under President Obama the unemployment rate went up and stayed up for the entire Obama presidency to date.  Which is definitely just as misleading as the misplacement of the November number.

 So I re-made the Fox News graph using the same data from the Bureau of Labor Statistics.  And this is what I got.

It is much cleaner looking than the Fox News chart, you see that November has the lowest unemployment rate in 2011 and it lets you know up top that you are looking at data only from 2011.  I left the second line in just in case you haven't been paying attention; the current president in 2011 is Barack Obama.   The misleading part of my chart however is how I spaced the months apart and it makes the line look a bit flatter than it should be.

Of course, I took it a step further and made a chart that shows the unemployment trend from 2001 to current.

Powered by Tableau

Now you can see why I felt the Fox News' original title was misleading.  We can see that since President Obama took office unemployment soared to a peak in October of 2009 of 10.1.  But since then the unemployment rate has been falling.  Even though this isn't a true interactive Tableau visualization, you can still over over the data points to see the specifics for that point.

I colored the line using the 3 month moving average.  Moving averages are often used to help smooth out data that fluctuates.  I am not sure if this is standard practice to use the moving average for the unemployment rate, but I thought it added a layer of depth and better understanding of the trends.  In fact, if you look real close at the last few months on the chart, you will see that it is starting to turn an ever so lighter shade of orange.  Perhaps we have turned the corner on unemployment?

A longer post than normal, and probably a bit more preachy than usual.  However, I have been neglecting my blog for far too long (due mainly to "writer's block").  But I am back!  And I have a few more visualizations over the next couple of weeks.  Stay tuned...

(Update 12/20/2011...Co-workers Sarah and Eli suggested that I place a line in the chart to show where Bush's term in office ends and Obama's begins.  Thanks for the suggestion!)