5 Tips to Be Aware of "How Charts Lie" [Book Review]
By: Kristin Hunter-Thomson
A critical eye is key to working with data
WHY DOES THIS MATTER?
Data are driving many decisions from personal to professional to political these days. And with the advent of ever more user-friendly graphing platforms there is a plethora of charts (and maps, graphs, plots, etc.) in the world.
This is a good thing!
We are visual creatures. We make meaning of pictures and images far faster than we make meaning of written or spoken words.
But this is why all these charts can also be a bad thing (when not approached with a critical eye).
For example, Alberto Cairo in his book How Charts Lie: Getting Smarter about Visual Information (2019) shares a story of a US Congressional hearing in 2015 with Planned Parenthood. Jason Chaffetz (Utah-R) shared a plot from the Americans United for Life organization like the chart below on the left to make a case about the counts of procedures that Planned Parenthood provided between 2006 and 2013 across two categories: cancer-screening-and-prevention services or abortions. However, the chart below on the right, made by Cairo using the same data but plotted along a common y-axis scale (which is appropriate as all of the data are counts of procedures across two categories), gives a very different picture of what has happened over time across these two categories. (See Let’s leverage perception science to our advantage! for a deeper dive into why our brains form conclusions from these graphs before our eyes catch up to orient to the numbers and scale.)
Anyone can, and is, making graphs. Most are good, some are misleading by mistake, and some are misleading on purpose. But they are all out there. It is our reality and that of our students.
Therefore, part of being able to successfully live in this data-driven society is 1) being able to read graphs that we come across with a critical eye, and 2) knowing how to make graphs well so that you do not mistakenly mislead readers.
WHAT CAN WE BE AWARE OF WHEN LOOKING AT CHARTS?
In the book, Cairo explores 5 main ways that charts lie (with real-world examples from a range of disciplines and media to demonstrate each):
-
by being designed poorly (like above)
-
by displaying dubious data
-
by displaying insufficient data
-
by concealing or confusing uncertainty
-
by suggesting misleading patterns
Each of these five ways of lying with charts are all around us everyday. It is helpful to realize that the drivers for and how charts created in these five ways all have similar outputs (i.e., a misleading chart) but different outcomes (e.g., the impact of who, how, where, and when are mislead by them). Cairo walks the reader through multiple examples of each way to lie with charts. In addition for each, Cairo provides theory behind why each is problematic and tips to make sure you can spot such a chart as well as tips to avoid making such mistakes yourself.
How Charts Lie: Getting Smarter about Visual Information (2019) should be a must read for all of us, but can be especially helpful for those of us teaching with data in our classrooms.
HOW DOES THIS CONNECT TO WHAT WE DO IN OUR CLASSROOMS?
Let’s take each way that charts lie in turn and think through what it could mean or how we could tackle it in our teaching with data (regardless of what subject area or grade level we teach)…
First, charts lie by being designed poorly. Helping students understand not only how to make a chart, but also why to make the decisions they are making to display the data towards their claim is important. This builds their skills in making a well designed chart. In addition, this helps them learn that all charts are the product of decisions made by the designer and that it is important to think of charts as human-made rather than infallible. Looking at examples of poorly designed charts or comparing two differently designed charts of the same data can be a productive way to practice this skill.
Second, charts lie by displaying dubious data. This goes to the old adage of “garbage in, garbage out.” No matter how aesthetically pleasing a chart may be, if the data are bad (aka inaccurate, faulty, limited, biased, etc.) then the chart will be bad and lie. We can help our students build their skills of avoiding this pitfall by supporting them to consider what the limitations are of the data they have in terms of what conclusions they can and cannot draw from the available data. Another critical aspect of this is knowing the source of the data. Students should learn to always look for and include the data source (even when they are the ones who collected the data). If they are working with a chart someone else created and it does not list the data source, how can they know whether to trust the data going into the chart? We want to strive for our students to ask that question themselves.
Third, charts lie by displaying insufficient data. Similar to the previous way that charts lie, if you “cherry-pick” the data that you include in a chart to better communicate a specific (often pre-determined) conclusion that is lying. This is a lie of omission, something many of us are taught early on not to do in our personal lives, so why would it be ok to do it in our charts? The difference between this and dubious data can sometimes be tricky for students to wrap their heads around. I often find the easiest way to explore this is to use a relatively large dataset and specifically remove or include different subsets of the data that alter the overall trend and takeaway message from the data. This helps illuminate the misleading nature of cherry-picking (and can reinforce the concept of data as a sample, an extra bonus).
Fourth, charts lie by concealing or confusing uncertainty. All data are information from a sample of the total population of instances of what we are measuring. The better the methodology for accurately choosing the sample and the better the protocols are followed for precisely collecting the data, the more likely that the sample accurately and precisely represents the total population. However, because it is always just a sample (aka subset of the whole) there will always be uncertainty in the data. While we exist in a world of uncertainty, we humans like to pretend it does not exist. In charts this often results in not plotting or communicating the uncertainty in the data. For younger students, the key here is knowing that uncertainty exists in data and looking at all of their data before they collapse it into a bar chart (for example ;)). For older students, they can move beyond that to consider how to visually represent the uncertainty in their data. (Looking for ideas of how? Consider joining the “Use Statistical Thinking with & Evidence from Data” Data Literacy Series workshop to explore classroom strategies.)
Finally, charts lie by suggesting misleading patterns. Being able to spot this and avoid it yourself comes with practice and time. The key here is to stop and think about what underlying mechanism could be at play within the phenomenon or system that leads you to think the pattern you observe in the data could be occurring. Helping students stop to process what the pattern means to the real-world and whether that makes sense is HUGE. It helps them again be active decision makers in the process of working with data and reminds them that just because a chart shows a pattern doesn’t mean that the pattern actually explains something meaningfully in the real world. Often I use real world examples to help my students talk through this (e.g., ice cream sales and crime rates are positively correlated in the summer, but not casually related). Prompt your students to not only state what the pattern they observe in the chart is but why that does or does not make sense for what they know about the phenomenon or system.
WHAT NEXT?
I highly encourage you to read Alberto Cairo’s book How Charts Lie: Getting Smarter about Visual Information (2019). Even if you do not think it is relevant to your teaching, it is worth reading as a citizen of this data-driven world.
If you teach older students (high school and above), consider having your students read excerpts or the whole book.
Remember, helping students build their data and graphical literacy takes time and practice. Cairo’s book reminds us of the many ways that charts lie and provides a roadmap for how to think about those different ways charts lie to better set ourselves up for success with all these charts.
