When good data break bad: Data visualization and eye movements
Thursday, June 23, 2016
Data is everywhere. Political campaigns, sports teams, and even music streaming sites rely on the collection and analysis of data to win, or to attain customers, and to sell targeted advertisements. Journalists use data to report the news and the public interprets data in consuming that news. Becoming data literate is no longer just a requirement of the scientist or the stock market expert, but of the average citizen, in professional and personal life.
Analyzing and summarizing data is difficult enough, but communicating findings effectively involves abstracting trends and highlighting similarities or differences. Often, communicating about data is done in a visual form—displays are created which represent characteristics of the data in visualizations.
Readers who followed our #symbodiment event will recognize that designing a visualization abstracts the numerical data into a perceptual representation that depicts a concept. When the trends in the data align with their perceptual representations, data visualizations can be powerfully and intuitively revealing.
However, data visualization can easily go awry when the perceptual mapping is misleading or arbitrary. Take a look at the graph below about the growth of Chinese drones (from the addictive-but-terrible viz.wtf).
Two issues with this graph make it difficult to interpret. First, the size of the bars does not correspond, proportionally, to the actual amounts ($500m is represented with eight dollar bills, $130m with five). This makes it difficult to compare the two bars to see the rate of growth. Second, the caption to the right draws attention to the right-most bar. In this case, it might be easy to correctly read the bar on the right first, but to do so, you have to overcome an automatic bias to read the graph from right to left.
By training students to properly read data visualizations, educators can ensure that such design mistakes do not result in misinterpretations of graphs. This was the aim of a recent study published in Psychonomic Bulletin and Review by Audrey Michal and colleagues. In email correspondence with me, Dr. Michal emphasized that correctly reading graphs is critical so that students "can make their own conclusions about the data and not just focus on whatever the designer is trying to emphasize."
To do so, Michal and colleagues asked college students and school-aged children (6- and 8-year-olds) to read a question based on a simple graph of fruit amounts, and then answer the question while viewing the graph. See below for a sample stimulus:
In addition to recording answers and response time, the researchers tracked where the students looked using an eyetracker. According to Dr. Michal, eyetracking allowed the researchers to investigate "the order in which people attend to graphs [which] determines how they interpret the graph."
Note that in the example above, the order of the words (blueberries, oranges) does not match the order in the graph (orange, blue). To answer the question, an astute graph reader should first attend to blueberries, then decide whether that bar is higher than the bar corresponding to oranges. College students had no trouble reading the graph this way, looking first significantly more often at the bar that was mentioned first. Children, on the other hand, were more likely to look at the left bar first, regardless of which one was mentioned first.
Intriguingly, for children, where the child looked predicted their reaction time for that trial. In the next figure below, each colored line represents the first saccade. A saccade is a rapid movement of the eye from one location to another. For children, the majority of all saccades occurred to the left bar, regardless of which was mentioned first (the "Target Bar"). Green lines are saccades for trials with fast response times; red lines are slow response times. Note that when children looked at the target bar first, they responded more quickly. The response times for 8-year-olds were much faster when they looked at the right bar first when the target bar was on the right (more green lines to the right, and more red lines to the right). This means that when children attended to the graphs in an order that made them more easily interpretable, the graph was interpreted more quickly.
Visualizations that take advantage of these perceptual biases should be easier to interpret, especially for young children. Positioning the text for a graph so that it can be read before the graph itself (thus cueing the viewer what to look for), and placing the first-mentioned data point on the left side of the graph are two suggestions from Dr. Michal and colleagues. Improving visualizations may help, but even good visualizations require skill, flexibility, and training.
In her email, Dr. Michal emphasized that "it can be difficult for designers to optimize visualizations so that every piece of information can be easily extracted. There are usually several comparisons or trends that could potentially be extracted from a graph... People need to be able to focus on whatever comparison is relevant for a current task or question, no matter how well-designed a graph is."
The increasing role of technology in modern life may make data visualizations even more readily interpretable. One way Michal and colleagues are taking advantage of technology is to craft dynamic visualizations: "In a new study, we are testing whether we can improve students' graph comprehension by spatiotemporally highlighting parts of a question with relevant information in the graph. For instance, when asked 'How many fewer oranges are there than blueberries?', it may help to simultaneously highlight the words 'fewer oranges.'"
Data visualization allows the complexity of the world to be boiled down to something that is simple, elegant, and possibly even beautiful. When done well, the results are (literally) eye catching. And if seeing is believing, then the results of studies like this one suggest that seeing accurately is believing accurately.
Reference for the article discussed in this post:
Michal, A. L., Uttal, D., Shah, P., & Franconeri, S.L. (2016). Visual routines for extracting magnitude relations.Psychonomic Bulletin and Review. DOI: 10.3758/s13423-016-1047-0.