Critique by Design

September, 2021

I wasn't specifically on the lookout for a music-related visualization, but when I stumbled upon one, I wasn't complaining! Pulled from a study published by Music Business Association in 2018, the visualization describes preferred music genres of 3000 U.S. consumers by age group. I was excited to work with the visualization because—as a drummer who enjoys playing several music genres—its data felt immediately interesting. More importantly though, I was able to quickly identify areas of the viz. that I liked and disliked. This helped me build a strong Effectiveness Profile (developed by Stephen Few), which guided my wireframing process and my eventual redesign.

The OG Viz.

This is the original visualization that I chose to work on:

Critiquing Cogently

After a brief study of the visualization, I recorded my general thoughts about it:

"At first glance, the visualization looks interesting.

The colors are attractive. I especially like how the yellow and purple look towards the top of each stack. The color wheel and its complementary colors attest to why this may have been appealing. Beyond the noticeable scatter of colors, I find myself being drawn to the chart because I’m a music enthusiast. I am in their target audience, and so I naturally want to learn more about how genres are perceived by different age groups. However, this may not be the case for everyone.

Within seconds, there is a growing sense of feeling lost. While the visualization isn't jarring or overtly bothersome, it gives little consideration to aesthetic appeal. The colors, percentages and legend begin to swim around with time. I want to understand what the numbers mean but, without focused searching, this feels hard. I feel like a visualization should make it easy for viewers to grasp data, not confusing. As a result, the information is interesting only for a couple of seconds. It leaves no lasting impression. I don’t feel like I learned something extraordinary. The information doesn’t move me. I don’t find new purpose after glancing at the chart. I don’t feel compelled to share this information with friends.

There are, however, things that I do like. I like that the visualization is credible and accurate. The study was conducted by Audiencenet and published by Music Business Association. It surveys favorite music genres of 3000 consumers in the United States as of July 2018, sorted by age group. This data is explained well in their Excel file, which also provides references and verified research methods. This offers pretty good context, though the visualization doesn’t rely on it to be understandable – which is great. The charts don’t show inherently complex information. While this information is presented in a more complex way than needed, at least no element feels inscrutable. I also like that the goals of the visualization are clear and that the graph conveys what it aims to. The visualization feels complete (aside from the fact that it does not include an “Other” response, raising the question: is every respondent’s answer represented or were some left out?)

If I could change a few things, I would begin with the colors. They bring out the notorious rainbow effect that we’ve frequently discussed in class. The percentages on the y-axis are also a little confusing; why do the numbers go beyond 300%? That takes a little more thinking, since it isn’t instantly intuitive. A final observation is that the comparisons between the age groups are unevenly distributed, so I would consider normalizing those as well."

Then, I used Stephen Few's "Data Visualization Effectiveness Profile" and ranked different attributes of the visualization on a scale of 10:

Usefulness. Is it useful for the intended audience? Does it communicate valuable information?
Rank given: 7

Completeness. Does the visualization have everything necessary to make it understandable?
Rank given: 9

Perceptibility. Can the reader understand the information with minimal effort? Is the visualization type appropriate? Does it use illogical comparisons?
Rank given: 5

Truthfulness. Is the visualization accurate, reliable and valid? Is it representing what it says it is, and in the most complete and truthful manner? Does it misrepresent the data or make comparisions that aren't correct?
Rank given: 10

Intuitiveness. Is it easy to understand and clearly communicates the information? If unfamiliar, does it include easy to understand instructions on how to interpret it?
Rank given: 5

Aesthetics. It is interesting / enjoyable to look at? Is it a good example of what a beautiful data visualization might look like? Is it somewhere in the middle - pleasing but otherwise not distracting to look at?
Rank given: 3

Engagement. Does it lead the audience to learn more about the topic? Does it inspire the audience to talk about the data or share it with others?
Rank given: 6

Working the Wireframes

Based on my observations and some of the high-level problems I identified through the critique, I began to brainstorm ideas for my redesign. I approached the problem-solving process using the Six Thinking Hats technique devised by Dr. Edward de Bono, and developed the following strategy:

Next, I considered simplifying the original dataset a little bit. I had noticed that the age groups were unevenly distributed, so I merged two age groups (16-19 and 20-24) to produce a new group. I also merged the genres "Rock" and "Classic Rock" together, since I noticed that at least 4 music genres had the word "Rock" in them.

My first wireframe included two pie chart iterations. The first set of pie charts allows viewers to observe the age distribution within each music genre, while the second set of pie charts allows viewers to observe the most popular music genres within a certain age group. I decided I liked the second set more because it presents a story that I would enjoy learning about more. (That is, I'd rather more easily learn "which genres each age group is listening to the most" as opposed to "which age groups listen to a particular genre the most").

However, I quickly realized that the pie charts were not intuitive. The colors felt crowded, and there was no way to compare the "absolute" numbers of listeners by age group. In addition, in both iterations, I faced some difficulty coming up with distinct colors to separate each pie slice.

After considering the problem for another day, I decided to simplify everything and go for a stacked bar chart. This kind of visualization would help me work with a limited number of colors, and let me show numbers as well as proportions. It would also let me reorganize the data to flow from the most popular genre to the least popular genre, which could elevate my visualization's aesthetic score.

...luckily enough, I went to bed satisfied with my wireframe!

Open to Opinions

Next, I tested my wireframe by sharing it with 3 individuals and asking for their feedback. The individuals were not randomly-picked, but their attributes (age, gender, occupation) varied enough that they represented a randomized sample population. I asked the following questions:

What do you see? Can you explain what's going on?
Can you describe how this visualization makes you feel?
How comfortable are you reading this visualization?
Who do you think is the intended audience for this?
Is there anything you find surprising or confusing?
Is there anything you would change or do differently?

Some of the responses I received are captured in this brain-dump:

Some feedback, interpretations and suggestions that were repeated more than once were:

Ordering of bars works well
Frequent tick marks not required- even if they were present, only a few would read them
Title is a little windy (too many parts to keep track of: genres, age groups, number of votes, 2018, the U.S.)
The use of conventional, identifiable colors adds to ease of reading
Rock music is popular among all age groups
Rock music should be highlighted
Ages 16-24 listen to the most music overall
Instrumental music is the least liked genre
Could consider flipping axes because vertical data feels more familiar
Legend should be made more prominent since age is very important to this visualization
The audience is likely to include producers, music creators, music lovers, investors and music bloggers.

The Redesign

Equipped with my new wireframe and suggestions (that proved invaluable), I built my redesign using Flourish. I tried to pick colors that wouldn't give an overwhelming "Rainbow Effect", yet still be differentiable enough to attribute to 6 different age groups. I ended up selecting a gradient that deepens with age (yellow to dark purple), and colored the dominant age group within each genre in red. Funnily, this happened to be the youngest group (individuals aged 16-24) in every genre! The new graph is below and can be interacted with.

Then and Now

The final side-by-side:

Appendix: Critiquing the Critique

I really enjoyed Stephen Few's evaluation method. It went beyond the Good Charts method in helping me consider elements I wouldn’t have paid attention to otherwise. Completeness and usefulness are two such elements that I wouldn’t naturally have articulated in my critiques. I also liked that Few's scale ranges from 1-10, and not from 1-5. This pushed me to think more critically and perceptively about rankings. Finally, I appreciate that the questions in this critique method were simple, well-explained and interesting.

Something I would add to this evaluation method is “Emotional Appeal.” This is closely related to “Engagement,” but is more spontaneous. For example, some visualizations are instantly pleasing while some can feel instantly frustrating. I think these immediate, visceral reactions are useful to record because they capture first impressions… and, often, first impressions last long!

I might even add a “How Much Time Did It Take For You To Fully Understand The Visualization?” metric. I think people tend to record higher scores under “Intuitiveness” (which captures how easy a visualization is to understand) because it gives them either a greater sense of accomplishment or greater sense of forgiveness. That is, I feel like we are prone to remarking that a graph is easier to understand than it actually is because of an unconscious bias within us to demonstrate we’re: (i) smart or (ii) forgiving of smaller flaws as long as the overall message is understood.

To provide an example, if I were asked whether a graph was intuitive or not, I would likely say yes unless there was a strong reason to convince me otherwise. Even if this binary question were expanded to a scale from 1 to 10, I would still lean towards numbers beyond 5 because going below that would be equivalent to saying, “no.” So, unless convinced otherwise, I would probably rank a graph above average (>5) on intuitiveness. To solve this problem, it might help to reframe the question to, “how much time did it take you to fully understand this?” I think we’d get more honest responses because we’re forced to come up with a number of our own and aren’t settling for a pre-written option.