Alternatives to Word Clouds

I'm aware that many listeners first found Data Skeptic due to the bold appeal I made in the first episode of 2016 where I proposed we kill the word cloud. I feel just as strongly today that the word cloud is a useless and ill-informed data visualization. I'm repeatedly grateful to see people on social media mention they're trying a different approach, and in a few cases, appologizing in advance for publishing a word cloud due to peer pressure. Stay strong, friends! Together, we can end worthlessly uninformative data visualizations.

What I didn't realize was how much SEO benefit I would get from this episode. Indeed, looking back at traffic for 2016, I find that "alternatives to word clouds" is one of my top 100 search queries. In the original episode, I suggested a basic bar graph (I called it a histogram, but it's actually just a bar graph) as a viable alternative. I still think this is the best suggestion for a more useful way of describe ngram frequencies.

Yet, it seems there is a desire from the general public to do something special with text that goes beyond the types of visualizations that have empirical backing. There seems to be a need for a bit of glitz and showmanship. Perhaps the path to exterminating my most hated data visulaization should focus not just on viable alternatives to textual summarization but also consider how to add a bit of "sex appeal" to them.

With that in mind, allow me to propose interactivity as the best way to add just enough intrigue to data presentation without resorting to click bait or nonsensical representations.

I still believe a frequency histogram is orders of magnitude superior to a word cloud, despite it having a lack luster rendering at a glance. If you're tempted to turn to the dark side because a colage of arbitrary words makes people node as though they've figured something out, consider taking that same underlying dataset and presenting it with some iteractive component instead.

What if, instead of spending your time rendering a word cloud, you focused on providing user interfaces that allowed some slicing and dicing of your dataset. I believe the core appeal of a word cloud is the illusion of insight. I assert that providing actual analytical tools that enable some interactive subsetting of datasets not only satisfies the need for some lure to get readers/viewers to take interest, but it also has the opportunity to enable discovery of actionable insights from the underlying dataset.

I'm seeing an emergence of commodity API services emerge which will spell check, sentiment score, topic extract, and provide any number of implied features from textual data. While many of these APIs fail to impress me, their ubiquity is hard to deny. The commercial world is definitely headed in the direction of delegating feature extraction from raw text to third party services. This can be dangerous when the providers of those services invest more in sales and marketing than they do in actual engineering. Yet, the message I'm seeing is that problems previously considered very difficult (like sentiment) are converging on specialized commodity cloud service offerings.

With a wide range of analysis APIs available, surely any data scientist worth their paycheck can gather a set of implied or derived features and come up with some interesting visual, aggregated summary of their underlying corpus. We always need to keep in mind that the objective is to uncover and expose some underlying wisdom derived directly from the datset. How much an art professor likes the assembage should not be at the forefront of our concerns.

If anyone has been in a situation where they could have done a word cloud but found a more useful alternative, please let met know. I'm eager to share approaches. While its true there has been a measurable decline in word cloud usage in 2016, my work is not yet done!