Napoleon's Influence on the Modern Data Stack : Hyperdimensional Analysis with Malloy
Data visualization is a passion of mine. I remember reading Edward Tufte’s book?The Visual Display of Quantitative Information?& stumbling across Charles Minard’s “Napoleon’s March.”
Most visualizations plot 2 dimensions. Napoleon’s March encodes 6 : the geography of the terrain, the route & the direction of the army, the headcount of the troops, the temperature of the battlefield, & the time of year of Napoleon’s doomed quest to conquer Russia. Click to enlarge.
It’s a hyperdimensional work of art.
When Looker founder Lloyd Tabb showed me his new project?Malloy?I thought immediately back to Napoleon’s March.
Malloy makes hyperdimensional data analysis straightforward. It’s a new type of semantic layer. An example helps explain Malloy’s advances.
领英推荐
This is?automobile recall data in the US?using?Malloy. Click to enlarge.
There are 7 dimensions shown here: the car maker, the historical recall total, the percent of recalls, recalls by year, recalls by type, the most recent recalls, & the biggest recalls by number of cars.
Within each cell, you’ll find different analyses. The by_type table subgroups by recall type. Meanwhile, the by_year_time_chart is a time series of recalls over the last 50 years. You can click on any data point to drill into it.
That’s what hyperdimensional means. Showing data by different axes: time, type, manufacturer, & establishing relationships across them.
Minard must have toiled for weeks to mosaic his magnum opus. Data analysts today could coalesce a dashboard similar to the one above but it would still take hours or days.
The auto recall visualization above requires?38 lines of Malloy code?which encode the layout, the graphics, & the aggregations. Malloy abstracts away the complexity.
We’re in the Decade of Data. The Modern Data Stack has created many powerful abstractions to enable more insightful data analysis.?The semantic layer?is an important component of that progress. That’s where Malloy fits in.
Had Napoleon lived today?his famous quote about readers might have read:?“Show me a family of [data analysts], and I will show you the people who move the world.”
Full Stack Growth Hacker & Fractional CMO. Prev @Postmates @Posh @Capture @Grooveshark
2 年The truth lies in the charts.
Obsessed by Customer Success - Building better outcomes with Caffeine, Advocacy and Customer Centricity
2 年Minard's chart is really the foundation of modern data representation. I have it on the wall of my home office as a daily reminder that Charts should NOT be simple. Over-simplification is a curse of modern analysis tools. Any data you can show in a 1 or 2 dimensional chart is probably blindingly obvious; the power and value of graphical representation is to show many dimensions at once. The human brain can easily process n-dimensional relationships; Minard shows 6, but he was limited to pen and ink. I'm so excited that modern tools are finally helping us break out of pie/bar/line charts and let us see the real complexity of our data in ways that humans can see, but computers struggle (until now) to represent...
Rejuvenating, Advising, Growing | ex-Snowflake, ex-RelationalAI, ex-Adobe, ex-MathWorks, ...
2 年Tomasz Tunguz, I attended a Tufte workshop in Palo Alto many years ago, and I have a copy of that Napolean's March poster, too. I was working on UIs at the time, and one thing Tufte said about visualizations that stuck with me was "no extra ink on the paper". Every pixel should carry information. Translated to UI's: no decorative, colorful, wiggling elements that don't convey information by their presence or action.
CFO | Grip on SaaS
2 年Jan Sparnaaij