The Case Against Personalized Analytics
When the algorithm serves up hyper-personalized data, reality gets fuzzy fast.
AI personalization of data is a seductive idea - who doesn’t want a perfectly tailored stream of just the insights they want in just the right format at just the right time? We now spend so much of our lives consuming algorithmically curated content that the promise seems obvious. Many vendors are building in this direction; Tableau pulse, for example. Gartner even has an ‘Analytics Catalog’ criteria on the BI Magic Quadrant which they position as ‘Netflix for Data.’ I had a conversation with a well known data executive who made this very point - ‘I don’t want to look at a dashboard anymore, that’s done. I want to see the data and ask the questions that matter to me specifically.’ And I get it, who doesn’t?
The promise of AI curated analytics is tremendous but it was frankly too difficult to build without a Netflix scale payoff. Now it seems within reach for data tools, so it’s time to ask - is this really a good idea?
MotherDuck is on a tear: Dives put live data apps right in your warehouse, and now Flights add agent native data pipelines with dedicated compute. DW, ETL, and BI in one simple, fast stack. I use it every day to run Super Data Brothers and I’m proud to have them as a sponsor.
What is Business Intelligence for?
BI has two purposes within an organization, and we often only think about the first; to distribute data to individuals and groups to facilitate decision making. The second purpose is less recognized but equally important; to establish a shared data vocabulary and culture for coordination, action and outcome tracking across the company. It’s the difference between what I need to know to do my job, and what we must collectively know to decide which jobs are worth doing and how well we’re doing them.
Enterprises may exist in the free market, but they are structured as enclosed, command economies where decisions flow from the top. In this environment, simply distributing data is not enough, it must be purposefully designed to reduce ambiguity, make data easy to understand and facilitate both decision making and action.
BI does this by producing a library of reports that people rely on to facilitate data informed conversations about the business. But it also does it within reports. Data visualization crystallizes, contextualizes, and communicates meaning at a point in time, and it’s a science as much as an art. Elements like visualization type and color encoding aren’t purely stylistic; they’re grounded in pre-attentive processing, the part of the visual system that registers meaning before you’ve had time to think. A line chart intuitively encodes change over time because of how that cognitive machinery works. And that’s exactly why shared form creates shared understanding: everyone processes the same visualization and arrives at the same read before a word is spoken. It’s how charts builds common ground that numbers or narratives can’t.
The Agent, The Feed And The Dashboard
Social media may have many virtues, but fostering a shared and stable reality across groups of people is not among them. Turning analytics into TikTok gets data to users faster and in a more targeted way, but it runs a real risk of fracturing data coherence across the organization. This is the future algorithmic BI offers.
First there’s the agent. Ask a question, AI goes off and fetches an answer, usually via chat. Chat agents are the first true self-service analytics tool that more than 50% of the target audience for BI can actually use, and that’s great. They do create some risk of data fragmentation when users stop relying on shared reports and instead just pump questions into the AI aether, but the payoff in terms of saved time and increased data access is clear. And most importantly, these self-service agents actually require a high degree of human… well, agency, to be of value. You need to be quite engaged with data to begin with to bother asking a question.
Data feeds are another story. Many analytics vendors are developing capabilities in which algorithms evaluate changing data conditions, personalize visualizations and send digests and alerts with no human agency required. This seems like the obvious next step, why not automate analysis and feed people answers before they even know they need them. Turning business people into passive consumers of data will necessarily erode their critical thinking skills, just as turning people into passive consumers of media has badly wounded our media savvy. But the greater risk is the data fragmentation.
If the algorithm is judging my engagement stats to deliver a curated data feed with the charts I like and the language I prefer, and you get the same, can we really have a productive conversation about the data, even if we looked at the exact same metrics?
This is where the much maligned dashboard actually excels. Standardized views of well understood metrics shared across the organization. We all see the same bar charts, we all know what they mean, we can quickly evaluate them and move on to what actually matters in data - using the numbers to deliver some kind of outcome.
Metric Layers Are Not Enough
You may still be thinking, well so what? Everyone recognizes the danger of mismatched metrics is amplified with AI, and we’ve got semantic layers and context graphs and whatever term will sprout up next year to cover the same basic concept and really, that’s good enough. A personal data feed sounds kinda cool.
And here’s the thing - it IS cool, and it will be very powerful if we do it right. A balance must be struck; personalization of data within an established metrics AND visualization framework that ensures the most important data points are not only accurate, but clearly and consistently communicated across the organization. There are many ways you could do this but the simplest is just add reporting and visualization data to your context. Three layers where you should try things
First, in the metrics definitions themselves. If you’re storing context for what ‘net sales’ means, how to calculate it, its lineage and it’s definition owner, drop the instructions for how to best visualize right there in the context. Any AI that reads the metric info will pick visualization rules up and render it consistently across users.
Next, create separate visualization and report context stores. Many visualizations combine multiple metrics and attributes in novel ways. Capture them as context, index them and let AI find them. There are lots of visualization libraries that let you do this today, or just roll your own simple markdown definition. When someone requests ‘net sales for the west region excluding camping products where average price per transaction was over $1,337’ you may want a standard way to view that data, not just represent it in SQL. So standardize it.
Third, capture the BI visual design elements as context. There are many ways to render a drop-down list; single or multi-select? Auto submit or a submit button? How many values before you put a vertical scroll bar on it? AI will freestyle these small design decisions every time it builds, which significantly increases the cognitive load of using your analytics because users always have to think about them. Thinking about anything but the data elements themselves hurts understanding.
All of this sounds complicated and at the enterprise level it is, but it’s not complicated for you to start today, now, by just getting these elements into markdown and making them available on smaller scale in Claude, ChatGPT or Gemini. At the very least, use these techniques to make the AI coded analytics you build consistent for your users, so they receive the same experience across reports and departments and focus brainpower on using the data, not debating about it.
In the days of the newspaper everyone read the same headline and the same article and argued from a shared understanding of the facts. Now, nobody can agree on what the facts even are. Don’t let this happen to your analytics program.