Hey Everyone! I can’t believe September is just a blink away. This week’s issue is inspired by a funny mishap I came across at work some weeks back. In a nutshell, I was building an AI insights summary section into a daily reporting pipeline. The summary pulled the previous day's production data, and the model wrote a short narrative at the top of the report.
On one of the early runs it produced a sentence flagging availability at 84% as an area of concern. Except 84% availability on that line was a good day. Not a problem to investigate. A result anyone would have been happy about.
The model was not hallucinating or making stuff up. It read the number correctly and wrote plain English, but it applied a rule nobody had given it: low percentage means bad. What it did not know is that availability in this context is scored against a target, and 84% cleared it. The number was right and the sentence was fluent, it’s just the meaning was backwards.
The fix was not a bigger model or a longer prompt telling it to be careful. It was pointing it at the quantity that actually carries meaning in that context, which is time lost and efficiency impact, not the raw value. That funny little bug did more to shape how I use AI in analytics than any tutorial or tool review.
Syntax is checkable. Meaning is not.
Here is the split I have landed on. AI earns its keep on code, because it tells me immediately when something is wrong. I ask for a Python transform, a SQL query, or a DAX measure, and I get something I can test in seconds. It runs or it errors out (typically). Bad output costs me maybe a couple minutes of revisions or fine tuning.
Interpretation has no such loop. When a model tells me what a number means, nothing turns red. The output is confident, well written, and completely plausible, and then it lands in a report a director reads on Monday morning. There is no error message for "this may or may not be technically accurate". If not double checked, that oversight is what can create a problem.
Three places I actually let AI drive
Translating logic between dialects. Reading a DAX measure and getting the equivalent SQL shape, or the other way around, saves me time I would otherwise spend re-deriving it.
First pass on boilerplate transforms. Reshaping, joins, date handling, the parts that are tedious rather than hard.
Explaining a query I did not write before I modify it. Especially useful for inherited code where the original author is long gone.
What to hand a model before you let it interpret anything
If you are going to have AI write commentary on your metrics, give it the context it cannot infer from the number itself:
The direction of good, and the target it is being scored against
The unit and the grain
The business consequence, meaning what it actually costs when the metric moves
What normal looks like, whether that is last period, a benchmark, or a threshold
What the reader is supposed to do about it
Nearly every confident wrong sentence I have gotten out of a model traces back to one of those five being missing.
🕹️ Trivia
Excel has had the same maximum row count since the 2007 release. What is it?
A. 65,536
B. 999,999
C. 1,048,576
D. Unlimited, memory permitting
Answer at the bottom of this issue
Interesting Reads (TL;DR)
Text-to-SQL: Comparison of LLM Accuracy by Ekrem Sarı
Thirty-six models run against 759 BIRD-SQL questions, with every query actually executed rather than judged. Worth reading for the uncomfortable side note that a large share of the benchmark's own gold answers turned out to be wrong. Read more →
Semantic Layer vs. Text-to-SQL: 2026 Benchmark Update by Jason Ganz
Klarna replaced ~700 support roles with an AI assistant it claimed handled 75% of chats, then started rehiring humans after quality dropped. The CEO's own framing is the tell: not a retreat from AI, but an admission that cost-first replacement produced worse service. Read more →
Will AI Replace Data Analysts? Here's What's Actually Happening by Naveen Renji
A grounded look at which specific tasks are compressing. Ad hoc query iteration going from thirty minutes to five is believable. The analyst disappearing, not so much. Read more →
Resources & Tools
DuckDB #data-visualization #productivity
An in-process analytical database you can run from a notebook or the command line with zero setup. Worth including because it's the kind of tool that rewards understanding why columnar analytics is fast rather than just memorizing syntax. Free and open source.
Evidence #data-visualization #productivity
A framework for building BI reports as code (SQL + Markdown), version-controlled like anything else. A good example of the "judgment travels, tooling changes" thesis in practice: the report logic outlives the dashboard tool. Open source.
This Week’s Quick Study
▶️ Where to Start as a New AI Governance Professional? by BABL AI Inc. (15 mins)
Interested in governance? Shay Brown, CEO of Babl AI, argues that governance starts with an inventory. Every AI system in the building, including the ones already embedded in software you pay for. Triage what you find by risk, route the high-risk cases through one central review that documents risks, controls, and compliance, and let ISO 42001 shape the process as you go. Start rough and iterate.
CLASSIFIEDS
FROM THE EDITOR
Summarize and understand anything on the web.
Turn any screen into instant insight, VizBuddy captures your browser and hands you a structured analyst-grade summary in seconds (Chrome only).
FROM THE EDITOR
Free Notion templates built for data professionals.
Trusted by 1,000+ users, download the templates designed to keep your goals, projects, and ideas in perfect sync.
🕹️ Answer
Excel has had the same maximum row count since the 2007 release. What is it?
A. 65,536
B. 999,999
C. 1,048,576 ✅
D. Unlimited, memory permitting
Which is 2 to the 20th, up from 2 to the 16th in the versions before it. If you are anywhere near the ceiling, the row limit is not your real problem.
How was this week’s issue?
Newsletter publishing is hard work and it’s just me running the show here. If you ever feel like extending a thanks, idea, or insult you can do that here.
Or email me directly at → [email protected]
If you’re feeling generous and want to toss a gift in my PO Box, I also keep an active wishlist of items or books I’m interested in.
