The Show Must Go On
By Toria · Published July 2, 2026
Data Explorer · Systems Thinker · Writer in Progress
Photo by Egor Kunovsky from Pexels
Rehearsals
Would AI have taken off if, instead of confident wrong answers, it had simply said "I don't know"? We never got to find out, because nobody waited for the answer.
In my line of work, before anything gets handed over, it gets tested. Properly tested, the kind where you try to break it before anyone else can. Usually, you start with something called an MVP, a Minimum Viable Product, a version that does the basics and is expected to be rough around the edges. It gets reiterated, sanded down, improved, until eventually it's ready for an audience.
Imagine you’re directing a play. The script is brilliant. The actors know every line. Rehearsals go well. Then, just before opening night, someone points out the scenery still isn’t finished. There’s no time left, so you hang up a few dust sheets and hope the audience is paying more attention to the performance than the backdrop.
That’s obviously not what happened with AI, I’ve exaggerated it slightly for dramatic effect, but it’s the closest analogy I’ve found for how this feels.
AI was tested, that's not in question. What's in question is the decision that followed.
Hallucination must have shown up in testing, What I find interesting is that it doesn’t seem to have been treated as a release blocker. Somewhere along the line, it seems this became a limitation users would have to live with.
Programs
This wasn't an isolated mistake. In 2023, a New York law firm made headlines after submitting a legal brief built on case citations that didn't exist; they'd taken the AI's output without independently verifying it. Around the same time, Deloitte was paid a substantial sum to produce a welfare compliance report for the Australian government, and later had to refund it after the report was found to contain fabricated quotes, invented citations, and references to books that didn't exist.
Two very different organisations, in two very different fields, tripped by the exact same flaw, a flaw that was known going in.
Props
If this had been any other product category, that decision likely wouldn't have made it past the people signing off on launch.
We’re currently migrating HR and Finance data where I work. I sometimes wonder what would happen if I pitched hallucinating salaries as some exciting new feature. A bit like a roulette wheel, you just get whatever amount it lands on that month. Some people might still get paid correctly, but most wouldn’t. I have a feeling nobody would take that pitch seriously.
But somehow, with AI, that same call was made and it stuck. Over time, it feels like we adapted around the limitation instead of expecting it to disappear and disclaimers got added.
That’s where it gets interesting for me. Disclaimers usually appear because we’ve learned something new. Smoking packets didn’t always carry health warnings; they appeared as the evidence became overwhelming. AI feels slightly different. Hallucination wasn’t something we discovered years after release. It was already understood to be part of how these models worked.
Which makes me wonder whether the disclaimer is describing an unexpected risk, or simply reminding us of a compromise that was always there.
Photo by Catherine Franken from Pexels
Opening Night
The trouble is, the show's already running, and you can’t just pause whilst you make adjustments – this isn’t TV. So the law has stepped into play catch up. A bit like halting a production for safety precautions due to the materials of the dust sheets (you’re going to have to work with me here). The EU AI Act is one example of regulation catching up with a technology that reached the public long before society had really agreed how it should be used.
There's also a growing push toward "responsible AI". I’ve seen this phrase appear a lot now, but all it is, is the industry's way of addressing the black box problem. With most flawed products, you can find the broken part and fix it directly.
One of the ways the industry is trying to address this is through explainability. Rather than simply giving you an answer, newer systems increasingly try to show how they arrived there. In theatre terms, it’s a little like adding a narrator to explain the missing scene instead of rebuilding the set. It helps the audience follow the story, but it doesn’t change the fact the scene was never there in the first place.
The Villain
This, I think, is the real cost of that decision. Testing can only take you so far. Developers can test the scenarios they expect, but they can’t predict every way millions of people will eventually use a product. Somewhere, a lawyer will rely on it for case law. Somewhere else, someone will use it to summarise medical advice or draft a government report. Those aren’t failures of imagination, they’re the reality of releasing software into the world.
Encore
Maybe there wasn’t another option. Perhaps waiting until hallucinations were largely solved would have meant delaying AI by years. I can understand why companies made that trade-off. What I’m less sure about is whether the public really understood that trade-off too.
None of this means AI isn’t useful. It clearly is. I use it myself, and I don’t think it’s going anywhere. What I’m still trying to work out is whether we accidentally blurred the line between “useful” and “ready”. Hallucination wasn’t hidden. It wasn’t discovered years later. It was a known limitation from the beginning. Maybe shipping anyway was the right decision. Maybe it was the only realistic decision. What I’m less certain about is whether we ever really stopped to think about what that meant. We don’t get a version-two opening night. The audience is already in their seats.
So I still come back to the question I started with.
Would AI have taken off if, instead of sounding certain, it had simply said, “I don’t know”?
Written by Toria
Data Constellations — For layered thinkers, quiet disruptors, and curious minds.
Keep exploring the patterns others don’t always see.