The model was fed the company’s contradictions
Meta description: One bad input can poison an AI run when the organization has not explained who it is, what it wants, and why. More on his work: Miklos Roth online.
Miklos Roth
The pilot impressed everyone, and I have learned to ask why. The answer, in these stories, is in the data: the pilot ran on a curated extract — cleaned, deduplicated, reconciled by the project team over six careful weeks. Then the system went to production, where it met the record the business actually keeps: the same customer under three names, the field that means different things in different regions, the spreadsheet that overrides the system of record by convention. The model began producing confident nonsense. The team called it hallucination. It was not. The model was repeating the company.
[AWAITING MIKLOS — scene: a pilot on a curated extract and a production run on the record the business actually keeps]
What data did the pilot quietly exclude?
The exclusion is never malicious, which is why it recurs. The pilot team takes the clean subset — the customers with complete records, the products with consistent naming, the period without the migration anomaly. This is rational pilot design: prove the concept before fighting the mess. The rationality has a cost nobody books: the pilot’s results become the business case, and the business case assumes the extract. The production environment is the full record, contradictions included. The gap between the two was never measured, because nobody’s job included measuring it.
Why does “hallucination” become the name for a record problem?
The contradictions a model faithfully repeats are usually the ones the organization handed it. A request for “twenty percent better performance” gives a model many plausible directions; it does not identify what performance means or what the company is willing to change. Historical results, relevant data, operating constraints, and the purpose of the analysis need to be present. Otherwise, what looks like a failure of artificial intelligence may actually be a failure to specify the task. The model has produced an answer to the question it received, not necessarily to the question the business needed to ask.
The word hallucination is convenient, and I have learned to hear it as a diagnostic of the team’s framing, not the system’s failure. When the model outputs something wrong, “hallucination” assigns the fault to the model’s nature — an unfortunate property of the technology, to be mitigated by better models. In the cases I examine, the wrong output often traces to a real contradiction in the company’s own records: two systems disagree, and the model, having no authority to resolve them, stated one. The model was faithful. The record was forked. Naming it hallucination ends the investigation one step too early.
Where do contradictions in the business enter the context?
The contradictions enter everywhere the context is assembled. A retrieval system pulls documents that were written for different purposes, at different times, under different definitions — and presents them to the model as equally true. A field populated by one team’s convention meets a report built on another’s. The company’s internal disagreements, harmless when humans route around them with judgment, become the model’s raw material. Humans carry a map of which record to distrust. The model has no map, because nobody ever wrote the map down.
What pattern marks data collapse versus model error?
The distinction I use is reproducibility through the record. Model error is erratic — same input, different wrongness. Data collapse is systematic: the error repeats, traces to the same records, and disappears when the record is corrected. In the companies I visit, the error logs show the systematic kind far more often than the team’s language suggests. The team has a model problem in its vocabulary and a record problem in its logs, and the vocabulary determines the budget, so the record problem waits.
I see this as a form of insufficient self-knowledge, not merely a technical prompting mistake. Leaders may assume that their objective is obvious because it has been repeated in meetings. To a model, the unstated reasoning is not there. My foundation is the four questions: Who are we? What do we want? Why do we want it? How do we want to achieve it? They sound elementary precisely because companies often proceed without adequately answering them.
[AWAITING MIKLOS — scene: second scene]
What question locates the record that matters?
Data quality programs in these companies tend to be comprehensive: a governance framework, a maturity model, a multi-year cleanup. The comprehensiveness is why they start too late and end too rarely. The question that moves faster is selective — it asks which specific record the company’s decisions actually trust, and what state that record is in. The cleanup that matters is narrow. Finding it requires a question the comprehensive program was designed to defer.
The question to ask of any failed run before blaming the model: have we explained clearly enough who we are, what we are trying to change, why that change matters, and what would count as a useful answer?
PR excerpt: [PR — awaiting Miklos material; cut 80–120 words from his paragraphs once available.]
