Not a chatbot on your website. An agent that reads your data, answers citing its source, and hands over to a person when it does not know — with the numbers to tell whether it is worth it.
by Nadia Maar ·
In two years artificial intelligence went from curiosity to budget line. And with the budget came the pitches: a chatbot for your website, a promise that “AI reads your documents”, a demo that works beautifully as long as the questions are chosen by whoever is demoing. Most of these projects die between pilot and production, and they always die at the same point: when somebody asks the assistant a real question and gets an invented answer delivered with perfect confidence.
This article is not about models. It is about where an agent sits inside a business process without breaking anything: what it may read, what it must refuse to do, how it notices it does not know, and what it really costs — including the lines nobody puts in the quote. At the end are the three cases where, facing a client, I said AI was the wrong answer.
The demo works because it is a controlled conversation: chosen questions, clean data, nobody asking about the exception. Production is the opposite — and that is where three problems surface that no amount of prompt work solves.
When the conversation moves to which model to use — GPT, Claude, an open model — before it is clear which documents it answers over and who is allowed to see them. Model choice is the least important decision and the easiest to change later: it is swapped in an afternoon. The architecture of sources and permissions is not.
The idea holding it all up is simple: the model does not answer from memory, it answers over documents retrieved at the moment of the question. The practical difference is that it can cite where every sentence came from — so it can be verified — and when it finds nothing it can say so instead of improvising.
The console above is the part you see: the queue moving, exceptions opening, the agent answering at the bottom. In the public demo its answers follow a script — it does not have your data underneath, and a real agent without real data would prove nothing. What the demo honestly shows is the shape of the interaction: where the agent sits in the workflow and what happens around it.
Not all cases are equal. These four share three properties: high volume, cheap errors, and a person who checks before anything goes outside.
This section costs projects and brings better ones. These are real situations where AI was the wrong answer, and saying so early was worth more than any quote.
Undocumented process — What Marco does was not written down anywhere: it was judgement accumulated over eleven years. Automating a process nobody can describe means freezing a version of it invented by the supplier — and finding out six months later, when Marco corrects every output by hand.
What we did instead: Three weeks writing the process down with Marco, and the document immediately paid for itself training two people. Automation turned out to help on two of the nine steps, and those we built.
Not enough volume — Forty requests handled in five minutes each is just over three hours a month. No system pays back on three hours, and maintenance time — the evaluation set, the updates when a document changes — exceeds them on its own.
What we did instead: A structured form instead of a free-text inbox, plus two routing rules. Same problem solved for a tenth of the cost, and nothing left to maintain.
Errors too expensive — The right question was not whether the model is wrong one time in a hundred: it was what happens that one time. On technical parameters that end up in production, a wrong answer does not produce a complaint — it produces damage discovered downstream.
What we did instead: Semantic search over the documents, with sources shown to a person who decides. Half the saving, none of the consequences — and the technician now takes ten seconds instead of twenty minutes to find the right sheet.
Start where AI saves time for the people doing the work, not where it speaks on your behalf. The first case is measured in hours and fixed in an afternoon; the second is measured in complaints and fixed with an apology call.
A single, well-scoped use case runs between a few thousand and twenty or thirty thousand euros to build. That is the easy figure. The other three lines are the ones that decide whether the project lives:
| Who forgets it: | ||
|---|---|---|
| Building the use case | €5,000 – 30,000 | nobody — it is the only line in the quote |
| Data preparation | 20–40% of the project | everyone, and it is your work rather than the supplier's |
| Evaluation and maintenance | 1–2 days a month | everyone — and it is the line that kills pilots |
| Metered model cost | €50 – 400/month | almost everyone, until the first invoice |
Data preparation is almost always the surprise: deciding which copy of a document is the good one, removing expired versions, establishing who may see what. No supplier can do it for you, because it requires knowing how the company works. Skip it and you discover halfway through that the system answers beautifully over the wrong documents.
The right question is not “can we use AI”. It is: which process, how many passes a month, at what cost when it is wrong, and over which data we can actually let a system read. Four answers found in half an hour by looking at how you work.
The audit is free and it is not a presentation. If it turns out the volume does not justify the project, I will tell you — that is section four of this article, and it has happened three times. If the case is there, out comes an estimate of cost, payback and risk, with a first step small enough to be stopped.
Between €5,000 and €30,000 to build, plus two lines almost nobody puts in the quote: integration with the systems you already run, typically 20–40% of the project, and model usage, from €50 to €400 a month depending on volume. The figure that matters is the total, because that is what you compare against the time the agent actually gives back.
Because the demo runs on hand-picked cases and production arrives with the real ones: the crookedly scanned document, the ambiguous question, the missing field. An agent you can put into production states the source of every answer, admits when it does not know, and hands over to a person with the context already assembled. Without those three things the pilot stays impressive and unusable.
The ones where an error is cheap and the volume is high: routing requests, extracting data from documents, searching internal material nobody can read through any more, drafting a reply a person reviews. What they have in common is that the AI saves time for the people doing the work rather than speaking on your behalf — that comes later, once there is a measured track record.
When the rule is deterministic and a line of code does it better; when the data it should reason over is not in order, because the agent will return the same confusion with more confidence; and when the answer reaches the customer with no human filter and one error costs more than the automation saves. Those are the three cases where the honest answer is no.