Enterprise Generative AI: The Four Barriers Between Pilot and Production
Every enterprise leader has now seen a compelling generative AI demo. Far fewer have shipped one into a regulated workflow and kept it there. The gap between the two is four unglamorous problems: what data you're willing to expose, who's allowed to see the output, whether the answer is actually true, and what it costs to prepare the data in the first place.
The model stopped being the differentiator
General-purpose models from a handful of providers are converging on similar capability, and whichever one leads this quarter rarely holds that lead for long. Enterprises building generative-native applications, or wiring generative capability into the systems they already run, are learning that the model is increasingly a commodity layer sitting on top of something that isn't: their own data, and the engineering that makes it safe and useful to query.
That data is what turns a general-purpose model into something that actually knows your business — your contracts, your claims history, your product catalogue, your case files. And it's precisely because that data is valuable that connecting it to a generative AI system is harder than the demo makes it look.
Barrier 1: what you're willing to send outside the perimeter
The first question any risk-averse organisation asks is where its data goes. For firms handling client information — legal, financial services, insurance, healthcare — the data in question usually isn't the company's own; it belongs to a client, a patient, or a policyholder, and the obligations attached to it are stricter still.
Sending that data to an external model as part of a prompt means it has left your security perimeter, even briefly, and for many organisations that's a non-starter regardless of what the provider's terms of service promise. The two workable paths are to depersonalise the data before it ever reaches an external model, reserving external calls for genuinely general tasks, or to keep the sensitive workload on a smaller model running inside your own environment. A large general-purpose model is rarely the right tool for a narrow, repeatable, sensitive task in any case — a compact model fine-tuned on your own data will usually match or beat it on that task, at a fraction of the cost, without the data ever leaving home.
Barrier 2: who's allowed to see what, once everything is searchable
Assume the data problem above is solved and everything sensitive stays inside the perimeter. A second, quieter problem appears the moment you index it: your organisation almost certainly has more than one permission model running at once, and indexing collapses all of them into a single searchable surface.
A support engineer who can see customer tickets but not salary bands, and a junior analyst who can see public dashboards but not board papers, now query the same index. Left unchecked, the model doesn't know the difference — it will retrieve and confidently summarise whatever is most relevant, whether or not the person asking was ever entitled to see it. The fix isn't a filter bolted on after retrieval; by the time content reaches that stage, it has already entered the pipeline, and a single missed log line or quoted passage leaks it anyway. The access control has to sit inside the retrieval step itself, inheriting the roles and attributes your business already enforces, so a query only ever returns what that specific user is cleared to read. We've written in more depth about what that looks like in practice in our piece on permission-aware RAG.
Barrier 3: when a wrong answer reaches a board pack
Language models fabricate. Not occasionally by exception, but as a structural property of how they generate text — they will state something confidently and incorrectly, and grounding the model in your own data reduces how often that happens without ever removing it entirely.
For a first draft of marketing copy, an invented statistic is an embarrassment. For a number that flows into a compliance filing or a shareholder report, it's a liability. The only reliable defence is to treat truthfulness as something you engineer rather than hope for: cross-check every generated claim against the source passage it's supposedly drawn from before a person ever sees it, attach a citation to every material statement so it can be traced back to an internal document, and keep a human in the loop on anything consequential. None of that is exotic — it's closer to how a finance team already checks a junior analyst's first draft. The discipline just has to be built into the system rather than left to the reviewer's attention span.
Barrier 4: the unglamorous cost of unstructured data
By most industry estimates, somewhere around 80% of enterprise data is unstructured — PDFs, scanned forms, emails, transcripts, meeting notes. None of it is usable by a model in its raw state, and none of the first three barriers can be solved on data that hasn't been properly parsed, labelled, and indexed with its permissions attached.
Doing that labelling at scale, on infrastructure that doesn't blow the budget, is the least visible part of a generative AI programme and the part most pilots underestimate. Processing at the point of ingestion, close to where the data already lives, tends to be considerably cheaper than shipping everything to a remote service and paying for the round trip — and it keeps sensitive content inside the perimeter a beat earlier than barrier one requires. This groundwork is what our DataVolve practice exists to do: not a clean-up task you do once and forget, but the continuous discipline that makes everything built on top of it trustworthy.
Where each barrier is best solved
| Barrier | Where it actually gets fixed | Gets it wrong when… |
|---|---|---|
| Data exposure to external models | Depersonalisation, or a fine-tuned internal model behind your own perimeter | Sensitive data is sent to a general-purpose external model "just for this one query" |
| Unauthorised internal access | Access control enforced inside the retrieval query, not after | Filtering happens in application code, after the data has already been retrieved |
| Unreliable or fabricated output | Citation, cross-referencing against source, human review on high-stakes answers | The model's fluency is mistaken for accuracy and nothing is checked |
| Cost of unstructured data | Labelling and indexing at the point of ingestion, close to the source | Everything is shipped to a remote service and labelled reactively, per query |
The thread connecting all four
None of these barriers is really about the model. They're about the engineering layer sitting between your data and whichever model you choose — the layer that decides what leaves your perimeter, who's allowed to see the output, whether a claim can be trusted, and what it costs to get there. That layer is harder to build than a demo and far harder for a competitor to copy than a choice of model, which is exactly why it's where a generative AI programme earns the trust to move from pilot into something the business actually depends on.
How Tarento approaches it
We build this as one connected system rather than four separate projects. DataVolve establishes trustworthy, well-labelled data before anything is indexed. Permission-aware retrieval inherits the access controls your business already runs, so generative AI can't become a route around your own governance. Grounding and citation checks keep every consequential answer traceable to a source a person can open. And through our enterprise integration and generative & agentic AI practices, we help you decide — deliberately, use case by use case — which work belongs on a large external model and which belongs on a smaller model running inside your own walls.
Traceability and restraint by default, not bolted on after the pilot succeeds. Plenty of vendors will hand you access to a model. The harder, more valuable job is the layer that lets you actually stand behind what it says.
Frequently asked questions
1. Why do so many generative AI pilots fail to reach production? Most pilots are built to prove the model works, not to solve the four problems that actually gate production use: whether sensitive data can be exposed to an external model, whether access controls carry over once data is indexed, whether the output can be trusted, and whether the unstructured data behind it has been properly prepared. A working demo answers none of these.
2. Is it safer to use a large external model or a smaller internal one? It depends on the task. A smaller model fine-tuned on your own data can match or beat a large general-purpose model on a narrow, repeatable job, at a fraction of the cost, and can run inside your own security perimeter. Large external models earn their place on broad, open-ended work where their scale genuinely helps. Most enterprises land on a deliberate hybrid, decided case by case.
3. Does grounding a model in company data stop it from hallucinating? It reduces the frequency considerably but doesn't remove it. Models can still introduce or misstate a claim even when handed the correct source passage. The reliable defence is engineered, not assumed: cross-reference generated claims against retrieved sources, require citations, and keep human review on high-stakes answers.
4. How does access control work once data from multiple systems is indexed together? It has to be enforced inside the retrieval step itself, filtering the search against the requesting user's role and attributes before anything is returned, rather than retrieving everything and filtering afterwards. That way content the user isn't entitled to see is never retrieved in the first place, which is the only version of the control that can't leak.
5. Why does unstructured data cost so much to prepare for AI? Roughly 80% of enterprise data is unstructured, and none of it is usable by a model until it has been parsed, chunked, labelled, and indexed with its permissions intact. Doing that labelling reactively, per query, on a remote service is expensive at scale. Doing it once, at the point of ingestion, close to where the data already lives, is considerably cheaper and forms the foundation on which everything else depends.
Ready to move past the pilot?
If your generative AI programme is stalling on data exposure, access control, trust in the output, or the cost of getting there, that's precisely the layer we build. Explore Tarento's AI services, and let's talk about what's actually holding your rollout back.
Five Enterprise Cloud Infrastructure Foundations That Define Reliability
