AI features are bounded by the data beneath them. Deduplication, structure, consistent definitions and access control are prerequisites for retrieval quality, not follow-up tasks.
The short answer
An AI system that reads your data cannot be more accurate than that data. If three records describe the same customer, the retrieval layer will surface whichever ranks highest — possibly the stale one. If a policy document exists in four versions, the model will confidently cite whichever it found.
No amount of prompt engineering fixes this, because it is not a prompting problem.
Duplication becomes contradiction
In an operational database, duplicate records are an annoyance that staff work around. In a retrieval system, they become contradictory evidence. The model has no basis for preferring one over another, and will often blend them into an answer that matches neither.
Deduplication and a resolved entity model are therefore prerequisites for AI features rather than data hygiene tasks to schedule afterwards.
Structure determines what can be retrieved
A scanned PDF with no text layer is invisible to retrieval. A 200-page document chunked naively returns fragments without the context needed to interpret them. A spreadsheet with merged cells and headers three rows down parses into noise.
Document pipelines — extraction, layout-aware parsing, sensible chunking with overlap, metadata attachment — are ordinary data engineering, and they determine the ceiling on what any model can do with the content.
Definitions have to be shared
If finance and sales calculate revenue differently, an AI assistant asked about revenue will be wrong for at least one of them, regardless of which number it finds.
Consistent, versioned, tested business definitions in the modelling layer are what make a single correct answer possible. This is the same work that makes reporting trustworthy; AI simply makes its absence more visible.
Freshness must be explicit
A pipeline that silently stops leaves the retrieval index serving last month’s data with complete confidence. There is no visual cue, as there would be on a dashboard with an obviously stale chart.
Freshness checks should block publication and alert. Where staleness is acceptable, the age of the data should be surfaced in the answer rather than left implicit.
Access control belongs in the data layer
Retrieval must respect the permissions of the person asking. If a document is restricted, it must not surface in their results — and that filter has to be applied at query time in the retrieval layer, not by asking the model to decline.
A model instructed not to reveal something it has already been given in context is not an access control mechanism. The document should never have been retrieved.
A sensible sequence
Identify the specific questions the AI feature must answer. Trace the data those answers depend on. Fix the duplication, structure and definitions on that path only — not across the whole estate. Then build the retrieval layer on ground you have verified.
This keeps the data work scoped to what the feature actually needs, which is what stops it becoming a two-year programme with no visible output.
Written by the Webnatrix engineering team. This is evergreen technical writing, maintained as our practice changes rather than published to a date.