Who this is for: CIOs, CDOs, heads of knowledge management, and the search, content operations, and information architecture leaders responsible for content that an AI system will retrieve. It assumes you own or influence a content estate, not that you build retrieval pipelines yourself.
What this covers: why taxonomy and metadata still determine retrieval quality after semantic search solved term variation, what AI can and cannot build, the metadata fields that matter most for AI retrieval, what the work has returned in practice, and a five-step sequence for closing the gap.
What this does not cover: embedding model selection, vector database comparison, chunking parameter tuning, prompt engineering, and LLM evaluation methodology. Those decisions sit downstream of the content work described here and are addressed separately.
Prerequisites for acting on this article: access to your search query logs, an inventory of content repositories, and at least one named owner for one content domain. If you have none of the three, start at step 1 of the roadmap rather than at the architecture.
Published: November 8, 2026. Last updated: November 8, 2026. Next review: November 2027.
Shelf life: The core claim, that taxonomy, metadata, governance, and lifecycle determine retrieval quality, is durable and predates current AI architectures. The specific retrieval architecture described here, hybrid vector search plus structured filtering plus a knowledge graph, reflects practice as of November 2026 and will change as tooling matures. Every statistic carries its source and date at the point of use.
Taxonomy and metadata tell a search engine or an AI assistant what a document is about, who it is for, and whether it is current. Without them, retrieval runs on similarity alone, and RAG systems answer from whatever the index returns. Structured, governed content is what search engines and large language models can retrieve, trust, and cite. There is no AI without IA.
The enterprises doing this work already know where the constraint sits. In the Komprise 2026 State of Unstructured Data Management report, published December 2025 and fielded in October 2025, classifying and tagging unstructured data was the top challenge in preparing data for AI, cited by 56% of respondents, up from 41% in 2024. That is a taxonomy and metadata problem stated plainly, and by that measure it is getting harder, not easier.
For most of the history of enterprise search, retrieval meant exact term matching. If one team called a document a brochure and another searched for collateral, the document stayed hidden. Synonym rings and thesauri existed to patch that gap, and maintaining them was real work.
Semantic search largely solved it. Embeddings place similar concepts near each other mathematically, so a query for budget cuts finds documents about cost reduction with no shared words. Term variation is no longer the problem it was, and anyone still building synonym files as a primary relevance strategy is solving a 2015 problem.
The structural problems did not go away. Three remain, and each has a direct consequence.
Semantic search cannot filter. It has no concept of department, region, product line, audience, or lifecycle status. Because it cannot tell an approved policy from a superseded draft, both being semantically about the same subject, a draft gets retrieved and cited as policy.
Semantic search does not know what things are. It knows that two chunks sit close together in vector space. It does not know that one is a procedure and the other is a warning that governs that procedure, or that a component belongs to an assembly. The result is answers that are topically correct and operationally wrong.
Semantic search has no context about the person asking. The same question from an engineer, a sales representative, and a customer should return different content. Similarity alone cannot make that distinction, so everyone receives the same answer and most of them receive the wrong one.
Taxonomy and metadata supply what embeddings cannot: the filters, the types, and the context.
Large language models draft taxonomies well. Point one at a content corpus and it will propose a hierarchy, cluster similar items, suggest labels, and flag duplicates and gaps. That work used to take weeks of manual card sorting. Earley Information Science built this capability into the VIA platform, which derives taxonomies and content models directly from content. Across repeated engagements the pattern holds: by the third or fourth domain, agents propose most of the structure and human effort shifts from building to validating.
The hard part is not the hierarchy. It is the ontology, meaning the associative relationships between concepts that sit in different branches of the structure:
None of these are parent-child relationships, and none can be inferred from term co-occurrence. A model can observe that two documents discuss similar topics. It cannot know that this methodology is the one that delivers that outcome in a regulated environment, or that this audit step requires that procedure and no other. Those relationships come from domain expertise and judgment, and they are what make retrieval precise instead of merely relevant.
Acceleration is still substantial. Earley Information Science cut knowledge graph development for a global consultancy from 18 months to 12 weeks. The compression comes from automating the mechanical work and concentrating expert time on the relationships that require it, not from removing the expert.
RAG works in sequence: retrieve, then generate, then cite. If retrieval misses the current policy, the model fills the gap from training data that knows nothing about your enterprise. If retrieval returns a superseded version, the model gives stale guidance with full confidence. Retrieval does not remove every hallucination, but no generation step can exceed the quality of the content it is given.
Hybrid retrieval is how this gets built. Earley Information Science calls its approach IAD-RAG, meaning information-architecture-directed retrieval, which shapes content and constrains retrieval through governed terminology, knowledge engineering, domain boundaries, and semantic controls. In practice, four steps:
Take the query, how do we handle returns for electronics. Semantic search alone returns documents that are about returns and about electronics. Hybrid retrieval returns the current return policy for that user's region. A knowledge graph adds the associative layer: compatible parts, prerequisites, and applicable hazards.
Six fields do most of the work for AI retrieval:
Add last-reviewed and next-review dates, and links between superseded and superseding versions. Lifecycle status is the field teams most often skip and the one that causes the most damage, because lifecycle status is what keeps a draft from being retrieved and cited as approved policy.
The goal is the metadata sweet spot. Too little metadata and AI cannot find the content. Too much and nobody applies it. Define required and optional fields, automate capture of dates and authors, use machine-assisted tagging for subject terms, and build the schema in stages. Progressive enhancement beats a perfect schema that never ships.
The returns show up wherever retrieval sits between a person and a decision. The following outcomes are drawn from Earley Information Science engagements and are reported without client names.
Product data syndicated to channel partners: $150 million in net new revenue. A diversified manufacturer digitized products one SKU at a time, organized by the division that made them, with no global consistency in attribution across a catalog of more than a million SKUs. The catalog was reorganized so that products with similar attributes sat together regardless of origin, and that metadata was syndicated to distributor sites. The company generated $150 million in net new revenue. The retrieval happened on someone else's site, which is only possible when the metadata travels with the product.
A virtual assistant built before large language models existed: 30% fewer help desk operations. A business insurance carrier launched one of the first virtual assistants on the market to help field agents research and write policies in a line of business they did not know well. Phone-based help desk support was expensive and overused. The assistant worked because the content behind it was componentized in DITA, tagged against a taxonomy, and retrieved through a search layer. Manual help desk operations fell 30% in the first six months. There was no generative model anywhere in it. The retrieval layer did the work, which is the same layer carrying RAG today.
Claims processing against long policy documents: 20% less handling time. A national health insurer gave claims processors structured access to complex policy content through content models and a taxonomy tested across 12 scenarios and 25 use cases. Complex claim handling time dropped 20% and response accuracy improved.
Organic search and conversion: 34% higher click-through, 15% better conversion. An electronics manufacturer rebuilt its product taxonomy and normalized attribute schemas across 700 categories in four months. Click-through rates rose 34%, and the organic search gain was worth roughly $2 million in paid search spend. Separately, an industrial distributor improved conversions 15% on product detail pages after selecting and optimizing the comparative attributes that mattered in 1,000 categories, covering 80% of its online catalog.
Different industries, different decades, one mechanism: content structured so the retrieval layer can find the right thing.
Six named Earley Information Science frameworks appear in this article. They are sequenced, not interchangeable, and each depends on the ones before it.
The dependency chain is not optional. You cannot run the Retrieval Accuracy Improvement Loop without retrieval in production. You cannot build IAD-RAG without a governed taxonomy. You cannot govern a taxonomy without named owners.
Earley Information Science assesses search and AI readiness on a five-level maturity scale, shared across its AI Readiness and Search Readiness assessments so results are comparable. The scale describes the organization's stance toward quality. Each level below stands alone.
Search and AI readiness Level 1, Not in place. The capability has not been started or does not exist. Search operates on defaults with no deliberate optimization. Problems find the team through complaints.
Search and AI readiness Level 2, Minimal. Awareness exists but action is minimal and complaint-driven. Fixes happen only when problems are reported.
Search and AI readiness Level 3, Developing. The organization is actively working on the capability with early implementation. Coverage is partial and application is inconsistent.
Search and AI readiness Level 4, Established. The capability is in place and working well. It covers most scenarios with defined processes and is measured and tuned, but it is not yet anticipatory.
Search and AI readiness Level 5, Optimized. The capability is mature, measured, and continuously improving, automated where appropriate. Problems are anticipated before users encounter them, and performance is tied to business outcomes.
Retrieval improves most in the move from Level 2 to Level 3, when terminology, metadata, and lifecycle stop being ad hoc and become governed.
Each step below lists what must be true before it starts, what the step produces, and the test for whether it is finished. Running a step without its prerequisite is the most common way this work stalls.
Prerequisite: access to search query logs and a list of content repositories.
Do: inventory content, taxonomy, and metadata. Pull zero-result queries and high-reformulation queries from the logs. Score a representative sample of the corpus for retrieval readiness so you know which repositories are ready and which are not.
Produces: a gap list ranked by query volume and a baseline retrieval readiness score.
Done when: you can name the top 20 failing queries and the repository each one should have been answered from.
Prerequisite: step 1 complete, plus access to subject matter experts for the highest-value domain.
Do: use AI to propose the taxonomy and content model. Spend expert time on the associative relationships a model cannot infer, such as which procedure applies to which step and which hazard governs which action.
Produces: a taxonomy, a metadata schema with required and optional fields, and a first set of modeled relationships.
Done when: subject matter experts validate the relationships rather than rebuild them.
Prerequisite: a taxonomy and metadata schema from step 2, plus executive sponsorship to assign ownership.
Do: assign content owners, lifecycle states, and review dates. Use the GEO Pillar Model to decide which decisions belong to Governance, which to Engineering, and which to Operations.
Produces: named owners per domain, an approval workflow, and lifecycle status applied to the priority corpus.
Done when: you can answer the question, where is our authoritative content on this topic for this audience. If you cannot answer it, do not proceed to step 4.
Prerequisite: step 3 complete. Governed metadata is the input the filter stage runs on, so this step cannot start without it.
Do: filter on metadata, search semantically within the filtered set, re-rank by authority and freshness, and return citations.
Produces: a retrieval pipeline whose results can be traced to a source and a reason.
Done when: precision and recall on a defined set of test queries beat the baseline from step 1.
Prerequisite: retrieval running in production with logging enabled.
Do: run the Retrieval Accuracy Improvement Loop, which monitors performance, diagnoses root causes, corrects content and structure, optimizes retrieval logic, and validates that accuracy improved. Run the Semantic Drift Prevention Loop alongside it to keep terminology stable as content changes. Track precision at K, citation accuracy, and groundedness next to business measures.
Produces: a rising retrieval readiness score and a documented error taxonomy. One public-facing corpus moved from 4.5 to 8.0 on the AIRR-10 scale through targeted remediation.
Done when: the loop runs on a defined cadence without being prompted, which is the Level 5 condition described above.
Search modernization is now an AI readiness project, and that is the case to take to your sponsor. Start with the audit. Identify the 100 questions your users or customers ask most, find the authoritative content for each, and check whether it is tagged, current, and owned. The gaps you find are your roadmap.
No. Vector search solved term variation, so a query for budget cuts now finds documents about cost reduction without a synonym file. It cannot filter by department, region, audience, or lifecycle status, and it cannot tell an approved policy from a superseded draft because both are semantically about the same subject. Taxonomy and metadata supply the filters and content types that embeddings do not carry.
Largely, yes. Large language models draft taxonomies well. They cluster content, propose hierarchies, suggest labels, and flag duplicates and gaps, compressing work that used to take weeks of manual card sorting. What they cannot do is establish the associative relationships of an ontology, such as which methodology produces a given outcome or which procedure applies to a given audit step. Those relationships require domain expertise and judgment.
A taxonomy is a hierarchy that classifies content into parent-child relationships. An ontology adds the associative relationships between concepts that sit in different branches, such as which template belongs to which methodology, which hazard governs which action, and which procedure applies to which audit step. Taxonomies make content browsable and filterable. Ontologies make retrieval precise, because an ontology encodes how concepts actually relate.
Hierarchies can be inferred from content. A model can observe that two documents cover similar topics and group them. Associative relationships cannot be inferred that way, because co-occurrence does not establish that a given methodology delivers a given outcome in a regulated environment, or that a given step requires one specific procedure and no other. Those connections exist in practitioners' heads and have to be elicited, modeled, and validated by people with domain expertise.
Hybrid search combines three retrieval methods: structured filters built on taxonomy and metadata, semantic search for similarity within the filtered set, and a knowledge graph for relationships between entities. A re-ranking step then applies business rules such as authority and freshness. The large language model generates its answer only from the content that survives these steps, which is what allows the answer to cite its sources.
Six fields do most of the work: content type, lifecycle status, audience, authority level, geographic scope, and product applicability. Add last-reviewed and next-review dates plus links to superseded versions. Lifecycle status is the field teams most often skip and the one that causes the most damage, because lifecycle status is what keeps a draft from being retrieved and cited as approved policy.
Enough that retrieval can find the right content, and no more than contributors will apply consistently. Earley Information Science calls this the metadata sweet spot: too little and AI cannot find the content, too much and nobody applies it. Define required and optional fields, automate capture of dates and authors, use assisted tagging for subject terms, and build the schema progressively rather than attempting a complete schema before launch.
Content lifecycle status is a retrieval control. Marking content as draft, in review, published, archived, or deprecated lets the retrieval layer exclude by default anything that should never be cited. Without it, a superseded policy and the current one look equally relevant to a similarity search, and the model will present whichever one it retrieves with full confidence and no signal that the source was wrong.
Classification and tagging. In the Komprise 2026 State of Unstructured Data Management report, 56% of respondents named classifying and tagging unstructured data as their top challenge in preparing data for AI, up from 41% in 2024. Data governance and security concerns ranked second at 46%. The constraint is information architecture work, not model selection.
Results follow wherever retrieval sits between a person and a decision. In Earley Information Science engagements, a diversified manufacturer generated $150 million in net new revenue by reorganizing product attribution and syndicating that metadata to distributor sites. A business insurance carrier cut manual help desk operations 30% in six months with an assistant retrieving from componentized content. A national health insurer reduced complex claim handling time 20%. An electronics manufacturer raised click-through rates 34% after a taxonomy and attribute rebuild.
Earley Information Science assesses readiness on five levels: Not in place, Minimal, Developing, Established, and Optimized. Level 1 means search runs on defaults and problems surface through complaints. Level 5 means the practice is measured, automated where appropriate, tied to business outcomes, and anticipates problems before users encounter them. The same scale is used across the AI Readiness and Search Readiness assessments so results are comparable.
Three things, in order. A governed taxonomy and metadata schema, because the filter stage of hybrid retrieval runs on metadata. Named content owners with lifecycle states applied, because retrieval cannot exclude deprecated content that was never marked deprecated. A baseline measurement from search logs, because without it there is no way to prove the new pipeline is better. Teams that skip the first two build retrieval that returns confident answers from the wrong documents.
Start by auditing current content, taxonomy, and metadata, and pull the failed queries from search logs. Use AI to draft the taxonomy, then spend expert time on the ontology relationships a model cannot infer. Assign owners and lifecycle states, then test whether you can answer the question, where is the authoritative content on this topic for this audience. If you can answer it, move to hybrid retrieval.
The Retrieval Accuracy Improvement Loop is the Earley Information Science cycle for keeping AI retrieval accurate over time. It has five phases: monitor retrieval performance, diagnose root causes such as metadata gaps or poor chunking, correct the content and structure, optimize retrieval logic, and validate that accuracy improved. The loop then restarts, so retrieval improves instead of degrading.
Contact Earley Information Science to discuss how to build the taxonomy, metadata, and retrieval architecture that make AI answers trustworthy. Contact us.