Who this is for: CIOs, CDOs, heads of knowledge management, and the search, content operations, and information architecture leaders responsible for content that an AI system will retrieve. It assumes you own or influence a content estate, not that you build retrieval pipelines yourself.
What this covers: why taxonomy and metadata still determine retrieval quality after semantic search solved term variation, what AI can and cannot build, the metadata fields that matter most for AI retrieval, what the work has returned in practice, and a five-step sequence for closing the gap.
What this does not cover: embedding model selection, vector database comparison, chunking parameter tuning, prompt engineering, and LLM evaluation methodology. Those decisions sit downstream of the content work described here and are addressed separately.
Prerequisites for acting on this article: access to your search query logs, an inventory of content repositories, and at least one named owner for one content domain. If you have none of the three, start at step 1 of the roadmap rather than at the architecture.
Published: November 8, 2026. Last updated: November 8, 2026. Next review: November 2027.
Shelf life: The core claim, that taxonomy, metadata, governance, and lifecycle determine retrieval quality, is durable and predates current AI architectures. The specific retrieval architecture described here, hybrid vector search plus structured filtering plus a knowledge graph, reflects practice as of November 2026 and will change as tooling matures. Every statistic carries its source and date at the point of use.
Key terms used in this article
- Taxonomy. A controlled hierarchy of terms that classifies content into parent-child relationships.
- Ontology. A model of concepts and the associative relationships between them, including relationships that cross branches of a taxonomy.
- Metadata. Structured attributes attached to a piece of content that describe what it is, who it is for, and whether it is current.
- Semantic search, also called vector search. Retrieval that matches on meaning by comparing numeric representations, called embeddings, rather than matching exact words.
- Retrieval-augmented generation, abbreviated RAG. An architecture in which a system retrieves source content first, then has a large language model generate an answer from that content and cite it.
- Hybrid retrieval. Retrieval that combines structured metadata filtering, semantic search, and knowledge graph relationships in one pipeline.
- Knowledge graph. A structure that stores entities and the explicit relationships between them, so a system can follow connections rather than infer them from similarity.
- Content lifecycle status. A metadata field recording whether content is draft, in review, published, archived, or deprecated.
- Chunk. The unit of content a retrieval system indexes and returns, usually a section or passage rather than a whole document.
Taxonomy and metadata tell a search engine or an AI assistant what a document is about, who it is for, and whether it is current. Without them, retrieval runs on similarity alone, and RAG systems answer from whatever the index returns. Structured, governed content is what search engines and large language models can retrieve, trust, and cite. There is no AI without IA.
The enterprises doing this work already know where the constraint sits. In the Komprise 2026 State of Unstructured Data Management report, published December 2025 and fielded in October 2025, classifying and tagging unstructured data was the top challenge in preparing data for AI, cited by 56% of respondents, up from 41% in 2024. That is a taxonomy and metadata problem stated plainly, and by that measure it is getting harder, not easier.
What vector search fixed, and what it did not
For most of the history of enterprise search, retrieval meant exact term matching. If one team called a document a brochure and another searched for collateral, the document stayed hidden. Synonym rings and thesauri existed to patch that gap, and maintaining them was real work.
Semantic search largely solved it. Embeddings place similar concepts near each other mathematically, so a query for budget cuts finds documents about cost reduction with no shared words. Term variation is no longer the problem it was, and anyone still building synonym files as a primary relevance strategy is solving a 2015 problem.
The structural problems did not go away. Three remain, and each has a direct consequence.
Semantic search cannot filter. It has no concept of department, region, product line, audience, or lifecycle status. Because it cannot tell an approved policy from a superseded draft, both being semantically about the same subject, a draft gets retrieved and cited as policy.
Semantic search does not know what things are. It knows that two chunks sit close together in vector space. It does not know that one is a procedure and the other is a warning that governs that procedure, or that a component belongs to an assembly. The result is answers that are topically correct and operationally wrong.
Semantic search has no context about the person asking. The same question from an engineer, a sales representative, and a customer should return different content. Similarity alone cannot make that distinction, so everyone receives the same answer and most of them receive the wrong one.
Taxonomy and metadata supply what embeddings cannot: the filters, the types, and the context.
What AI can build, and what still takes judgment
Large language models draft taxonomies well. Point one at a content corpus and it will propose a hierarchy, cluster similar items, suggest labels, and flag duplicates and gaps. That work used to take weeks of manual card sorting. Earley Information Science built this capability into the VIA platform, which derives taxonomies and content models directly from content. Across repeated engagements the pattern holds: by the third or fourth domain, agents propose most of the structure and human effort shifts from building to validating.
The hard part is not the hierarchy. It is the ontology, meaning the associative relationships between concepts that sit in different branches of the structure:
- Which methodologies produce a given outcome
- Which templates belong to a given methodology
- Which procedure applies to a given audit step
- Which template fits a given industry
- Which hazard governs a given action
None of these are parent-child relationships, and none can be inferred from term co-occurrence. A model can observe that two documents discuss similar topics. It cannot know that this methodology is the one that delivers that outcome in a regulated environment, or that this audit step requires that procedure and no other. Those relationships come from domain expertise and judgment, and they are what make retrieval precise instead of merely relevant.
Acceleration is still substantial. Earley Information Science cut knowledge graph development for a global consultancy from 18 months to 12 weeks. The compression comes from automating the mechanical work and concentrating expert time on the relationships that require it, not from removing the expert.
Retrieval quality sets the ceiling on AI answer quality
RAG works in sequence: retrieve, then generate, then cite. If retrieval misses the current policy, the model fills the gap from training data that knows nothing about your enterprise. If retrieval returns a superseded version, the model gives stale guidance with full confidence. Retrieval does not remove every hallucination, but no generation step can exceed the quality of the content it is given.
Hybrid retrieval is how this gets built. Earley Information Science calls its approach IAD-RAG, meaning information-architecture-directed retrieval, which shapes content and constrains retrieval through governed terminology, knowledge engineering, domain boundaries, and semantic controls. In practice, four steps:
- Parse the question for intent and entities.
- Filter by taxonomy and metadata, for example current, customer-facing, US region, electronics category.
- Run semantic search within the filtered set.
- Re-rank by authority and freshness, then pass the surviving content to the large language model with citations.
Take the query, how do we handle returns for electronics. Semantic search alone returns documents that are about returns and about electronics. Hybrid retrieval returns the current return policy for that user's region. A knowledge graph adds the associative layer: compatible parts, prerequisites, and applicable hazards.
The metadata that makes content retrievable
Six fields do most of the work for AI retrieval:
- Content type. Policy, procedure, product documentation, or another named type.
- Lifecycle status. Draft, in review, published, archived, or deprecated.
- Audience. Employee, customer, or partner.
- Authority level. Official policy, guidance, or opinion.
- Geographic scope. The regions where the content applies.
- Product applicability. The products, versions, or configurations the content covers.
Add last-reviewed and next-review dates, and links between superseded and superseding versions. Lifecycle status is the field teams most often skip and the one that causes the most damage, because lifecycle status is what keeps a draft from being retrieved and cited as approved policy.
The goal is the metadata sweet spot. Too little metadata and AI cannot find the content. Too much and nobody applies it. Define required and optional fields, automate capture of dates and authors, use machine-assisted tagging for subject terms, and build the schema in stages. Progressive enhancement beats a perfect schema that never ships.
What this work has returned
The returns show up wherever retrieval sits between a person and a decision. The following outcomes are drawn from Earley Information Science engagements and are reported without client names.
Product data syndicated to channel partners: $150 million in net new revenue. A diversified manufacturer digitized products one SKU at a time, organized by the division that made them, with no global consistency in attribution across a catalog of more than a million SKUs. The catalog was reorganized so that products with similar attributes sat together regardless of origin, and that metadata was syndicated to distributor sites. The company generated $150 million in net new revenue. The retrieval happened on someone else's site, which is only possible when the metadata travels with the product.
A virtual assistant built before large language models existed: 30% fewer help desk operations. A business insurance carrier launched one of the first virtual assistants on the market to help field agents research and write policies in a line of business they did not know well. Phone-based help desk support was expensive and overused. The assistant worked because the content behind it was componentized in DITA, tagged against a taxonomy, and retrieved through a search layer. Manual help desk operations fell 30% in the first six months. There was no generative model anywhere in it. The retrieval layer did the work, which is the same layer carrying RAG today.
Claims processing against long policy documents: 20% less handling time. A national health insurer gave claims processors structured access to complex policy content through content models and a taxonomy tested across 12 scenarios and 25 use cases. Complex claim handling time dropped 20% and response accuracy improved.
Organic search and conversion: 34% higher click-through, 15% better conversion. An electronics manufacturer rebuilt its product taxonomy and normalized attribute schemas across 700 categories in four months. Click-through rates rose 34%, and the organic search gain was worth roughly $2 million in paid search spend. Separately, an industrial distributor improved conversions 15% on product detail pages after selecting and optimizing the comparative attributes that mattered in 1,000 categories, covering 80% of its online catalog.
Different industries, different decades, one mechanism: content structured so the retrieval layer can find the right thing.
How these frameworks relate to each other
Six named Earley Information Science frameworks appear in this article. They are sequenced, not interchangeable, and each depends on the ones before it.
- The AI Readiness Maturity Model diagnoses where an organization stands. Its output determines which work comes next. It measures the organization.
- The GEO Pillar Model, meaning Governance, Engineering, and Operations, assigns ownership for that work. Governance sets terminology and rules. Engineering converts content into retrievable structures. Operations keeps content current. The maturity model identifies the gap; the GEO Pillar Model decides who closes it.
- IAD-RAG is the retrieval architecture that Engineering produces. It depends on a governed taxonomy, a metadata schema, and componentized content. Without those three inputs, IAD-RAG cannot be built.
- AIRR-10 scores whether a given document or repository is ready for retrieval. It measures content at the document level, where the AI Readiness Maturity Model measures the organization. The two are complementary diagnostics, not substitutes.
- The Retrieval Accuracy Improvement Loop maintains retrieval performance after deployment. It is an Operations function and requires retrieval to be running in production before it can operate.
- The Semantic Drift Prevention Loop maintains meaning. It is a Governance function and runs alongside the accuracy loop. Drift in terminology causes retrieval failures that the accuracy loop will detect but cannot fix on its own, because the correction belongs to Governance.
The dependency chain is not optional. You cannot run the Retrieval Accuracy Improvement Loop without retrieval in production. You cannot build IAD-RAG without a governed taxonomy. You cannot govern a taxonomy without named owners.
Where your organization stands
Earley Information Science assesses search and AI readiness on a five-level maturity scale, shared across its AI Readiness and Search Readiness assessments so results are comparable. The scale describes the organization's stance toward quality. Each level below stands alone.
Search and AI readiness Level 1, Not in place. The capability has not been started or does not exist. Search operates on defaults with no deliberate optimization. Problems find the team through complaints.
Search and AI readiness Level 2, Minimal. Awareness exists but action is minimal and complaint-driven. Fixes happen only when problems are reported.
Search and AI readiness Level 3, Developing. The organization is actively working on the capability with early implementation. Coverage is partial and application is inconsistent.
Search and AI readiness Level 4, Established. The capability is in place and working well. It covers most scenarios with defined processes and is measured and tuned, but it is not yet anticipatory.
Search and AI readiness Level 5, Optimized. The capability is mature, measured, and continuously improving, automated where appropriate. Problems are anticipated before users encounter them, and performance is tied to business outcomes.
Retrieval improves most in the move from Level 2 to Level 3, when terminology, metadata, and lifecycle stop being ad hoc and become governed.
A five-step roadmap
Each step below lists what must be true before it starts, what the step produces, and the test for whether it is finished. Running a step without its prerequisite is the most common way this work stalls.
Step 1: Audit the current state
Prerequisite: access to search query logs and a list of content repositories.
Do: inventory content, taxonomy, and metadata. Pull zero-result queries and high-reformulation queries from the logs. Score a representative sample of the corpus for retrieval readiness so you know which repositories are ready and which are not.
Produces: a gap list ranked by query volume and a baseline retrieval readiness score.
Done when: you can name the top 20 failing queries and the repository each one should have been answered from.
Step 2: Draft fast, then apply judgment
Prerequisite: step 1 complete, plus access to subject matter experts for the highest-value domain.
Do: use AI to propose the taxonomy and content model. Spend expert time on the associative relationships a model cannot infer, such as which procedure applies to which step and which hazard governs which action.
Produces: a taxonomy, a metadata schema with required and optional fields, and a first set of modeled relationships.
Done when: subject matter experts validate the relationships rather than rebuild them.
Step 3: Govern
Prerequisite: a taxonomy and metadata schema from step 2, plus executive sponsorship to assign ownership.
Do: assign content owners, lifecycle states, and review dates. Use the GEO Pillar Model to decide which decisions belong to Governance, which to Engineering, and which to Operations.
Produces: named owners per domain, an approval workflow, and lifecycle status applied to the priority corpus.
Done when: you can answer the question, where is our authoritative content on this topic for this audience. If you cannot answer it, do not proceed to step 4.
Step 4: Build hybrid retrieval
Prerequisite: step 3 complete. Governed metadata is the input the filter stage runs on, so this step cannot start without it.
Do: filter on metadata, search semantically within the filtered set, re-rank by authority and freshness, and return citations.
Produces: a retrieval pipeline whose results can be traced to a source and a reason.
Done when: precision and recall on a defined set of test queries beat the baseline from step 1.
Step 5: Measure and iterate
Prerequisite: retrieval running in production with logging enabled.
Do: run the Retrieval Accuracy Improvement Loop, which monitors performance, diagnoses root causes, corrects content and structure, optimizes retrieval logic, and validates that accuracy improved. Run the Semantic Drift Prevention Loop alongside it to keep terminology stable as content changes. Track precision at K, citation accuracy, and groundedness next to business measures.
Produces: a rising retrieval readiness score and a documented error taxonomy. One public-facing corpus moved from 4.5 to 8.0 on the AIRR-10 scale through targeted remediation.
Done when: the loop runs on a defined cadence without being prompted, which is the Level 5 condition described above.
What to do next
Search modernization is now an AI readiness project, and that is the case to take to your sponsor. Start with the audit. Identify the 100 questions your users or customers ask most, find the authoritative content for each, and check whether it is tagged, current, and owned. The gaps you find are your roadmap.
Frequently asked questions
Does vector search make taxonomy and metadata unnecessary?
No. Vector search solved term variation, so a query for budget cuts now finds documents about cost reduction without a synonym file. It cannot filter by department, region, audience, or lifecycle status, and it cannot tell an approved policy from a superseded draft because both are semantically about the same subject. Taxonomy and metadata supply the filters and content types that embeddings do not carry.
Can AI build an enterprise taxonomy?
Largely, yes. Large language models draft taxonomies well. They cluster content, propose hierarchies, suggest labels, and flag duplicates and gaps, compressing work that used to take weeks of manual card sorting. What they cannot do is establish the associative relationships of an ontology, such as which methodology produces a given outcome or which procedure applies to a given audit step. Those relationships require domain expertise and judgment.
What is the difference between a taxonomy and an ontology?
A taxonomy is a hierarchy that classifies content into parent-child relationships. An ontology adds the associative relationships between concepts that sit in different branches, such as which template belongs to which methodology, which hazard governs which action, and which procedure applies to which audit step. Taxonomies make content browsable and filterable. Ontologies make retrieval precise, because an ontology encodes how concepts actually relate.
Why is an ontology harder to build than a taxonomy?
Hierarchies can be inferred from content. A model can observe that two documents cover similar topics and group them. Associative relationships cannot be inferred that way, because co-occurrence does not establish that a given methodology delivers a given outcome in a regulated environment, or that a given step requires one specific procedure and no other. Those connections exist in practitioners' heads and have to be elicited, modeled, and validated by people with domain expertise.
What is hybrid search in a RAG architecture?
Hybrid search combines three retrieval methods: structured filters built on taxonomy and metadata, semantic search for similarity within the filtered set, and a knowledge graph for relationships between entities. A re-ranking step then applies business rules such as authority and freshness. The large language model generates its answer only from the content that survives these steps, which is what allows the answer to cite its sources.
Which metadata fields matter most for AI retrieval?
Six fields do most of the work: content type, lifecycle status, audience, authority level, geographic scope, and product applicability. Add last-reviewed and next-review dates plus links to superseded versions. Lifecycle status is the field teams most often skip and the one that causes the most damage, because lifecycle status is what keeps a draft from being retrieved and cited as approved policy.
How much metadata is enough?
Enough that retrieval can find the right content, and no more than contributors will apply consistently. Earley Information Science calls this the metadata sweet spot: too little and AI cannot find the content, too much and nobody applies it. Define required and optional fields, automate capture of dates and authors, use assisted tagging for subject terms, and build the schema progressively rather than attempting a complete schema before launch.
How does content lifecycle status affect AI accuracy?
Content lifecycle status is a retrieval control. Marking content as draft, in review, published, archived, or deprecated lets the retrieval layer exclude by default anything that should never be cited. Without it, a superseded policy and the current one look equally relevant to a similarity search, and the model will present whichever one it retrieves with full confidence and no signal that the source was wrong.
What is the biggest obstacle to preparing enterprise data for AI?
Classification and tagging. In the Komprise 2026 State of Unstructured Data Management report, 56% of respondents named classifying and tagging unstructured data as their top challenge in preparing data for AI, up from 41% in 2024. Data governance and security concerns ranked second at 46%. The constraint is information architecture work, not model selection.
What business results come from taxonomy and metadata work?
Results follow wherever retrieval sits between a person and a decision. In Earley Information Science engagements, a diversified manufacturer generated $150 million in net new revenue by reorganizing product attribution and syndicating that metadata to distributor sites. A business insurance carrier cut manual help desk operations 30% in six months with an assistant retrieving from componentized content. A national health insurer reduced complex claim handling time 20%. An electronics manufacturer raised click-through rates 34% after a taxonomy and attribute rebuild.
What are the levels of search and AI readiness maturity?
Earley Information Science assesses readiness on five levels: Not in place, Minimal, Developing, Established, and Optimized. Level 1 means search runs on defaults and problems surface through complaints. Level 5 means the practice is measured, automated where appropriate, tied to business outcomes, and anticipates problems before users encounter them. The same scale is used across the AI Readiness and Search Readiness assessments so results are comparable.
What do I need in place before building a RAG system?
Three things, in order. A governed taxonomy and metadata schema, because the filter stage of hybrid retrieval runs on metadata. Named content owners with lifecycle states applied, because retrieval cannot exclude deprecated content that was never marked deprecated. A baseline measurement from search logs, because without it there is no way to prove the new pipeline is better. Teams that skip the first two build retrieval that returns confident answers from the wrong documents.
Where should an enterprise start with taxonomy and metadata for AI retrieval?
Start by auditing current content, taxonomy, and metadata, and pull the failed queries from search logs. Use AI to draft the taxonomy, then spend expert time on the ontology relationships a model cannot infer. Assign owners and lifecycle states, then test whether you can answer the question, where is the authoritative content on this topic for this audience. If you can answer it, move to hybrid retrieval.
What is the Retrieval Accuracy Improvement Loop?
The Retrieval Accuracy Improvement Loop is the Earley Information Science cycle for keeping AI retrieval accurate over time. It has five phases: monitor retrieval performance, diagnose root causes such as metadata gaps or poor chunking, correct the content and structure, optimize retrieval logic, and validate that accuracy improved. The loop then restarts, so retrieval improves instead of degrading.
Sources
- Komprise, 2026 State of Unstructured Data Management, published December 2025, fielded October 2025. https://www.komprise.com/report
- Earley Information Science, AI Retrieval Readiness Framework (AIRR-10), Version 3.0, June 2026.
- Earley Information Science, Ecommerce Search AI Foundation Diagnostic, Version 1.4, November 2026.
- Earley Information Science client engagement outcomes, reported without client names.
Contact Earley Information Science to discuss how to build the taxonomy, metadata, and retrieval architecture that make AI answers trustworthy. Contact us.