Global – October 29, 2026 -- Some 34 percent of employees still struggle to find the information they need to do their jobs, according to Gartner's Market Guide for Enterprise AI Search, published in September 2025. Enterprises are now rearchitecting search around retrieval-augmented generation (RAG) and agentic AI, a shift that can multiply inference costs by three to five times at scale.
Gartner finds 36 percent of AI tool users still cannot access relevant information
Roughly 49 percent of workers now rely primarily on AI tools such as Copilot or Gemini for data discovery rather than traditional search interfaces. Yet 36 percent of those users still report difficulty reaching relevant information.
Gartner forecasts that by 2028, 60 percent of organizations will run more than six enterprise AI search platforms. It also expects 60 percent of enterprise applications to embed AI search, up from roughly 20 percent today. The firm describes a move from information retrieval, which returns a list of documents, to information synthesis, which returns a grounded, contextual answer.
No single architecture dominates. Enterprises are combining unified platforms that index many sources, in-application search, and federated search that queries separate indexes without centralizing them.
Production RAG pipelines combine two indexes, fusion and reranking
Documents are typically split into chunks of 512 to 1,024 tokens with some overlap. Each chunk is converted into a dense vector embedding using models such as OpenAI's text-embedding-3-large, Voyage AI's models, BGE-M3 or Cohere's Embed family.
Production systems run a lexical index, commonly BM25, alongside a dense vector index. Reciprocal Rank Fusion merges the two rankings and tends to outperform either method alone on content that mixes structured identifiers with prose. A cross-encoder reranker then reorders the candidates before the selected passages reach the language model.
Agentic RAG replaces the single retrieval pass with a reasoning loop
Standard RAG embeds the query, retrieves, reranks and generates in a fixed sequence, retrieving exactly once. Agentic RAG lets a language model orchestrate the process through four capabilities: query decomposition, dynamic tool routing, self-evaluation of retrieved context, and multi-agent collaboration.
The costs differ sharply by pattern. A simple router agent adds a few hundred milliseconds of classification overhead. A multi-agent retrieval pattern can add five to fifteen seconds of latency and typically issues four to six language model calls where standard RAG issues one.
Enterprises in production commonly route simple queries straight to standard RAG and use smaller models for orchestration steps. They also cache decomposition results for recurring query patterns.
Search agents need permissions that mirror the requesting employee
An agentic search system is functionally an AI agent with read access to much of an enterprise's knowledge and, in some deployments, write access. AppsTek says such an agent needs a defined owner and a permission boundary matching the employee's existing access rather than a broader service-level credential. It also needs an audit trail linking each retrieved passage and generated answer to its source.
Where the underlying model runs carries data residency, licensing and jurisdictional implications. AppsTek treats that choice as part of the search architecture decision, not a separate one.
Six questions frame the deployment decision
AppsTek advises establishing baseline retrieval quality before adding any agentic layer. It also recommends identifying which queries need multi-step reasoning and setting a latency and cost ceiling per query before rollout. Teams should evaluate retrieval and generation separately so they can tell which stage produced a wrong answer.