AI Context Retrieval, Measured: The Honest Numbers
AI context retrieval in a default coding assistant behaves the way a person would if they had no map: it greps the repository, gets back a pile of matches, and opens whole files one at a time to work out which one it actually needed. That works. It also burns context on files that turn out to be the wrong guess, and the agent pays that tax on every lookup, all session long.
Inside Enterprise Intelligence, we replaced that brute-force search with a single lookup against a graph we call the Dependency Relationship Graph, or DRG: a committed, version-controlled map of how every document, agent, skill, and reference file in a project relates to every other one. Instead of grepping and reading, the agent asks the graph for the artifact it needs and gets back the exact path, plus how that artifact connects to everything else.
We have talked about DRG internally for a while using engineering estimates: roughly 2 to 5 thousand tokens avoided per lookup, something like 20% off the always-on startup context. Estimates are fine for planning. They are not something we are willing to put in front of a customer. So we ran a real, controlled measurement, and we are publishing the result with the same caveats we would want to see if we were the buyer reading this.
What we actually measured
The test isolates one specific operation: finding the right file for a “locate the document/agent/script that does X” task. It does not measure whole-session savings, and it does not include the always-on context block every session loads at startup, that is a separate, already-disclosed cost we deliberately excluded so we would not double-count it.
Two arms answered the same 32 pre-registered tasks, with identical prompt wording:
- Arm A (control): search with
ripgrep, the same engine behind a standard AI coding assistant’s built-in search, then confirm with a file read. - Arm B (treatment): a single
/drg-lookupquery against the graph, then confirm with a file read.
The 32 tasks were chosen before either arm ran, stratified across documents, agents, source files, skills, and reference docs, and deliberately includes targets with weak, auto-generated graph metadata so the set could not be stacked in DRG’s favor. We tracked tokens with a documented formula (characters divided by four, the same approximation both arms were charged under), and a separate, blind grader, who never saw which arm produced which result, scored how relevant the retrieved content actually was. A first run of this test failed our own internal review for a too-small sample and a contaminated test corpus; the numbers below are from the corrected re-run, not the first attempt.
The headline number, reported the way we’d want to read it
Across the 21 of 32 tasks where both methods actually found the right file, graph-based context retrieval used 43.1% fewer tokens on average (median 45.3%). That is the real, contracted figure. It is also not a stable number: the spread ran from a 32% loss on one task to a 92% saving on another, with a standard deviation of 40.3, nearly as large as the mean itself. Four of those 21 tasks were outright token losses for the graph.
We are not going to round that off into a clean marketing percentage, and we are not going to call it statistically significant. It isn’t: n=21 is a directional sample from one codebase at one point in time, not a controlled trial at scale. The honest summary is: graph lookup is usually cheaper, sometimes dramatically so, occasionally more expensive, and the size of the win depends heavily on what kind of file you’re looking for.
Where graph-based context retrieval for AI coding wins big, and where it’s a wash
Broken out by file type, the pattern is consistent and explainable:
- Documents: close to a wash, averaging +3.5%. Both methods often end up reading the same large document anyway, so the discovery-step savings are small relative to that shared cost.
- Agent definitions: +48.7%.
- Source files and scripts: +66.6%. Unaided search either dumps a large, noisy candidate set or misses the target file entirely here.
- Skills: +86.2%, the largest category win.
- Reference library docs: +57.0%.
And by graph metadata quality: where a node carries only weak, auto-generated description text, the graph’s advantage shrinks to +18.8%. Where the metadata is richer, it widens to +50.7%. That is exactly the failure mode we expected going in, the graph is a curated index, not semantic search, so it is only as good as the metadata behind it, and we deliberately seeded the task set with weak-metadata cases instead of hiding them.
Correctness and AI agent token savings, not just speed
Token count is only half the question. We also measured whether each method actually found the right file at all, and how much of what it retrieved was relevant rather than noise.
- Correctness: unaided search reached the correct file 68.8% of the time (22 of 32 tasks); graph lookup reached it 90.6% of the time (29 of 32). This was measured under a single-retry search protocol; a person reformulating a failed query by hand would likely narrow the gap somewhat.
- Relevance, blind-graded: on the headline quality metric, token-weighted precision, the graph spent 92.4% of its retrieved tokens on genuinely relevant content, versus 42.8% for unaided search, a 2.2x gap across all 32 tasks, narrowing to 1.65x on the tasks where both methods succeeded.
That second number is, frankly, the more decisive one. Unaided search is not just slower at finding the right thing. Most of what it pulls into the agent’s context on the way there is noise.
Where the graph genuinely lost
We are not interested in averaging away the cases that don’t flatter the result. On one task, the graph’s primary query matched three other documents from the same family, other specs covering the same subsystem, which suppressed the fallback query before it could fire. The correct file never made it into the candidate set, and the graph never found it. That said, the failure here was not a token blowout: the graph spent roughly 37,000 tokens reaching the wrong answer, versus about 58,000 for unaided search to reach the right one on the same task. So this loss is a correctness miss, not a cost overrun, which matters if you’re deciding which failure mode you’d rather have.
On two other tasks, neither method found the target. One was a file literally named after one concept (network-topology) whose actual content and description used a different term (integration topology), so both the graph and unaided search matched the wrong files and the query that would have worked never got tried. That is a name-versus-content drift problem neither tool solves on its own, and it is a more useful failure to understand than a flattering win.
A separate, structural point worth being honest about: no node in our graph currently carries tags. Every “rich metadata” label in this measurement means a detailed description with no tags attached, not the fuller curation we’d eventually like. A more thoroughly tagged graph could shift these numbers in either direction, and we have not re-run the test against one yet.
Why the result is dispersed, and what it means for AI context window efficiency
A meaningful share of the gap comes from a structural quirk we did not fully anticipate: spaced, multi-word search phrases (the kind a person actually types) tend to return zero matches against the graph’s hyphenated node identifiers, which forces the graph onto a single-token fallback query. Unaided full-text search, by contrast, often matches something on that same spaced phrase, which suppresses its own fallback and either dumps a huge candidate list or misses the target outright. Some of the measured gap is genuinely “the graph’s curated path tokens are easier to match,” not pure semantic superiority, and that distinction matters if you’re trying to predict how this generalizes to a different codebase or a different way of phrasing a query.
What the Dependency Relationship Graph is, and what it explicitly is not
/drg-lookup is substring matching over a curated metadata index, node id, file path, tags, and a short description, not embeddings and not vector search. That is a deliberate design choice: it is fast, it is auditable, and a human can read exactly why a result matched. It is also why metadata quality matters as much as the data above shows. This is one specific implementation of graph-based context retrieval for AI coding agents, not a claim that graph lookup beats every retrieval method in every setting. If you’re weighing this against embeddings or RAG, we cover that broader comparison separately.
The underlying graph-lookup technique, resolving an artifact by querying a relationship graph instead of scanning a workspace, is part of the architecture covered by the same provisional patent application behind our autonomous build system (specifically, the discovery-subsystem claims in that filing). We are precise about that scope on purpose: those claims describe graph-based artifact resolution within an autonomous build, where each phase needs to find what a prior phase produced. The broader, always-on lookup tool measured in this post, spanning an entire project’s documents, agents, and skills, uses the identical core technique in a wider application than what the filing’s claims cover. Same mechanism, and we’d rather state the scope precisely than let “patent pending” do more rhetorical work than the filing actually does.
The numbers we’re standing behind, and the ones we’re not
To be specific about what this measurement does and does not support:
- It does support: graph-based lookup finds the correct file more often than unaided search, and what it retrieves is substantially more relevant per token, with real margins (90.6% vs 68.8% correctness; 2.2x token-weighted precision).
- It supports, with caveats: a typical per-lookup token saving in the neighborhood of 43%, understanding that the spread around that number is wide and four tasks out of 21 lost.
- It does not support: a flat “DRG saves X% of tokens” claim with no qualification, a whole-session savings figure, or an exact token count rather than a character-based estimate (we did not have access to a real tokenizer cross-check for this run, and we say so in the underlying data rather than rounding it away).
Why we’re publishing the messy version
We could have led with the 2.2x relevance number and left it there, it’s the strongest, most stable finding in the data. We are including the dispersed token result anyway, four losses and all, because a measurement you only publish when it’s flattering isn’t a measurement, it’s marketing with a citation. The same honesty standard we apply to governance claims in Enterprise Intelligence applies here: if we’re not willing to show you the task where it lost, we shouldn’t be telling you about the tasks where it won.
See how your agents are finding context today
If your AI coding agents are greping your repository and reading whole files to find what they need, that is the exact tax this measurement quantifies. See what a graph-based context layer looks like against your own codebase in a 30-minute discovery call at encephalon.net/book.
