skip to note
back to notes

apr 18–19, 2026 · atlas

unseen corpora or it doesn’t count

8 corpora · 6 unseen during authoring

atlas research draft discussing results across unseen corpora
atlas research draft discussing results across unseen corpora
timestamp
on apr 19, the research draft checked whether the result held on repositories atlas had never seen.
benchmark setup
i mapped symbols, calls, and tests, then compared the agent against plain text tools before trusting the graph.
suspicious result
the first result looked too good, so i gave the judge code tools and added unseen corpora before trusting it.
the pattern
i turn an idea into a tool, measure it, look for the embarrassing counterexample, and keep whatever survives.

tags

  • #code intelligence
  • #evals
  • #sqlite graph
  • #mcp