Claude Code Never Used a Vector DB? Not Quite — But the Bigger Trend Is Real
We turned ours off too a couple of months ago. It wasn't broken. The model just got better at searching than our retrieval was.
Toffu's memory used to be the standard setup. Everything a customer told us, and every file they uploaded, got broken into small "memories," embedded and stored in Pinecone.
Before each message we ran a similarity search and put the top 3 matches into the prompt, whether the agent asked for them or not.
It worked well for a long time. Back then models weren't good at deciding what to look up, so we did it for them.
That's no longer true. Today's models are good at searching when you give them the right tools, so we let the agent decide:
- What it must always know (brand, goals, rules) lives in the customer's instructions, which the agent always has.
- Working notes are saved by name and found by keyword, not by similarity score.
- Past conversations and uploaded files are searched with plain keyword queries.
- Live numbers come straight from the ad accounts, never from a stored snapshot.
No embeddings, no index to keep in sync, no memories pulled in on a hunch. Every lookup shows up in the trace, so you can see exactly what the agent searched for and why.
We weren't first. Claude Code shipped with no vector index at all, only grep and file reads, and the Anthropic team said agentic search beat RAG in their testing.
Cursor just followed. From their forum: "models have gotten really good at using grep / indexed search, so the older dedicated semantic search path was rapidly no longer helping in a meaningful way." They no longer compute or store embeddings of your code.
Vector DBs weren't a mistake. They made up for something models couldn't do yet. When the model gets better, check what infrastructure you built around its old weaknesses. Some of it is now in the way.
A pattern is showing up across multiple AI coding and agent products: teams that built retrieval-augmented generation (RAG) with vector embeddings are finding that, as models get better at using search tools directly, the dedicated vector index adds less value than it used to.
Toffu is one example — their memory system used to embed every customer message and uploaded file into Pinecone, running a similarity search before each message and injecting the top 3 matches regardless of whether the agent needed them. That worked when models weren't good at deciding what to look up themselves. According to the company, that's no longer the bottleneck: today the agent decides what to search, using a mix of always-available instructions (brand, goals, rules), keyword-searched working notes, keyword-searched past conversations and files, and live data pulled directly from source systems rather than stored snapshots — with every lookup visible in the trace.
Claude Code followed a similar arc, though not from a standing start: Anthropic's team built RAG with a local vector database into early versions of Claude Code, tested it head-to-head against agentic search (grep, glob, file reads, iterative refinement), and found agentic search "outperformed everything, by a lot" — surprising enough that they dropped the vector DB entirely in favor of the simpler approach.
Whether Cursor has made an equivalent move is less clear. Cursor's own public research (as of their November 2025 blog post) describes the opposite emphasis: they built a custom-trained embedding model specifically for their coding agent and found that combining it with grep produces meaningfully better results than either alone, with the improvement holding across frontier models and growing on larger codebases.