Vector search gets you 80% of the way. But agents that need to reason across entities, track relationships, and maintain state over time need something more structured. Knowledge graphs are that missing layer.
Where vector search breaks down
Embedding-based retrieval works beautifully for "find me documents similar to this query." But it fails at questions that require understanding connections between entities:
"Which companies that raised Series B in 2024 are using our competitor's API?"
Requires joining: companies → funding rounds → API usage → competitor relationships
"If we change pricing, which enterprise customers are most likely to churn based on their usage pattern?"
Requires: customer → plan → usage metrics → historical churn patterns → similar profiles
"What's the blast radius if this microservice goes down?"
Requires traversal: service → dependencies → downstream services → SLOs → team ownership
These questions need graph traversal, not cosine similarity. You can't embed a relationship — you can only encode it structurally.
Knowledge graphs as agent memory
When an agent operates over time, it accumulates facts. Without structure, those facts become a flat pile of text that degrades retrieval quality as it grows. A knowledge graph provides:
Entity deduplication
"Anthropic," "anthropic.com," and "the company behind Claude" all resolve to one node.
Typed relationships
Not just "related to" but "employs," "competes with," "depends on," "was acquired by."
Temporal awareness
Facts have timestamps. "CEO of X" can change. The graph tracks current and historical state.
The hybrid architecture: vectors + graphs
The winning pattern isn't either/or. It's a hybrid where vector search handles initial retrieval and the graph handles reasoning:
Fast retrieval of relevant entities and passages. "Find me nodes related to this query."
Starting from retrieved entities, walk relationships. Expand context by 2-3 hops. Gather connected facts.
Agent receives both the raw text chunks and the structured graph context. Can now reason about relationships.
What this looks like in practice
A customer support agent with a knowledge graph can answer: "Has this customer experienced this issue before, and if so, what resolved it last time?" — because it can traverse:
Customer(acme_corp)
→ had_ticket(#4521, "API timeout", resolved: 2024-08)
→ resolution(increased_timeout, worked: true)
→ related_to(service: payments-api)
→ had_ticket(#5102, "API timeout", open: 2025-01)
→ same_service(payments-api)
→ similar_pattern(timeout_after_deploy)
Agent reasoning: "Same service, same pattern.
Suggest the fix from #4521 first."Without the graph, the agent would have to search through thousands of tickets, hope the embeddings surface the right one, and still wouldn't have the structured relationship between the two incidents.
When to invest in a knowledge graph
Not every AI application needs one. Use this decision framework:
Multi-hop reasoning needed. Entities have complex relationships. State changes over time. Agent needs to explain its reasoning chain.
Simple Q&A over documents. Single-hop retrieval sufficient. Entities don't relate to each other. Content is flat/unstructured.
The future of AI agents is structured memory. Vector search was step one. Knowledge graphs are step two. The agents that can reason about relationships — not just retrieve similar text — will be the ones that actually work autonomously.