LangChain + Neo4j Integration Guide: Building Graph-Powered AI Agents

What you'll build

By the end of this guide you will have a working LangChain agent that treats a Neo4j graph as its source of truth. Instead of answering from parametric memory, the agent checks the graph first: it queries Neo4j for the entities and relationships a question touches, pulls back a small set of grounded facts, and only then composes an answer. If the graph has no relevant nodes, the agent says so instead of guessing.

We will build this up in layers rather than jump straight to the most complex version. You will start with a raw connection, move to LangChain's built-in question-answering chain, then replace that chain with hand-written queries you control, and finally wrap those queries as tools an agent can choose to call. Each layer is a legitimate stopping point depending on how much control your use case needs — this mirrors the deeper walkthrough in LangChain + Neo4j: Building Grounded AI Agents That Never Hallucinate, which covers why graph grounding matters before you write any code.

Prerequisites

Before you start, make sure you have:

• Python 3.10 or newer. LangChain's current major version targets modern Python; older interpreters will hit dependency resolution issues.
• The langchain and langchain-community packages installed. The Neo4j graph wrapper and the Cypher QA chain both live in langchain-community, separate from the core langchain package.
• A running Neo4j instance — a local install, a Docker container, or an AuraDB instance — with a database already populated, or at least a schema you can create nodes and relationships against. You will need its Bolt URI, a username, and a password.
• Basic familiarity with Cypher. You do not need to be fluent, but you should be comfortable reading a MATCH ... RETURN pattern, because you will be debugging generated queries and writing your own by Step 3.
• An API key for whichever LLM provider you plan to use with LangChain (the examples below are provider-agnostic).

If you are new to LangChain itself, our What Is LangChain? explainer covers the framework's core building blocks before you dive into the graph-specific pieces here.

Step 1: Connect LangChain to Neo4j

Start by installing langchain-community, which ships the Neo4jGraph class LangChain uses to talk to your database. This class is a thin, opinionated wrapper around the official Neo4j Python driver: it opens a connection, exposes a query method for running Cypher, and maintains a cached copy of your graph's schema so downstream chains know what labels, relationship types, and properties exist without querying the database every time.

Creating the connection is a matter of instantiating Neo4jGraph with three pieces of information: the Bolt URI of your instance (something like bolt://localhost:7687 for a local install, or the neo4j+s:// URI AuraDB gives you), a username, and a password. In practice these three values should come from environment variables rather than being written into source code, since the same connection object gets reused across every chain and agent you build in the steps that follow.

from langchain_community.graphs import Neo4jGraph

graph = Neo4jGraph(
    url=os.environ["NEO4J_URI"],
    username=os.environ["NEO4J_USERNAME"],
    password=os.environ["NEO4J_PASSWORD"],
)

graph.refresh_schema()
print(graph.schema)

The refresh_schema() call is worth understanding early, because it comes up again in Step 5. On construction, Neo4jGraph introspects your database once — reading node labels, relationship types, and property keys — and stores the result as a text description on graph.schema. Every chain that generates Cypher from natural language reads this cached description rather than querying the database schema live, which means the object goes stale the moment you add a new label or property and do not call refresh_schema() again.

Step 2: GraphCypherQAChain

With a working connection, the fastest way to get an end-to-end question-answering flow is GraphCypherQAChain, LangChain's built-in chain for graph databases. The idea is simple: give the chain your Neo4jGraph object and an LLM, and it handles the entire round trip — turning a natural-language question into Cypher, running that Cypher against your graph, and turning the raw rows back into a natural-language answer.

Under the hood this is really two LLM calls stitched together. The chain first prompts the model with your cached graph schema and the user's question, asking it to produce a single Cypher query. It runs that query through graph.query() and gets back a list of records. Then it prompts the model a second time, this time with the question and the query results, asking it to phrase a final answer in plain language.

from langchain.chains import GraphCypherQAChain
from langchain_openai import ChatOpenAI

chain = GraphCypherQAChain.from_llm(
    llm=ChatOpenAI(model="gpt-4o", temperature=0),
    graph=graph,
    verbose=True,
    allow_dangerous_requests=True,
)

response = chain.invoke({"query": "Which vendors supply Acme Corp?"})
print(response["result"])

Two details matter here beyond getting the demo running. verbose=True prints the generated Cypher to your console, which you will want on for every query while you are evaluating this chain — it is the only way to see what the model actually asked the database, as opposed to what you expected it to ask. And note the explicit allow_dangerous_requests=True flag: LangChain requires you to opt in deliberately, because this chain lets an LLM generate and execute arbitrary database queries. That flag is a signal, not a formality — read it as the library telling you this pattern needs a production plan before it needs traffic.

Also remember to call graph.refresh_schema() immediately before building the chain if your graph has changed since the connection was opened. A chain built against a stale schema will confidently generate Cypher that references labels or properties that no longer exist, or miss ones that were just added.

Step 3: Custom retrieval chain

GraphCypherQAChain is the fastest path to a demo, but handing an LLM a live database connection and asking it to write its own queries does not hold up well once real users are asking unpredictable questions. The fix most teams converge on is a custom retrieval chain: instead of generating Cypher at request time, you write and test a small library of Cypher queries in advance, parameterize them, and let the chain select and fill in a query rather than compose one from scratch.

The shape of a custom retrieval chain has three stages. First, an extraction step pulls the structured pieces out of the question — an entity name, a date range, a category — usually with a small, constrained LLM call or a simple parser. Second, a retrieval step takes those extracted values and runs one of your pre-written, parameterized Cypher queries through graph.query(), passing the values as query parameters rather than interpolating them into a query string. Third, a formatting step turns the returned rows into a compact block of context text, which gets passed to the LLM alongside the original question for the final answer.

def retrieve_vendor_context(company_name: str) -> str:
    records = graph.query(
        """
        MATCH (c:Company {name: $company_name})
              -[:SUPPLIED_BY]->(v:Vendor)
        RETURN v.name AS vendor, v.category AS category
        LIMIT 25
        """,
        params={"company_name": company_name},
    )
    if not records:
        return "No vendor relationships found in the graph."
    lines = [f"- {r['vendor']} ({r['category']})" for r in records]
    return "Known vendors:\n" + "\n".join(lines)

Notice that the query text itself is fixed; only company_name varies, and it is passed through the driver's parameter binding rather than string formatting. That single choice is what makes this pattern safe to point at user input in a way auto-generated Cypher never fully is — the query structure the database executes is always one you wrote and reviewed. This is also the pattern behind GraphRAG implementations we ship for clients: known, tested retrieval queries feeding grounded context to the model, rather than a model improvising database access on every request.

Step 4: Agent with graph tools

The custom retrieval chain from Step 3 assumes you already know a question needs the graph. An agent adds a decision on top of that: given a free-form question, should it query Neo4j at all, and if so, which retrieval function should it call? This is the right layer when your assistant handles a mix of questions, some grounded in the graph and some not, and you want the model itself to route between them.

The mechanics are straightforward if you already have retrieval functions like retrieve_vendor_context from Step 3: wrap each one as a LangChain tool with a name, a description, and a typed input schema, then hand the list of tools to an agent along with an LLM. The description is doing more work than it looks — it is the only thing the agent reads to decide whether a given tool is relevant to the question in front of it, so vague or overlapping descriptions are the most common reason an agent picks the wrong tool or skips the graph entirely.

from langchain_core.tools import tool
from langgraph.prebuilt import create_react_agent

@tool
def vendor_lookup(company_name: str) -> str:
    """Look up vendors supplying a given company, using the
    Neo4j knowledge graph. Use this when the question asks
    about suppliers, vendors, or who a company sources from."""
    return retrieve_vendor_context(company_name)

agent = create_react_agent(
    model=ChatOpenAI(model="gpt-4o", temperature=0),
    tools=[vendor_lookup],
)

result = agent.invoke({"messages": [("user", "Who supplies Acme Corp?")]})

The agent decides, per turn, whether the question needs vendor_lookup, needs a different tool, or needs no tool at all. That is the essential difference from Steps 2 and 3: those chains always run their graph query; this agent only runs one when it judges the question calls for it. In production you will typically register several narrow tools like this rather than one broad "query the graph" tool, for the same reason narrow, named tools show up throughout our enterprise knowledge graph consulting work — each one is independently reviewable, loggable, and easy to reason about when something goes wrong. In production, these tools are often exposed through governed MCP servers that add an additional layer of access control between the agent and the systems it can reach.

Step 5: Production considerations

Everything above works in a notebook. Getting it to hold up under real traffic, with real users, takes a handful of operational changes:

• Connection pooling. Do not instantiate a new Neo4jGraph (and therefore a new driver session) per request. Create one connection at application startup and reuse it; the underlying Neo4j driver already pools connections internally, and re-creating it defeats that.
• Query timeouts. Set an explicit timeout on every Cypher call. An unbounded query against a large graph, or one written by an LLM without a LIMIT clause, can hold a connection open far longer than a request budget allows.
• Schema caching. Call refresh_schema() on a deliberate schedule or after known migrations, not on every request. Refreshing per request adds a database round trip to every single call for a schema that rarely changes minute to minute.
• Error handling. Wrap every graph.query() call so a malformed query, a timeout, or a connection drop returns a clear fallback message instead of propagating a raw driver exception back to the user or, worse, back into the LLM's context.
• Cypher injection prevention. Never build a query by interpolating a variable into a string. Pass every user-derived value as a bound parameter, the same way you would use parameterized SQL — this closes off the graph equivalent of a SQL injection attack and is non-negotiable for any query path that touches user input, generated or hand-written.

These are the same hardening steps that separate a working demo from something you can run in an enterprise environment — and they apply whether the graph is feeding a chain, a retriever, or an agent tool. See how we shipped 10 production MCP servers for Optevo to see what production-hardened agent tooling looks like at scale.

Common pitfalls

• Auto-generated Cypher hitting the wrong labels. GraphCypherQAChain generates queries from your schema description, not your data's actual naming conventions. If your graph has both Company and Organization labels for historical reasons, expect the model to guess wrong some fraction of the time — this is a strong argument for consolidating labels before wiring up natural-language querying, not after.
• Missing indexes causing slow queries. A Cypher MATCH on an unindexed property forces Neo4j to scan every node with that label. This is invisible in a demo against a few hundred nodes and becomes the dominant source of latency the moment a graph reaches production scale. Index every property you filter or match on regularly.
• Not refreshing schema after graph changes. A cached Neo4jGraph schema does not know about a label or property added five minutes ago. Chains built on stale schema either omit new fields from generated queries or hallucinate ones that used to exist, and the symptom looks identical to a model reasoning failure until you check when the schema was last refreshed.
• Forgetting to close connections. A Neo4jGraph holds an open driver session for its lifetime. Long-running scripts, notebooks, and serverless functions that create a new connection per invocation without closing the previous one will exhaust the connection pool on your Neo4j instance over time.

For a deeper look at what breaks in graph-backed AI systems once they leave the prototype stage, see The Complete Guide to Intelligent Automation, which walks through how LangChain, Neo4j, and the rest of the automation stack fit together end to end.

Frequently asked questions