Knowledge Graph Production Readiness Checklist

None of these checks require exotic tooling: every one of them is a query, a profile, or a written decision you can make this week. What they have in common is that skipping any of them produces a system that demos perfectly and fails quietly in production, usually months after launch, when the person who built it has moved on to the next project.

1.Have you verified permission scoping as both an insider and an outsider?

Run your full test question suite twice: once as a fully authorized user, once as a user deliberately outside the relevant boundary. Diff the two result sets line by line. Any fact that appears in both (including a count, a title, or the bare confirmation that something exists) is a leak, not a passing test. This is the single check most teams skip, because it requires deliberately trying to fail rather than confirming success.

2.Do you know your filter rate before any model call runs?

A meaningful share of any real world corpus (short acknowledgements, boilerplate, system generated noise) carries nothing worth extracting, and none of it needs a model call to identify. Measure the share your pipeline rejects before spending a token. If you don't have this number, you don't know whether you're paying to extract structure from content that was never going to produce one.

3.Have you profiled your highest traffic queries for supernode expansion?

Use your database's query plan tool on the queries that will actually run most often in production, and read the real work performed: not the wall clock time on a small test dataset, which will look fine right up until it doesn't. If the work done is the same order of magnitude as a node's total relationship count, a supernode is being expanded, and the fix is almost always to start the query from the more selective side.

4.Can you mechanically verify every citation in an answer is real?

Every fact placed into the model's context should carry an identifier. The answer generation prompt should require the model to cite that identifier after every claim it makes. Then check the output programmatically: every cited identifier should be traceable back to something that was genuinely in the context. If that check isn't automated and running in CI, it isn't a guarantee: it's a hope.

5.Does a named person own the vocabulary going forward?

Entity and relationship types drift within weeks of launch if nobody is responsible for them: the same real world concept gets tagged three different ways, and every query filtering on one variant silently returns a fraction of what it should. This is a staffing decision, not a technical one, and it needs to be made before launch, not discovered after search quality has already degraded.

6.Have you decided your deletion and erasure policy before loading production data?

Decide up front whether records get anonymised in place or hard deleted, and what happens to anything that references a deleted record. Retrofitting an erasure policy onto a graph that assumed every node was permanent is a materially harder problem than deciding this before the first production row is loaded.