Vector Search Architecture Beyond the Demo

AI Engineering

I model production vector search as two systems joined by a contract: an indexing plane that maintains searchable evidence and a query plane that retrieves authorized, relevant results.

The demo hides the hard contracts

A demo embeds ten documents, performs nearest-neighbour search, and prints five passages. Production must answer harder questions: which version of a document is searchable, who may see it, how deletion propagates, whether a filtered query preserves recall, and how a new embedding model is cut over without mixing vector spaces.

Dense retrieval is valuable because it can match semantic similarity beyond exact terms. The original retrieval-augmented generation paper combined generation with retrieved non-parametric memory, but retrieval remains only one subsystem. A fluent answer can still misuse a relevant passage; retrieval quality and generation quality must be evaluated separately.

I start with the retrieval contract: input query and identity context; eligible corpus and freshness target; returned identifiers, passages, scores, and versions; latency percentile; and the metric by which relevance will be judged.

Indexing is a governed data product

The indexing plane starts with authoritative sources, not arbitrary files. Every item needs a stable source ID, tenant or entitlement metadata, document version, effective time, deletion state, and provenance. Normalize formats, but retain a link to the original evidence. Chunking should follow semantic boundaries where possible and record parent-child relationships so a retrieved fragment can be interpreted in context.

Embedding records require at least the embedding model/version, dimensions, normalization rule, chunker version, and creation time. Vectors produced by different models generally do not share a meaningful distance space. Never silently write a new model’s vectors into an old index.

Treat reindexing as a migration. Build a shadow index, replay changes into both old and new versions, evaluate them against the same judgments, then switch an alias or routing rule. Keep a rollback window and verify that deletions and entitlement changes reached both copies. Freshness is an observable service level: measure source-to-search lag and deletion lag, not merely job success.

Query-time retrieval is more than ANN

At query time, normalize the query using the same contract as the index and carry caller identity into retrieval. Authorization should constrain eligibility before results leave the search boundary. Post-filtering an unrestricted top-k can leak metadata and destroy recall when most candidates are forbidden.

Semantic search should usually work alongside lexical retrieval. Exact identifiers, error codes, account products, and regulatory phrases often favour term-based methods; paraphrased questions favour embeddings. Hybrid fusion—such as reciprocal-rank fusion—combines ranks without pretending incomparable raw scores share a scale. A reranker can then use richer query-document interactions on a small candidate set.

Approximate nearest-neighbour indexes exchange perfect recall for speed and memory. The HNSW paper describes a multilayer graph approach with tunable search complexity. The operational consequence is not “HNSW is fast”; it is that build parameters, query exploration, filters, data distribution, and hardware jointly determine recall and latency.

The pgvector documentation distinguishes exact search from HNSW and IVFFlat and documents filtering, iterative scans, and recall measurement. Its repository showed version 0.8.2 when this article was researched on 19 July 2026; re-check the current release and behaviour before implementing. Product-specific defaults are not universal benchmarks.

Measure recall against a trustworthy reference

For a labelled query set, run exact search or a high-quality exhaustive reference and compare the approximate result set. Report recall@k or another named metric alongside p50/p95/p99 latency. Record:

• corpus size and vector dimensions;
• distance metric and vector normalization;
• index type and parameters;
• candidate and final k;
• filter selectivity and tenant distribution;
• hardware, concurrency, cache state, and software version.

Without those conditions, “10 ms search” is marketing, not engineering evidence. Evaluate by slice: short versus long queries, rare vocabulary, fresh documents, languages, tenants, and restrictive filters. Also measure end-to-end relevance such as nDCG or judged precision, because high ANN recall can faithfully retrieve the wrong content if the embedding or chunks are poor.

A failure scenario: the invisible tenant regression

Here is how I would test the filter boundary with a synthetic case. Imagine an index with one million chunks. An entitlement filter leaves only 0.2% eligible for a particular team. The ANN engine finds globally close neighbours, then a post-filter removes almost all of them. Latency appears healthy, but the team receives empty results.

Possible remedies include pre-filter-aware indexing, partitioning by a stable tenancy boundary, iterative scanning with a bounded budget, or retrieving a larger candidate pool. Each changes cost and recall. The correct choice depends on tenant size distribution and isolation requirements; it cannot be inferred from unfiltered benchmarks.

The same scenario shows why access control belongs in the architecture, not in a prompt. Cache keys, logging, reranking, and answer generation must preserve the authorization context. Deletion and revocation need tested end-to-end propagation.

Operate retrieval as a versioned service

Expose the chosen index, embedding, chunker, lexical configuration, filter policy, fusion method, and reranker version in traces. Monitor empty-result rate, source freshness, deletion lag, filter selectivity, score distribution, exact-versus-ANN recall samples, and human relevance feedback. Alert on slice regressions rather than one global average.

My design rule is simple: keep indexing and query flows separate, join them through a versioned retrieval contract, and test every optimization against exact search and user relevance. That is how vector search graduates from an impressive demo to an evidence service that can be upgraded, audited, and rolled back.

Related reading

• AI Agents in Regulated FinTech: Control Before Autonomy
• RAG for Financial Operations: Evidence Over Eloquence

References

• Malkov and Yashunin — Efficient and Robust Approximate Nearest Neighbor Search Using HNSW
• pgvector — official documentation
• Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks