Skip to content

feat(vchordrq): add opt-in e-means for internal index builds - #483

Open
ze33m wants to merge 1 commit into
supervc-stack:mainfrom
ze33m:vchordrq-emeans
Open

ze33m wants to merge 1 commit into
supervc-stack:mainfrom
ze33m:vchordrq-emeans

Conversation

@ze33m

@ze33m ze33m commented Nov 12, 2026

Copy link
Copy Markdown

Summary

Add opt-in e-means training for leaf centroids in internal vchordrq
index builds. Enable it with kmeans_algorithm.e_means = { epochs = 5 }
under [build.internal].

The Rust trainer adapts the mini-batch updates from Faiss 1.16.0 and uses
VectorChord's RaBitQ for centroid assignment.
Paper · Faiss reference

Benchmark

On VectorChord 1.1.1 and PostgreSQL 17.11, five epochs of e-means
reduced CREATE INDEX time from 18.52 s to 13.43 s (27.5%).
Centroid training took 4.46 s versus 9.04 s for Lloyd.

The workload used 167,689 halfvec(4096) embeddings, inner product,
1,637 lists, 16 training threads, and 2,000 queries. Both configurations
were measured once on the same node. Recall@10 against exact float32
search differed by at most 0.23 percentage points across 64, 128, 256,
and 512 probes.

Validation

Separately verified on upstream main at 0768657 with PostgreSQL 17.11.
make build, CREATE EXTENSION, five Rust tests, and three SQL regression
files passed. The new SQL cases cover index creation, nearest-neighbor
search, and rejection of epochs = 0.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@ze33m

ze33m commented Nov 12, 2026

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant