The output length is min(k, n_allowed), where n_allowed counts distinct allowed vectors — when fewer vectors are allowed than k you get exactly that many results rather than padded fallbacks.
Drop-in replacements for the in-tree reference vector / document stores in each framework. Same public surface, same persistence semantics, same retriever and pipeline wiring — swap the import and keep your pipeline.
cargo add turbovec
use turbovec::TurboQuantIndex;
let mut index = TurboQuantIndex::new(1536, 4).unwrap();
index.add(&vectors);
let results = index.search(&queries, 10);
index.write("index.tv").unwrap();
let loaded = TurboQuantIndex::load("index.tv").unwrap();
For stable external ids that survive deletes:
use turbovec::IdMapIndex;
let mut index = IdMapIndex::new(1536, 4).unwrap(); index.add_with_ids(&vectors, &[1001, 1002, 1003]).unwrap(); let (scores, ids) = index.search(&queries, 10); index.remove(1002); index.write(“index.tvim”).unwrap(); let loaded = IdMapIndex::load(“index.tvim”).unwrap();
Recall
TurboQuant vs FAISS IndexPQ (LUT256, nbits=8) — the paper’s Section 4.4 baseline. 100K vectors, k=64. FAISS PQ sub-quantizer counts sized to match TurboQuant’s bit rate (m=d/4 at 2-bit, m=d/2 at 4-bit).
The charts plot calibrated TurboQuant (TQ+). Across OpenAI d=1536 and d=3072, TQ+ beats FAISS at R@1 on three of four cells (by 0.9–2.9 points; d=1536 4-bit trails by 0.7), and both reach 1.0 by k=8 (≥0.997 already at k≤4). GloVe d=200 is the harder regime — at low dim the asymptotic Beta assumption is looser. TQ+ lands ahead of FAISS at R@1 at both bit widths (+1.9 at 4-bit, +0.8 at 2-bit), with FAISS keeping a slim edge at 2-bit from k≈8. Uncalibrated numbers are in the JSONs (tq_recalls).
A note on baselines. We compare against FAISS IndexPQ (LUT256, nbits=8, float32 LUT) because it’s the default production-grade PQ most users would reach for. This is a stronger baseline than the custom u8-LUT PQ in the TurboQuant paper — FAISS uses a higher-precision LUT at scoring time and k-means++ for codebook training. We reproduce the paper’s TurboQuant numbers on OpenAI d=1536 / d=3072 and hit similar numbers to other community reference implementations on low-dim embeddings (see turboquant-py at d=384). On GloVe (d=200) — the low-dim regime where the asymptotic Beta assumption is loosest — TurboQuant lands ahead of FAISS at 4-bit but trails it at 2-bit; TQ+ calibration recovers the 2-bit deficit at R@1 (0.572 vs FAISS’s 0.564), with FAISS keeping a slim edge at deeper k.
Full results: d=1536 2-bit, d=1536 4-bit, d=3072 2-bit, d=3072 4-bit, GloVe 2-bit, GloVe 4-bit.
All benchmarks: 100K vectors, 1K queries, k=64, median of 5 runs.
On ARM, TurboQuant beats FAISS FastScan in every config, averaging 3.5× at 4-bit (3.4–3.7× across cells — the SDOT/SMMLA dot-product kernels score the vector-major layout directly) and 26% at 2-bit (22–29%).
On x86, TurboQuant wins every config, averaging 3.4× at 4-bit (3.2–3.5× across cells — the AVX-512 VNNI dot-product kernel on the vector-major layout) and 20% at 2-bit (5–32%), where the vpermb LUT scan carries the short 2-bit accumulate loop.
Same corpus as the search cells: 100K OpenAI vectors, median of 5 runs, timed loops including the Python-call overhead a caller actually pays per op. Insertion measures per-vector add() latency on a warm, populated index (built untimed) at n=1 — a single-vector add() — and n=100 — a 100-vector batch, showing how far batching amortizes the per-call overhead — against add() into the trained, populated FAISS IndexPQFastScan (training untimed). A single add() lands in 6.3–19.7 µs depending on the cell (7.6–13.9× faster than a FAISS single add), and a 100-vector batch amortizes TurboQuant to 4.6–16.3 µs/vector (4.6–15.1× faster than the same batch into FAISS). Removal measures per-op remove-by-id latency at n=1 (the steady per-op rate over 1000 removes) and n=100 (the first 100 removes on a fresh index): IdMapIndex.remove(id) — O(1) swap-and-pop plus the id-map bookkeeping — lands at 0.44–1.22 µs and 0.59–1.37 µs per op across the cells. The FAISS column is the same user-visible operation, remove_ids on an IndexIDMap over IndexPQFastScan, which repacks the stored codes on every call: 0.19–1.02 s per single remove at 100K, with cost doubling alongside code size — which is why the removal charts use a log-scale axis. Charts show the single-threaded cells (RAYON_NUM_THREADS=1); the _mt cells are measured too and match at n=1, since a single add is serial. Scripts: benchmarks/suite/.
Full results: d=1536 2-bit insert, d=1536 4-bit insert, d=3072 2-bit insert, d=3072 4-bit insert, and the matching speed_remove_* and _mt files.
Full results: d=1536 2-bit insert, d=1536 4-bit insert, d=3072 2-bit insert, d=3072 4-bit insert, and the matching speed_remove_* and _mt files.
Same corpus as the search cells: 100K OpenAI vectors, median of 5 runs. TurboQuant serializes to a single .tv file with an fsync + atomic rename; FAISS is write_index / read_index on the precision-matched IndexPQFastScan (sub-quantizer count matched to TurboQuant’s bit rate, as in the search cells). Save (warm) is a write after a search has run, so the blocked layout cache is populated. Load → first search opens a fresh index and times the first query — separating bare deserialization (the page cache is warm throughout, so this is layout work, not cold-storage I/O) from the first-query cost. Round-trip chains the checkpoint/resume cycle an embedding store actually pays — mutate 1K vectors → save → reopen → serve the first query; FAISS has no measured equivalent for this path, so it is shown for TurboQuant only. On the smaller payloads the round-trip can come in below the isolated post-mutation (“dirty”) write: the two are timed in separate suite steps, and at small file sizes the standalone fsync in the dirty-write step dominates and inflates it — a measurement artifact of the harness, not a repack win in the combined path. Single-threaded cells pin RAYON_NUM_THREADS=1. Scripts: benchmarks/suite/.
Full results: d=1536 2-bit persist ST, MT, d=1536 4-bit persist ST, MT, d=3072 2-bit persist ST, MT, d=3072 4-bit persist ST, MT.
Full results: d=1536 2-bit persist ST, MT, d=1536 4-bit persist ST, MT, d=3072 2-bit persist ST, MT, d=3072 4-bit persist ST, MT.
Each vector is a direction on a high-dimensional hypersphere. TurboQuant compresses these directions using a simple insight: after applying a random rotation, every coordinate follows a known distribution – regardless of the input data.
1. Normalize. Strip the length (norm) from each vector and store it as a single float. Now every vector is a unit direction on the hypersphere.
2. Random rotation. Multiply all vectors by the same random orthogonal matrix. After rotation, each coordinate independently follows a Beta distribution that converges to Gaussian N(0, 1/d) in high dimensions. This holds for any input data – the rotation makes the coordinate distribution predictable.
3. Per-coordinate calibration (TQ+). The Beta distribution from step 2 is asymptotic — at finite dimensions, individual coordinates drift from the canonical shape (especially low-bit and word-vector-style embeddings). TQ+ fits two scalars per coordinate — a shift and a scale — mapping each coordinate’s empirical quantiles onto the codebook’s outermost centroids. The probability level comes from the codebook, so it tracks the bit width (~0.933 at 2-bit, ~0.996 at 4-bit) rather than being fixed. The Lloyd-Max codebook then quantizes against the target distribution it was designed for. The fit is explicit: call index.calibrate(sample) once with a random, representative sample of your vectors (~1024 rows is enough — a draw of that size matches fitting on the whole corpus) before adding; afterwards the calibration is committed and reused by every add — no retraining, no rebuilds, no separate train phase. An index you never calibrate is plain TurboQuant. index.calibration_state reports "uncalibrated" or "calibrated". Recall gain: up to +2.2pp at @1 on the cells that drift most (e.g. GloVe at 2-bit).
4. Lloyd-Max scalar quantization. Since the distribution is known, we can precompute the optimal way to bucket each coordinate. For 2-bit, that’s 4 buckets; for 4-bit, 16 buckets. The Lloyd-Max algorithm finds bucket boundaries and centroids that minimize mean squared error. These are computed once from the math, not from the data.
5. Bit-pack. Each coordinate is now a small integer (0-3 for 2-bit, 0-15 for 4-bit). Pack these tightly into bytes. A 1536-dim vector goes from 6,144 bytes (FP32) to 384 bytes (2-bit). That’s 16x compression.
6. Length-renormalized scoring. Scalar quantization systematically underestimates inner products — the reconstructed unit direction is a little shorter than the original. We compute one scalar per vector at encode time — the inner product of the rotated unit vector with its own centroid reconstruction — and store ||v|| / ⟨u, x̂⟩ alongside each compressed vector. The search kernel multiplies the per-candidate score by this scalar before heap insertion, turning the inner-product estimator from downward-biased into unbiased at zero search-time cost and zero extra storage. The recall gain shows up most at low bit widths, where the quantization shrinkage is largest.
Encoding cost: one extra d-dimensional dot product per vector to compute ⟨u, x̂⟩. On 1M vectors at d=1536 this is sub-second of additional encode time — a one-shot price paid at ingest, not at query.
Search. Instead of decompressing every database vector, we rotate the query once into the same domain and score directly against the codebook values. The scoring kernel uses SIMD intrinsics (NEON on ARM; AVX-512BW on modern x86, falling back to AVX2, then to a scalar path on pre-AVX2 CPUs) with nibble-split lookup tables for maximum throughput.