@@ -69,6 +69,68 @@ For larger datasets, create an IVF-PQ (Inverted File with Product Quantization)
6969
7070IVF-PQ indexes are stored alongside data in R2 and loaded on first query.
7171
72+ ### HNSW
73+
74+ For datasets where you need fast approximate nearest neighbor search with high recall, build an HNSW (Hierarchical Navigable Small World) index:
75+
76+ ``` typescript
77+ import { HnswIndex } from " querymode"
78+
79+ // Build index
80+ const index = new HnswIndex ({
81+ dim: 128 ,
82+ metric: " cosine" ,
83+ M: 16 , // max connections per node (default: 16)
84+ efConstruction: 200 , // construction beam width (default: 200)
85+ })
86+
87+ // Add vectors one at a time
88+ index .add (0 , vec0 )
89+ index .add (1 , vec1 )
90+
91+ // Or batch add from a contiguous Float32Array
92+ const allVectors = new Float32Array (1000 * 128 ) // 1000 vectors, 128 dims
93+ index .addBatch (allVectors , 128 )
94+
95+ // Search
96+ const { indices, scores } = index .search (queryVec , 10 , /* efSearch */ 50 )
97+ // indices: Uint32Array of nearest neighbor IDs
98+ // scores: Float32Array of distances (lower = more similar)
99+ ```
100+
101+ #### HNSW tuning
102+
103+ | Parameter | Default | Effect |
104+ | -----------| ---------| --------|
105+ | ` M ` | 16 | Higher = better recall, more memory. 12-48 typical. |
106+ | ` efConstruction ` | 200 | Higher = better index quality, slower build. 100-400 typical. |
107+ | ` efSearch ` | topK | Higher = better recall at query time, slower search. Set to 2-4x topK. |
108+
109+ #### Serialization
110+
111+ HNSW indexes can be serialized to binary for storage (R2, disk) and deserialized on load:
112+
113+ ``` typescript
114+ // Save
115+ const binary: ArrayBuffer = index .serialize ()
116+ await bucket .put (" indexes/embeddings.hnsw" , binary )
117+
118+ // Load
119+ const data = await bucket .get (" indexes/embeddings.hnsw" )
120+ const restored = HnswIndex .deserialize (await data .arrayBuffer (), " cosine" )
121+ const results = restored .search (queryVec , 10 )
122+ ```
123+
124+ ### IVF-PQ vs HNSW
125+
126+ | | IVF-PQ | HNSW |
127+ | --| --------| ------|
128+ | ** Speed** | Fast (quantized distances) | Fast (graph traversal) |
129+ | ** Memory** | Low (compressed codes) | High (full vectors + graph) |
130+ | ** Recall** | Good with enough probes | Excellent |
131+ | ** Build time** | Requires training (k-means) | Incremental (add one at a time) |
132+ | ** Best for** | Large datasets (>1M vectors) | Medium datasets (<1M vectors) |
133+
72134## Combining with filters
73135
74136Vector search composes with all other DataFrame operations:
0 commit comments