HNSW+PQ - Exploring ANN algorithms Part 2.1

Weaviate has introduced vector compression algorithms in its latest version v1.18, aiming to offer similar performance at a fraction of memory requirements and cost. The main goal is to balance recall performance and memory management. Product Quantization (PQ) is the chosen compression algorithm for vectors. Experiments on datasets like Sift1M, Gist, and DeepImage96 show that PQ can significantly reduce memory usage while maintaining acceptable recall rates and latency. Weaviate's HNSW+PQ feature allows HNSW to work directly with compressed vectors, improving memory efficiency without sacrificing performance.