What's new in pgvector v0.7.0

The latest version of pgvector introduces several approaches to leverage redundancy in real-world embedding datasets, leading to memory and performance savings with minimal impact on precision. These include float16 vector representation, sparse vectors, bit vectors, and new distance functions such as L1, Hamming, and Jaccard distances. The use of half vectors (float16) can reduce memory consumption by 50%, while sparse vectors save significant storage space for vectors with many zero components. Bit vectors allow for fast pre-selection from a dataset before performing an additional search within the subset. These improvements have led to over 100x speedup compared to one year ago, making pgvector more efficient and effective in handling real-world embedding datasets.