Company
Date Published
Nov. 8, 2021
Author
Zilliz
Word count
3015
Language
English
Hacker News points
None

Summary

In the era of Big Data, unstructured data represents roughly 80-90% of all stored data. Traditional analytical methods fail to pull out useful information from these growing data lakes. To address this issue, researchers are focusing on building general-purpose vector database systems that can handle high-dimensional vector data and support advanced query semantics. The article discusses the design and challenges faced when building such a system, including optimizing cost-to-performance ratio relative to load, automated system configuration and tuning, and supporting advanced query semantics. It also introduces Milvus, an AI-oriented general-purpose vector database system developed by Zilliz's Research and Developement team.