/plushcap/analysis/neon/neon-easy-embeddings-indexing-pipelines-with-redpanda-and-neon

Easy Embeddings Indexing Pipelines with Redpanda and Neon

What's this blog post about?

This blog post discusses how to set up an embeddings indexing pipeline using Redpanda and Neon without writing any code. The process involves reading data from Redpanda, computing embeddings using Ollama, and writing output into Neon for efficient storage and querying of embeddings. Redpanda's compatibility with the Apache Kafka® protocol ensures smooth data ingestion and handling, while Neon's serverless Postgres architecture and pg_vector extension provide efficient storage and querying of embeddings. This setup simplifies the process of building indexing pipelines, making them scalable and high-performing for various applications such as e-commerce transactions or customer reviews.

Company
Neon

Date published
Sept. 6, 2024

Author(s)
Tyler Rockwood

Word count
563

Hacker News points
None found.

Language
English


By Matt Makai. 2021-2024.