/plushcap/analysis/incident-io/finding-relationships-in-your-data-with-embeddings

Finding relationships in your data with embeddings

What's this blog post about?

The text discusses how vector embeddings can be used to find relationships in data and highlights a project that utilized embeddings to power incident-related features. It explains the concept of embeddings as an array of numbers representing a model's interpretation of a given block of text, and how they can be used for search, clustering, recommendations, and anomaly detection. The author shares their experience in using OpenAI's API for generating embeddings and discusses prompt engineering, measuring prompt effectiveness, and running the feature in production. They also cover storing embeddings using Postgres extension pgvector and handling prompt and model changes. Finally, they share how the concept of linking incidents together proved valuable and inspired users to manually do it themselves.

Company
Incident.io

Date published
Jan. 19, 2024

Author(s)
Rob Liddle

Word count
2975

Hacker News points
None found.

Language
English


By Matt Makai. 2021-2024.