Company
Date Published
April 16, 2024
Author
Lakshan Perera
Word count
997
Language
English
Hacker News points
33

Summary

We're making it super easy to run AI models within Supabase Edge Functions. A new built-in API is available within the Edge Runtime to run inference workloads in just a few lines of code, allowing developers to generate embeddings using models like `gte-small`, store and retrieve with pgvector, and use Large Language Models like `llama2` and `mistral`. This release solves technical challenges for developers who want to generate embeddings from their database content, offloading the compute-intensive task to background workers. Embedding generation uses the ONNX runtime under the hood, a cross-platform inferencing library that supports multiple execution providers from CPU to specialized GPUs. The new API is available today for all Edge Functions users in both local, hosted, and self-hosted platforms, with lower costs compared to proprietary LLMs like OpenAI and Claude, which charge per token or provide APIs to generate text embeddings. We're working with Ollama to make Large Language Models possible with Supabase: local development, self-hosted, and on the platform, with experimental support for `Llama` & `Mistral`.