Choosing the Right Embedding Model for Your Data

Retrieval Augmented Generation (RAG) is an approach in Generative AI that utilizes data to enhance the knowledge of Language Learning Model (LLM) generators, such as ChatGPT. RAG consists of two LLMs: embedding and generator models, both used in inference mode. The HuggingFace MTEB leaderboard provides a comprehensive list of text embedding models, where users can filter by language or specialty domain like law. Users should be cautious when selecting models as some may be overfitted, resulting in deceptively high rankings. ResNet50 is a popular Convolutional Neural Network (CNN) model for image data and PANNs are commonly used embedding models for audio data. Multimodal embedding models like SigLIP or Unum can handle text, image, audio, or video data simultaneously. For multimodal applications involving sound or video, a generative LLM is often employed to convert the input into text before using RAG techniques.