An Overview on RAG Evaluation

Retrieval Augmented Generation (RAG) is a popular application of Large Language Models (LLMs) and Vector Databases that involves augmenting inputs to an LLM with context retrieved from a vector database like Weaviate. RAG applications are commonly used for chatbots and question-answering systems. Evaluating the performance of RAG is crucial, and it involves three components: indexing, retrieval, and generation. Recent advances in using LLMs to evaluate RAG systems have accelerated their development. This article presents an overview of RAG metrics, tunable knobs, experiment tracking, and the transition from RAG to Agent Evaluation.