The Haystack US conference brought together search engineers and adjacent enthusiasts to discuss the latest advancements in search technology, particularly those related to Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs). The conference highlighted various approaches to testing RAGs and semantic search implementations, including novel methods for evaluating results. The importance of relevance was also emphasized, with several presentations discussing ways to improve ranking across data sources. The event featured talks on multimodal search, LLM-powered features like query understanding and reranking, and the challenges of fine-tuning models for scale. Multiple speakers discussed the benefits and limitations of using RAG and LLMs in search systems, including the need for proper framing, testing, and evaluation to ensure accurate results. The conference also explored the concept of multi-agent workflows, which can improve the cost and performance of LLMs by breaking down complex problems into narrow subtasks.