Use OpenAI CLIP, LangGraph, & RAG to Generate Competitive Restaurant Insights

In this article, we explored how OpenCLIP embeddings can be used in combination with Deep Lake and LangChain for extracting insights from restaurant reviews. We started by preparing the data using Scrapy to scrape the Yelp website, then processed it into a suitable format for ingestion into Deep Lake. After that, we utilized OpenCLIP embeddings to categorize images based on their content, specifically focusing on food items. Finally, we used LangChain and LangGraph to create an agent group with an agent supervisor to delegate tasks between independent agents, orchestrating interactions and workflows efficiently. The results showed that the OpenCLIP embeddings are surprisingly accurate in not just recognizing food in general but also the particular dish. In combination with Deep Lake, it then provides valuable insights into the restaurant reviews and can help you better imagine what people enjoy eating in your neighbourhood. This can be especially helpful if the dining options are rich and it’s difficult to check restaurants one-by-one. The biggest weakness, however, is the data preparation process which is highly time-consuming. To scrape, process and ingest data from 130 restaurants, the total runtime was around 4.5 hours, nevertheless, there are still ways to make this faster and more efficient. Overall, we see that the OpenCLIP embeddings are very powerful and can be very useful for LangChain as well, even though there is currently no integration. The highest potential we see in the unsupervised categorization and image search by text without any context, which as you could see worked pretty well and is far from being limited only to restaurant data. Additionally, LangGraph played a significant role in enhancing our capabilities. It provided a framework for building stateful, multi-actor applications with LLMs, allowing us to coordinate multiple chains across multiple steps of computation in a cyclic manner. This facilitated efficient task distribution and coordination, particularly through the use of Agent Supervisor, which delegated tasks between independent agents within the system, orchestrating interactions and workflows effectively. We hope that you find this article interesting and useful for your future projects and hopefully see you next time.