WhyLabs has released the private beta version of LangKit, a purpose-built monitoring solution for Language Learning Models (LLMs). The adoption of proprietary LLMs presents unique observability challenges. LangKit allows continuous evaluation of LLM-powered applications using essential metrics extracted from prompts, responses, and user interactions. It helps detect issues across quality, sentiment, governance, and security aspects. Unlike other monitoring tools, WhyLabs doesn't store user input or model output but gathers all essential metrics about prompt, response data, and user interaction data as it flows through the model. LangKit effortlessly scales from recording just a handful of sample prompts and responses to handling millions of LLM interactions per hour.