LangSmith is a platform for testing and debugging LLM-powered applications, and its new production logging and automations features aim to make it easier for developers to capture and act on user data in real-time. The platform now supports advanced filtering of runs based on latency, errors, feedback, metadata, and full text search, allowing users to efficiently inspect their data. Monitoring provides aggregate statistics over time, including LLM-specific metrics, and interactive charts enable easy exploration of the data. Threads allow for viewing entire conversations in a single view, while automations enable automated actions based on filtered datapoints, such as adding runs to datasets or annotation queues, and online evaluation allows for efficient human-in-the-loop feedback collection. These features are designed to help developers improve their applications by leveraging user feedback and data insights.