Seamless SIEM – Part 2: Anomaly Detection with Machine Learning and ksqlDB

This text discusses building GenAI (Generative Artificial Intelligence) applications faster using the Confluent Platform. It explains how to consume streams of osquery logs, detect anomalous behavior using machine learning, and handle these anomalous events in ksqlDB and the Confluent Platform. The use case involves training an LDA model to learn from osquery logs and deploying it to score real-time logs for anomalous behavior. Subsequent ksqlDB statements are used to route scored events to topics GOOD, BAD, and UGLY. The text also mentions ways to expand the use case, such as publishing bad and ugly logs to a SIEM system or external datastores using Kafka Connect for additional investigation and real-time alerting.