Deploying Kafka Streams and KSQL with Gradle – Part 3: KSQL User-Defined Functions and Kafka Streams

In this three-part series, we discussed building an event streaming architecture using Apache Kafka, KSQL, and Kafka Streams for a customer project. We also explored how Gradle helped address the challenges faced during development, building, and deploying the KSQL portion of our application. Additionally, we delved into using Gradle to build and deploy KSQL user-defined functions (UDFs) and Kafka Streams microservices. In part 3, we focused on using Gradle to build and deploy UDFs and Kafka Streams microservices. We utilized the Spock framework for writing test specifications and employed Groovy instead of Java to write our UDFs due to its ability to handle data types more efficiently. Furthermore, we demonstrated how to create distribution artifacts complete with start scripts that can be packaged up as part of the build process using the Application plugin. Throughout this series, we highlighted how Gradle can be used to build all things Kafka with both built-in functionality and plugins contributed by the community. We also introduced the gradle-analytics plugin, which captures information about what's going on inside of Gradle and sends that activity to various cloud services like AWS Kinesis, Amazon S3, Cloud Pub/Sub, or Google Cloud Storage.