Monitoring Apache Spark applications running on Amazon EMR
What's this blog post about?
In this guest post by Priya Matpadi, Principal Engineer at Lookout, we learn about implementing monitoring for a Spark streaming application running on Amazon EMR using Datadog. The author explains why they chose to use Datadog and how they integrated it with EMR. They also provide scripts for installing the Datadog Agent on EMR nodes and configuring the Spark check. Finally, the author discusses monitoring custom application metrics in Datadog and creating a comprehensive dashboard for visibility into the health of their Spark streaming application.
Company
Datadog
Date published
Nov. 5, 2018
Author(s)
Priya Matpadi
Word count
1825
Hacker News points
None found.
Language
English