How To Use Google Cloud Dataflow to Ingest Data From Google Cloud Pub/Sub to Redis Enterprise
Google Cloud Dataflow is a fully-managed service for transforming and enriching data as a stream or in batch mode, using Java and Python APIs with the Apache Beam software development kit. It provides a serverless architecture that can shard and process large datasets or high-volume live streams of data in parallel. A Dataflow template is an Apache Beam pipeline written in Java or Python, allowing users to execute pre-built pipelines while specifying their own data, environment, or parameters. Google Cloud Dataflow supports various native managed services that drive real-time user experiences. The custom template introduced in this tutorial ingests data through Google Cloud Pub/Sub to a Redis Enterprise database as key-value strings. Support for other data types is planned for future development.
Company
Redis
Date published
April 11, 2023
Author(s)
Gilbert Lau
Word count
660
Language
English
Hacker News points
None found.