How to connect stateful workloads across Kubernetes clusters

Apache Cassandra's shared-nothing architecture makes it ideal for deployments across multiple physical datacenters. To provide a solution that spans multiple regions and clouds, DataStax selected Kubernetes as their orchestration platform. They chose to create an ad-hoc overlay virtual application network, which consists of routing and transport components. The routing component involves assigning a unique subnet for the overlay network and using deterministic formulas to convert between pod names and virtual IP addresses. For packet transport, they use a sidecar overlay network with an init container that sets up packet routing and forwarding logic, while the sidecar proxies traffic to the right endpoint. This solution provides shared-nothing architecture, low complexity, low operational burden, and high degree of isolation between tenants.