Building Highly Available and Scalable Online Applications on Ray at Ant Group

Ant Group has developed a serverless platform called Ant Ray Serving based on the distributed computing framework, Ray. The platform allows users to publish Java/Python code as online services and provides basic capabilities such as deployment, scaling, traffic routing, and monitoring. It supports 5 different usage scenarios within Ant and Alibaba and has reached a scale of 60,000 cores and 5,000 nodes. The platform is designed to help users focus on their own business logic while using Ray's distributed capabilities to develop online applications.