Ray Serve is a scalable model-serving library built on top of Ray for building end-to-end AI applications, providing a simple Python API for serving deep learning neural networks and arbitrary business logic. The integration with NVIDIA Triton Inference Server software and the NVIDIA TensorRT-LLM library aims to optimize model inference and reduce GPU costs. Anyscale is teaming up with NVIDIA to combine developer productivity with cutting-edge optimizations, enabling faster deployment of AI applications to production. RayLLM is an LLM-serving solution built on top of Ray Serve, providing pre-configured open-source LLMs and a fully OpenAI-compatible API. Triton Inference Server supports various deep learning frameworks and provides optimizations that accelerate inference on GPUs and CPUs. The partnership allows developers to leverage advanced inference serving capabilities, improve model performance, and simplify AI development with Python.