Exploring the GPU Load Testing Through Generative AI Workloads[Testμ 2024]

In this session, Vishnu Murty Karrotu discusses how to tackle the challenge of accurately simulating real-world AI workloads for GPU load testing. He introduces GPUs and their parallel processing capabilities, distinguishing them from CPUs. Generative AI is highlighted as a revolutionary technology that creates new data and content, enhancing the efficiency and effectiveness of GPUs in generating and managing discrete data. Two key types of workloads are discussed: training deep neural networks and real-time inference and processing. Various applications of Generative AI, such as text completion, image generation, music composition, and video generation, are also explained. The system test process is outlined, including deployment/configuration, simulating customer workload, service/update, manage/monitor, and retire. Proposed solutions for handling these challenges involve utilizing open-source technologies to build a JaaS (Java as a Service) solution. Key technologies such as JMeter, Docker, Docker Swarm, and Elasticsearch are discussed in detail. The session concludes with a demonstration of how to use the AI libraries Stable Diffusion and Dell AI Chatbot, along with monitoring GPU usage based on actions or inputs provided. The Q&A session covers differences between generative AI workloads and traditional workloads, additional performance metrics tracked during testing, and exploring other generative AI models for GPU load testing.