How To Build an ML Platform from Scratch

Recently, I attended MLOps World in Austin where various panels and keynotes were held discussing the state of production LLMs. The event highlighted the importance of proper ML infrastructure as data science teams grow and models are deployed to production. During this event, a guide was presented on how to build a basic ML Platform using open-source tools like Cookiecutter, DVC, MLFlow, FastAPI, Pulumi, and more. This guide also covered monitoring for model drift using Aporia. The final code is available on GitHub. The guide emphasized that building an ML platform can be a huge project requiring significant time and resources. Therefore, the toy ML Platform discussed in this guide only includes basic features to teach fundamental principles of building your own ML platform. This how-to guide was based on a workshop done for the MLOps Community. The toy ML Platform uses DVC for data versioning, MLFlow for experiments management, FastAPI for model serving, and Aporia for model monitoring. All these components are built on top of AWS, but they could also be deployed on Azure, Google Cloud or any other cloud provider. The guide also emphasized the importance of evaluating alternatives when building your own machine learning platform as different tools may be more appropriate for specific use cases. The first component in this ML Platform is a model template built using Cookiecutter for templating and Poetry for package management. This model template contains basic training and serving code, with the training code using the MLFlow client to track experiments. These experiments are sent to an MLFlow Server that runs on top of Kubernetes (EKS). The model artifact is saved in a S3 bucket (the Artifact Storage), while metadata about experiments is stored in a PostgreSQL database. The guide also covered setting up CI/CD using GitHub Actions, and versioning data with DVC. It concluded by discussing how to use Cookiecutter for converting the model template into a real template that users can easily customize when they clone it.