Company
Date Published
Aug. 7, 2020
Author
Ben Lorica, Ion Stoica
Word count
1373
Language
English
Hacker News points
None

Summary

Model servers are crucial components of software infrastructure for productionizing machine learning. They can use a trained model artifact produced using various popular tools and support multiple programming languages. Key features companies should look for in a model server include user interfaces, low-latency capabilities, microservice architecture, deployment options, and advanced monitoring and observability capabilities. Model servers should also integrate with model management tools and support efficient batch and online scoring of large datasets.