Optimizing Feature Materialization Costs in a CI/CD Environment
In a CI/CD environment, producing features as code can help organizations efficiently address new machine learning use cases or improve existing ones without breaking entire systems. To achieve this, Tecton has established best practices and product features to ensure that new features are brought online in a performant and cost-effective manner. This is achieved through strategies such as GitHub Flow, where experimentation on new features and data sources can happen in a notebook, followed by promotion to a staging environment for further testing before joining production. Tecton development workspaces provide a cost-effective way to save on infrastructure costs while ensuring effective data sourcing and joins. Features can be developed interactively in notebooks and promoted to a staging branch before merging back into the main branch. A testing/staging environment is used to materialize features in offline and online stores, with options such as post_processor functions to limit data materialization costs.
Company
Tecton
Date published
Sept. 18, 2023
Author(s)
Nick Acosta
Word count
873
Language
English
Hacker News points
None found.