metaflow.org

Metaflow provides several key features to support the ML lifecycle:

• Modeling: Use any Python libraries for models and business logic. Metaflow helps manage libraries locally and in the cloud.
• Deployment: Deploy workflows to production with a single command and integrate with surrounding systems seamlessly.
• Versioning: Metaflow tracks and stores variables inside the flow automatically for easy experiment tracking and debugging.
• Orchestration: Create robust workflows in plain Python. Develop and debug them locally, deploy to production without changes.
• Compute: Leverage the cloud to execute functions at scale. Use GPUs, multiple cores, and large amounts of memory as needed.
• Data: Access data from data warehouses. Metaflow flows data across steps, versioning everything on the way.

Metaflow is used by companies like 23andMe, CNN, and Realtor.com to accelerate their ML and data science efforts. At 23andMe, complex multi-stage workflows are codified and orchestrated using Metaflow. CNN's data science team was able to test twice as many models in Q1 2021 as they did in all of 2020. Realtor.com shaved months off the time it takes to build a productionized machine learning model. Metaflow is built for ML/AI engineers and data scientists, not just for machines.

Metaflow is open-source, so it is free to use. You can get started easily on a laptop. When you are ready to scale, you can deploy the Metaflow stack on your cloud account or on-premise Kubernetes cluster. Metaflow integrates seamlessly with your existing infrastructure, security, and data governance policies. To get a taste of Metaflow in the cloud, you can try Metaflow Sandbox in the browser.