Create an MLOps workflow for your entire team.

Join our Slack Community and be part of the ZenML family.

Features · Roadmap · Report Bug · Sign up for Cloud · Read Blog · Contribute to Open Source · Projects Showcase

🎉 Version 0.57.0 is out. Check out the release notes here.
🖥️ Download our VS Code Extension here.

🤖 Introduction

🤹 ZenML is an extensible, open-source MLOps framework for creating portable, production-ready machine learning pipelines. By decoupling infrastructure from code, ZenML enables developers across your organization to collaborate more effectively as they develop to production.

• 💼 ZenML gives data scientists the freedom to fully focus on modeling and experimentation while writing code that is production-ready from the get-go.

• 👨‍💻 ZenML empowers ML engineers to take ownership of the entire ML lifecycle end-to-end. Adopting ZenML means fewer handover points and more visibility on what is happening in your organization.

• 🛫 ZenML enables MLOps infrastructure experts to define, deploy, and manage sophisticated production environments that are easy to use for colleagues.

🛠️ Why ZenML?

ZenML offers a systematic approach to structuring your machine learning codebase for a seamless transition to production. It's an ideal solution for teams grappling with establishing an internal standard for coordinating ML operations. ZenML provides not just a tool, but a workflow strategy that guides you in integrating all your tools and infrastructure effectively.

Use ZenML if:

• You need to easily automate ML workflows on services like an Airflow cluster or AWS Sagemaker Pipelines.
• Your ML tasks require repeatability and reproducibility.
• Automating and standardizing ML workflows across your team is a challenge.
• Your team integrates multiple tools with no central platform.
• You'd like a single place to track data, code, configuration, and models along with your cloud artifact storage.
• Collaboration and hand-overs between multiple teams is difficult.

☄️ What makes ZenML different?

ZenML marries the capabilities of a classic pipeline tool like Airflow and a metadata tracking service like MLflow. Furthermore, both these types of tools can seamlessly co-exist with ZenML, providing a comprehensive, end-to-end ML experience.

It excels at:

• Enabling creation of simple, pythonic ML pipelines that function locally and on any orchestration backend.
• Automating versioning of data and models on remote artifact storage like S3.
• Abstracting infrastructure and run configuration from code through a simple YAML config.
• Logging complex metadata for models and artifacts.
• Automatically containerizing and deploying your workflows to the cloud, connected to your code repository.
• Connecting your secret store to your ML workflows.

However, ZenML doesn't:

• Automatically create visuals and track experiments: It integrates with experiment trackers that specialize in this task.
• Package and deploy models: ZenML catalogs models and metadata, streamlining model deployment. Refer to ZenML model deployers for more information.
• Handle distributed computation: While ZenML pipelines scale vertically with ease, it works with tools like Spark for intricate distributed workflows.

🤸 Quickstart

Install ZenML via PyPI. Python 3.8 - 3.11 is required:

pip install "zenml[server]"
# you'll also need the `notebook` package installed to run Jupyter notebooks:
# OPTIONALLY: `pip install notebook`

Take a tour with the guided quickstart by running:

zenml go

🔋 Deploy ZenML

For full functionality ZenML should be deployed on the cloud to enable collaborative features as the central MLOps interface for teams.

Currently, there are two main options to deploy ZenML:

• ZenML Cloud: With ZenML Cloud, you can utilize a control plane to create ZenML servers, also known as tenants. These tenants are managed and maintained by ZenML's dedicated team, alleviating the burden of server management from your end.

• Self-hosted deployment: Alternatively, you have the flexibility to deploy ZenML on your own self-hosted environment. This can be achieved through various methods, including using our CLI, Docker, Helm, or HuggingFace Spaces.

🖼️ Learning

The best way to learn about ZenML is the docs. We recommend beginning with the Starter Guide to get up and running quickly.

For inspiration, here are some other examples and use cases:

1. E2E Batch Inference: Feature engineering, training, and inference pipelines for tabular machine learning.
2. Basic NLP with BERT: Feature engineering, training, and inference focused on NLP.
3. LLM RAG Pipeline with Langchain and OpenAI: Using Langchain to create a simple RAG pipeline.
4. Huggingface Model to Sagemaker Endpoint: Automated MLOps on Amazon Sagemaker and HuggingFace.

Use ZenML with VS Code

ZenML has a VS Code extension that allows you to inspect your stacks and pipeline runs directly from your editor. The extension also allows you to switch your stacks without needing to type any CLI commands.

🖥️ VS Code Extension in Action!

🗺 Roadmap

ZenML is being built in public. The roadmap is a regularly updated source of truth for the ZenML community to understand where the product is going in the short, medium, and long term.

ZenML is managed by a core team of developers that are responsible for making key decisions and incorporating feedback from the community. The team oversees feedback via various channels, and you can directly influence the roadmap as follows:

• Vote on your most wanted feature on our Discussion board.
• Start a thread in our Slack channel.
• Create an issue on our GitHub repo.

🙌 Contributing and Community

We would love to develop ZenML together with our community! The best way to get started is to select any issue from the good-first-issue label and open up a Pull Request! If you would like to contribute, please review our Contributing Guide for all relevant details.

🆘 Getting Help

The first point of call should be our Slack group. Ask your questions about bugs or specific use cases, and someone from the core team will respond. Or, if you prefer, open an issue on our GitHub repo.

Vulnerability affecting zenml<0.46.7 (CVE-2024-25723)

We have identified a critical security vulnerability in ZenML versions prior to 0.46.7. This vulnerability potentially allows unauthorized users to take ownership of ZenML accounts through the user activation feature. Please read our blog post for more information on how we've addressed this.

📜 License

ZenML is distributed under the terms of the Apache License Version 2.0. A complete version of the license is available in the LICENSE file in this repository. Any contribution made to this project will be licensed under the Apache License Version 2.0.