Streamline Your Processes with airSlate SignNow's Pipeline Management App for Engineering

How to create outlook signature

(mellow music) - Hello and welcome to this new episode of Machine Learning Essentials Series. I'm Nishan Thacker and in this video, we'll talk about MLOps. Now, the simplest way to understand MLOps is to look at it as the application of Dev Ops principles and practices to the machine learning workflow. The goal of MLOps is very simple: one, faster experimentation and model development, two, faster deployment of updated models and to production and three, quality assurance. MLOps comes into play to streamline the ML process and define a seamless handoff between data scientists and the ML engineer or the developer who's working to take the models to production. To get to know MLOps better, we'll try and break it into three core concepts. Why do we need envelopes? What is MLOps? And what are the benefits of MLOps? Let's get started. So why is MLOps needed? To answer this question, let's take a look at the typical machine learning workflow. To build a machine learning model, you start with data, which traditionally takes up the most amount of time to clean up and get in shape. Data can come in different formats and different sources. The better the quality of data, the better the quality and efficacy of your model. Thus, as you keep getting increasingly better quality of data, you want to use that to build your new model. This is our first need for MLOps. Version of the source data and its attributes like quality, et cetera so that you can draw lineage to the underlying datasets that helped you build the model. The next step in the ML workflow is to build the model. This comprises of several sub-steps like feature selection/generation, algorithm selection, hyperparameter tuning, fitting the model, etc. This is also called experimentation as you're in a trial and error sort of experiment with various combinations of features, algorithms, and hyperparameters until you get the combination that generates a suitable model for you and your business. But in this case, the trials that do not reap the desired result are also important as they inform the next set of combinations to try. This is where another set of needs arise for MLOps. First, to track metrics of the experiment runs so that one can look back to determine what to attribute to tweak further. In addition to that, since all this experimentation requires some amount of code to be written, there is also the need to source control the code, its environment, and any other dependencies for reproducibility purposes. Also, in most cases, as data scientists iterate upon these steps, a lot of the sub-steps remain the same and don't need to be run again. MLOps can ensure that by enabling the use of concepts like ML pipelines, you can essentially checkpoint these sub-steps to only run if something warrants a change. All these tasks from experimentation, to optimization, is what is contained in the Build pipeline as part of MLOps. Now, once the data scientist has been able to create a model which has acceptable efficacy, this model needs to be deployed so that users or applications can start leveraging it. But before deploying the newly created model, it is important to validate it both technically and from a business standpoint. This validation may even need a staged deployment process. Again, MLOps can help accomplish this in an automated and predictable way, by creating release pipelines that evaluate, test, and package the models into containers so that they can run anywhere. It also allows for a QA phase to be created where all required validation can happen in a low-cost testing environment and then allowing for gated or controlled rollout into production to allow for alpha/beta or red/blue testing. Here, it is important to note that alongside model metrics, validation may also involve assessing the model's most influential features and the model's fairness. These responsible AI practices ensure that models that are created follow ethical standards. These also ensure that such automated processes, eventually also have provisions for human-in-the-loop and that there is appropriate audit awareness on the behavior of the models. Now, you may think that once the model is in production, that is the end of the process, but no, that is just the beginning. The job of MLOps now shifts to monitoring the model. This monitoring not only happens from technical metrics perspective like performance, and response latency to support high query through put but also for model drift. Wait, what is Model drift? Well, a deployed model is based on a definition of what the business case needs. These needs may evolve. For example, if you were detecting credit card fraud and the business now evolved its thinking around what is a fraudulent transaction, it will need a rethink or retraining of the model. This is a simplistic example of model concept drift. Another drift that is important to understand is data drift. This is when you train a model on the demographics of a set of users and now you`re observing that the population it is being utilized on doesn't match that same demographic. If you're from the data analytics or the warehousing world, think of these as slowly changing dimensions. Other examples of data drift include changes in the data due to seasonality, changes in consumer preferences, the addition of new products, et cetera. MLOps can listen to these changes and trigger automated retraining of the models so that the new model can now cater to the new requirements. Well, that was a pretty high-level view of the ML process and where MLOps is needed as part of it. Let's now look at what is MLOps and how it accomplishes these tasks. At the core, MLOps is a process that enables data scientists and IT or engineering teams to collaborate and increase the pace of model development, alongside continuous integration and deployment with proper monitoring, validation and governance of the machine learning models. As you must have observed when we talk about the need for MLOps, it is not just a set of steps you follow or a product you deploy, but it is a process you engage in at a depth that depends on your specific needs. If your scenario just warrants you to engage at the experiment tracking and automated deployment, that is all of the process you follow. If another scenario warrants tracking code, data, and drift to automatically retrain based on certain criteria, the engagement in the process is more involved. All this while, we've been talking about MLOps as a concept. let's spend a few minutes talking about how you can engage in MLOps with Azure Machine Learning. First, you can create reproducible ML pipelines. Machine Learning pipelines within Azure ML allow you to define repeatable and reusable steps for your data preparation, training, and scoring processes. This allows for the same checkpointing capabilities we spoke about earlier. Second, it enables the creation of reusable ML environments for training and deploying models. This again, helps with reproducibility. Third, it provides capabilities to register, package, and deploy models from anywhere. You can also track the associated metadata required to use the models. It helps with maintaining a centralized repository of models and deployments irrespective of where they were created. Fourth, it enables capturing governance data for the end-to-end ML lifecycle. The logged information can include who is publishing models, why changes were made, and when models were deployed or used in production. Fifth, it has provisions for alerting on events in the ML lifecycle, events such as experiment completion, model registration, model deployment and data drift detection. These can help trigger retraining pipelines or other actions. Next, it helps monitor ML applications for operational and ML-related issues. Compare model inputs between training and inference, explore model-specific metrics, and provide monitoring and alerts on your ML infrastructure as well. And finally, it provisions automating the end-to-end ML lifecycle with Azure Pipelines or GitHub Actions. Using these, you can frequently update models, test new models, and continuously roll out new ML models alongside other applications and services. Now this is the most comprehensive set of MLOps capabilities offered anywhere. The purpose of these capabilities is to facilitate data science teams to collaborate effectively with ML engineers and build a strong MLOps practice. What's more, Azure ML also integrates with Mlflow to enable you to leverage a platform-agnostic ML lifecycle management service and bring MLOps capabilities to MLflow pipelines. Now that we understand MLOps a little better, let's look at what benefits it provides and there are quite a few. To start with, it enables ML Models scalability and management. Depending on scope, thousands of models can be under control of MLOps. Next, MLOps provides reusability and reproducibility of ML pipelines, which is terrific. And it is not just about data scientist and ML engineer productivity, but also audit and regulatory requirements. Also, it provides effortless CI/CD to serve up-to-date ML models with lineage, tracing back to the data it is trained upon. In addition to these, MLOps helps maintaining ML Model's health and governance with simplified management of the model after deployment. And finally, it advocates for responsible AI practices ensuring model interpretability and fairness. Ultimately, MLOps is a useful technique for people, processes and technology to come together to optimize the creation of Machine Learning solutions. Now, given there are so many moving parts, it is common for teams to take time to adapt to the MLOps way of thinking. But keeping the rigor and sticking to the process leads to a robust, scalable and enterprise-ready ML practice. Now, as a bonus for sticking with me on this video for so long, let me leave you with a set of best practices for MLOps on Azure. Create models with reusable ML pipelines using the Azure Machine Learning pipeline components. It will help you iterate faster. Automation is key for robust MLOps. Use GitHub actions or Azure Pipelines to automate the full deployment and monitoring and retraining process. Monitor performance, not only for the efficacy of the model but also the underlying infrastructure like memory usage and query throughput etc. Monitor data drift and utilize the insights to retrain the model depending on business-specific thresholds. This helps keep the model always updated. And enable automatic audit trail creation for all artifacts in your MLOps process to ensure asset integrity and meet regulatory requirements. Well, that's it for this video. Hope you got a good understanding of MLOps now. To try MLOps on Azure, please spin up a free Azure Machine Learning workspace at aka.ms/aml-trial and download the 30-day learning journey from the data scientist resources page at aka.ms/data-scientists. We also have a dedicated GitHub repository with MLOps templates and samples at aka.ms/mlops. That's it for now, see you in another episode of Machine Learning Essentials. Thank you. (mellow music)