Get organized with our pipeline tracking spreadsheet for Animal science

How to create outlook signature

hey everyone my name is oindrala chatterjee and i'm a data scientist working in the office of the cto at red hat in this video i will show you how you can track metrics from your machine learning experiments and runs using kubeflow pipelines and elira q flow pipelines is a platform for building and deploying scalable machine learning workflows it allows us to automate the running of jupiter notebooks and scripts using a simple workflow to see an example of how you can create an automated workflow for your machine learning notebooks using kubeflow pipelines you can check out the description for a video demo so for the purpose of this demo you need an existing kubeflow pipeline configured with components such as notebooks or scripts for which you want to track metrics so as you can see here we have already configured a kubeflow pipeline using elira which consists of two notebooks the first notebook is the demo one create table then the second notebook is demo one join tables so using the example of the notebook demo one create tables we will discuss how you can configure the notebook appropriately to track metrics during the execution of this notebook so firstly to enable tracking of metrics during the execution of this notebook the notebook must have an output component called ml pipelinemetrics.json so in this notebook we declare a file where we want to store the captured metrics it must always be named ml pipelinemetrics.json and it should be a json serialized metric dictionary which consists of all the metrics that are captured during the running of this loan book next let's see an example of a metric that we are capturing during the running of this notebook so as you can see here we capture upload df1 time which is essentially the time taken to upload a parquet file to s3 storage uh we are also capturing other metrics such as the time taken to execute a certain create table query and we call that time to create table one so once we have declared variables for each metric that we want to capture during the running of this notebook we want to aggregate all of those metrics into a metrics dictionary so here we aggregate the metrics captured into a dictionary called metrics and the dictionary essentially consists of each metric that we are capturing uh we give a name of the metric uh the variable name and we can choose uh the format to be either raw or one of the predefined formats supported by kubeflow so once we have defined the metrics dictionary we can save and export the metrics dictionary onto the metrics file part which we defined earlier so once we have the notebook configured to track these metrics let's trigger the kubeflow pipeline and see the metrics being captured in action so to run the pipeline i can go to run pipeline select the queue flow pipelines runtime which we have already created and hit ok so upon submitting the pipeline clicking on run details we are taken to the kubeflow pipelines ui where we can see the running notebooks and debug any logs during the execution of this notebook so as you can see here the notebook has already started executing any logs that are generated during the running of this notebook are seen here once the notebooks have run successfully it should look something like this to view the captured metrics during the running of these notebooks we can go to run output where we can see a metrics table which captures all of these metrics for the two notebooks in this pipeline now to compare runs uh within an experiment we can go to the experiments tab go to all runs where we can see each run to compare two runs or more we can select those runs click on compare runs and here we can see an overview and a table comparing the various metrics between all the different runs this can be especially helpful when you're tracking multiple model training runs and can also be used to capture model performance metrics so that was it for the demo happy experiment tracking