Send Grownup Default with airSlate SignNow

Send grownup default

and so up first as you might be able to see um is bruno and frederick both speaking from france in the middle of the night it is after midnight for them but they have arrived today thank you very much and so over to you brendon frederick talking to us about api and how wrist and proc mail can work together thank you very much for the introduction and welcome everybody especially for those of you down on there uh we'll try to make our best to cover the topic in in the short amount of time that we have so first let me introduce myself my name is bruno connect i'm working in juret pacquiao for the last 20 years now and but i've been doing with my my friend attacks over there uh stuff since 93 0 99 pf13 for those of you who know that and i've been doing various set of stuff in the linux and open source area being upstream for some projects and i've been happy to to be in fact in australia for previous linux australia conferences back in 2007 2013 and 2014 and it's very uh i'm very happy and glad to be able to to talk with with you again this year and i will now pass a mic to to felik who will introduce himself well uh good day everyone my name is frederic pastron and i've been working for hp for the last 20 years as well i think we joined hp nearly in the same year with no ni i've been working in hp in the pre-sales organization ever since i mean moving back and forth between you know i would say local region like france to emea then to worldwide and i've been focusing on infrastructure management solution mainly from the early days of hpc and down to the cloud days with openstack and now kubernetes and all this stuff i'm currently part of the team which is the hp dev experience team and we are delivering a hack shack and we're delivering a program that is called the hp developer community program and uh hack shack is one of the physical aspect uh of it that we were taking or taking care of during the events moving now to virtual and as part of it we were delivering workshops and all of these hands-on training that we used to do live we're now doing them virtual and we're making them available as well so they're using jupyter notebooks and uh really that's about it i would say and when i'm not doing the computing stuff and working for hp as you can see the background i'm playing with ify and building up some musical stream as well and sometimes asking bruno to come over and give me some help with some coding and that's about it yeah and i can i can also bring some some cds that i have behind me as well to to so that we can hear stuff together so let's first talk about uh what we were talking about so frederick gave you information about the the activity which is doing workshop on demand and how does it work and what infrastructure did we put in place and what type of tools did we use to make all that work so first we have a student who was willing to pass a workshop on demand and so he has first to look at our portals the registration portal register itself and through the http and his web browser it will be able to make a query to our system our system will then discuss with a complete infrastructure which is private in fact which is not reachable from the outside from the internet so the first question we had when we when we wanted to put that infrastructure in place and say okay we have the registration portal which is hosted on netlify how do we make the discussion with something which is internal and then we talk about okay everybody was doing you know new stuff there's a new kid in play once they want to play with rest api they want to do https so some new and really enjoying stuff and i'm i'm a little guy and i said okay but you know you are behind the firewall it's very complicated from a network perspective you will have to put some proxies and we will have security issues and it will be very difficult so say i would like to propose something which is from my point of view much simpler which is to discuss from the portal uh here to the jupiter notebook servers the server hosting in fact there's a majority of activities that we need to handle to set up the infrastructure correctly that server can really easily have the mail as an smtp server receive mail from the outside it's difficult to have an interactive connection which is live between a client and server but that's very easy when you deal with smtp because it's uh asynchronous by essence so we put that in place and that server in fact which is hosting our jupiter hub infrastructure is also equipped with space fix and proc mail and process requests coming from that portal then using this information it will talk to the backend appliances which could be physical servers which could be virtual machines which could be containers using ssh connection using rest api and it will set up book in fact the environment deploys a notebook for the students create the student environment and once all that is done behind the scene it will then talk to our backend system and register really the user in our in our database which manage which is managing all the user information workshop information and which serve also as a base for for https queries from the posifix jupiter hub server once this is done we will then inform the user through again mail that he has successfully registered that the workshop on demand is ready to use and we give him the credentials that we will have to use to connect to the system and to be able to perform the workshop and from now on he has four hours to be able to perform the workshop he has booked through the system so he logged on the system he has access to the notebook we have multiple type of workshops we will disclose that a bit later on and he will talk during his notebook jupiter notebook so which is based on on different set of kernels depending on on the workshop we want to make and during those execution of code through the notebook the jupyter notebook he will talk to the backend infrastructure which could host as we said different type of appliance depending whether he is performing a git 101 type of workshop or a more complex envelopes hp container platform type of workshop because we have a very large range of more introductory type of workshops and more high-end type of workshops depending on what people want to uh explore with regards to technology so that's how the infrastructure is working and we will demonstrate that to you uh just a bit later on so we used in fact a mail api not an arrest and a very modern stuff https and certificates and all that type of stuff no no no we are all guys we do all type of stuff on our site to complement what is difficult to do with a full rest api uh not having a direct connection between the frontal netify and that jupiter hub machine uh male is very convenient because mail can go through very complex network topology without any problem going through the firewalls going to indirect not non-interconnected networks very easily it provides for free a queueing system you send mail it's put in various queues up to the final system it's re-delivered in case you have issues of course we want to have the most interactive way of working between the user making the request and the deployment of the infrastructure however having a queuing system is very helpful in that in that context then the server receiving the mail is using a small tool called procmail which helps managing the api requests and we have like a rest api we have verbs so we have the verb create we have the verb cleanup we have the verb reset and then we have some parameters that are not passed through json or that type of mechanism that you use with rest environment but here we just use some parameters on the subject line and some other parameters in the body of the mail so typically here we have the the number of the student on on the subject line and the name of the workshop in the body and then we call a script so we pass all that through a pipe to the proc my action script that we have created with the match which corresponds to the parameter which has been detected in the subject line so it's easy to filter input it's easy to manage parameters and it's very easy to call a script which does all the dirty stuff behind the scene and that script is a mix of shell script unseable playbooks because we have a large infrastructure to deploy behind once we receive the request from from the outside also it was pretty easy to to perform the test because we have another small script which has support for the same verbs as what our production environment has and we just have to send a mail using those verbs to the local host system when we do the test on the machine and it's pretty convenient to also make our test uh in parallel with uh with a production and we have logging and mechanisms like that coming for free so as i said the the infrastructure of to to provision it we use mainly ansible environmental so there are plenty of tools available that's the one we know the best so that's the one we decided to use and uncivil is in charge in our environment to manage different type of activities first it's in charge of the platform installation so our jupiter hub machine itself is deployed automatically from a deployment server with a minimum set of packages and software installed on it and then we have a large ansible playbook which does all the setup and we did that because we wanted to be able to provide that feature to other teams inside hpe able to reuse our development so they can set up very easily their own jupiter hub environment sorry my time is flying time yeah yeah i is doing conformity and convergence each night we run an ansible playbook to be sure that the platforms take consistent and also it does the student instantiation and the notebook instantiation for us so that every student is recreated from scratch every time a new notebook is generated and we benefit from some interesting features inside the notebooks we can use jinja extensions for example inside the notebook to customize a notebook with regards to the student we made the request so using the student id and using some variables we can really associate a student to a set of machines to a set of vms etc etc okay now it's time to jump on the video okay so play the video so this is the hack shack and i think the links were copied over in the chat room so you can see it's a nice website where you can actually perform different things you can check out for some challenges some information some replays videos of all the action and all the workshops that you have and we'll be going to the workshop section so within the workshop page you can actually see some different tiles appearing there and they're displaying the different workshops that we are proposing in our environment so there's a few of them around the esmerald portfolio a few around programming and some other stuff around open source like uh bromide which is our you know uh open source project within hp so i choose uh python 101 because i'm not a very good developer so i need to take some courses on python so what why not this one so it's a discovery workshop on on python so i just register into my email my name company name and i read the terms and condition that tells me that i need to i'm running for four hours only and at the end of the four hours everything will be wiped out from the system uh including the work that i did so i need to to to work during this uh these four hours which is largely enough to perform most of the most of the workshop and actually checked for the video as well at the same time so i'm registering and so at the very same time this front-end application is calling the back-end database and registering myself as a new customer so i can see my name the workshop i selected the notebook and the email that will be sent to me is a welcome email that's the first one the first of the series of four that will be sent and i can see that i've been assigned the student id so this information the student id will be passed to the back to the backend as you can see and this is the script that is running in the back end so we can see the verb creation okay that the bruno talked to you about a moment ago and so we can see that is the customer name and customer email address and the id of the student that will be delivering on so the verb is to create so i'll be i mean the the the automation will be creating the necessary uh environment for the student six nine six i would say so it's checking the workshop id if i remember well and from there you'll be you know doing some other stuff around uh you know preparing the notebook copying the notebook generating a random password informing the back end as well that we've been doing all the necessary work uh for him to tell the customer that his environment is ready so obviously we'll be at the at the very end so we're here just generating the random password then we're doing the interval copy so we're using the playbook to actually copy over all the necessary notebooks into the students folder and we're erasing everything by default before copying that's a kind of a security measure that we took and at the end we do we do actually two api calls passing over the the password to the back end to be sent to the customer and uh obviously setting up the customer as an active so if i go back to the database and i do a refresh on the table i will see that now i'm not anymore in i mean i need to check that i receive my emails which is the case okay so this is my wonderful mailbox okay this is the mail that i received so in the first mail this is a welcome message i mean giving me information about how to run the workshop providing the information on the video replay that is uh i would say link to the workshop and so on so forth in the second email i get the credentials and i can actually click on the start workshop and register i mean and open up the browser to actually do the workshop this actually feature will disappear over time we will be providing an autoloading feature directly in the email where the customer wouldn't have to take any order the work the the password and uh and type it in so just a very quick well you can move on this one is just to show that uh we've been deploying the workshop in the back end okay so if you can move just a bit quicker because i don't think we have that much time left yeah you can you can't see it anymore okay so we'll be jumping to the workshop itself so i'm logging in now with the provided password okay and i'll be launching my notebook and then i'll be doing the note the workshop itself so i'll be opening up you know the readme to introduce sorry the workshop then doing the lab itself which is a simple api simple python 101 labs so it's going to be fairly easy so obviously we'll need to show that we have plenty of kernels available this one is the python 3 tunnel obviously based workshop and this is the workshop itself so what is python and now you just execute the different cells so the notebooks are based you know on markdown and uh cells where you can actually run some codes so this is only just simple python code for people who want to learn license so you can move on quicker if you don't mind and we'll move to i'll just see the rest so once i'm done with the with the with the sorry the workshop i can fill up the survey obviously because we need your feedback and uh actually that helps us to define what will be the future of the content that we'll be providing to you so i'm logging out and obviously uh what i'll be doing now switch up the time to go quicker in the in the in the environment so rather than waiting for the four hours i'm just you know hacking the database to cheating here to tell him uh this is the end of the of the of the four hour window and just do the do the necessary automation now to clean up the environment okay so we've got two minutes left so you can pass a quicker on the on the rest so just to show you that the database will show now that i'm in expiring mode and then i will be expired then i'll receive one minute left so it's getting way faster than i think okay so just right there i think we can i mean go with the end of the video because it will only show the very same thing which is the reset environment providing the very same step of step the verb will be different in the sense that now it's going to be a clean up verb and you will be doing all the necessary action that is to say i mean cleaning up the environment erasing the content changing the password informing the backend database that we have done the cleanup so with the gener providing the new password setting the customer to inactive and that's about it really that's the and the automation could be much more complex in some cases because some of the workshops would need some ldap would need some reset procedure that would be more i would say complex than just erasing a simple content of a folder and so depending on the workshops we are doing uh different things so it's time to wrap up the uh ewing is saying so we can shut up shut the video and maybe go back to the very end of the slides and uh so we yeah so everything is managed under git we have a currently 10 workshop developed using that way it's built on a on the ubuntu 2004 uh distribution right now we were working on a port to centos as well to have uh to cover other needs internally inside inside hp we have multiple jupiter cameras automatically installed during the installation procedures and we are working to publish three more workshops per month and work on the ci cd stuff and some other one today and here you have in the presentation and presentation is available online you will have all the links to be able to discuss with the developer community and access the last one is allowing you to access the workshop on demand that you have seen to pass a workshop if you are interested thank you very much um we're going to have those slides up on the sysadmin.miniconf.org site on the program there so you'll be able to grab those links from there thank you bruno thank you frederick um both of them who said that they will jump into the chat for a few minutes yeah we'll say that answer any questions that you might have um that we didn't get a chance to get to um and then we will be back in about 10 minutes with our next presentation um from michael thank you very much bye