Empower your security operations with airSlate SignNow's pipeline management system for Security

How to create outlook signature

so hi everyone welcome to this webinar on cicd security my name is Alex Jones I'm a tech lead for the cncf I also work as an engineering director at canonical and I deal with all things kubernetes uh today I have Ben hirschberg who's joining me as CTO Amo and I'll let Ben introduce himself before we go further hi everyone it's great to be here um I mean Ben as Alex said I'm CTO of armo and maintainer of cubescape we are working on kubernetes Security Solutions here so I'm pretty excited to start to talk about securing cicd Pipelines well I'm really excited because it's not every day that I get a chance to sit down with somebody and talk about things that often are just beyond um the curve I think of most Engineers who are getting up and running uh with Security Solutions particularly in Cloud native environments so I'm excited to have you uh with me Ben um and actually I thought one of the first things we could talk about are the trends that are changing in the cloud native ecosystem so if I share my screen for a moment and I hope everyone can can see this clearly you know five ten years ago we had this very simplistic model of development test and prod in terms of environments right and we we had gating at the time but for folks who have worked uh in Banks or in large Enterprises you may well be familiar with these Gates being things like servicenow right and it may be a ticket that goes to an external service um and that that creates a few bars right because often these systems that they go out to will require some sort of manual intervention they may well be arbitrary and in the in the context of security they don't mean a great deal right it would typically be Hazard Dev team submitted the right paperwork to progress to this environment and also what does that mean right with the dawn of with oci so with with images that are being produced like a Docker image we have an artifact that would go across these environments but prior to that there would have been a rebuild or possibly even artifacts that were dependencies that were rebuilt across these environments so we've solved a lot of problems in the past few years in terms of the provenance and the artifacts but we still see that many companies are struggling to provide these this kind of high quality gating but bring it into the into the modern era so you'll see that there are folks that have kind of automated Gates that can check for things such as does the liveliness probe work can I can I spin it out so an automated gate might do a bunch of stuff you know it might might do some tests right it might run some unit tests it might run some integration tests but very rarely do we see automated Gates um that run anything of security testing right so and Live security testing so I'm really interested as well just before I get any further down this path um to know sort of your thoughts about in terms of if you take the kind of typical CI CD approach and you apply it to the cloud native ecosystem what are your thoughts in terms of are we creating a larger attack surface these days and do we have more people working in the space what are sort of some of the trends that you think are starting to emerge right now and what are your customers and users telling you so yeah I think that uh you know as of today we are we are really living a new world okay and and some of us even start to forget what was the old world but but really today the you know the speed the way we are you know we are deploying things are are are changing and have already changed so if you know as a technological leader of a company okay I'm talking to you know other uh you know technological readers of other companies and and you know every I noticed that like in the last two years everyone is telling that however is saying to the other well we are deploying new new things in our production like 10 times a day or or 100 times a day or even you know more and and this becomes some kind of uh you know of um of a badge of work that we are able to deploy things very fast in our production but from the security perspective you know it's raised a lot of questions okay um as you know infrastructure as a code and uh you know evolved and the way we are pushing changes not just touching only the software itself but also the infrastructure around it and kubernetes uh is you know is in this case part of the infrastructure um it raised a lot of security questions okay because you know theoretically those who are were pushing these changes into our git repos right our our have a specific you know uh uh um uh roles in the company and made their main role is actually is not necessarily security okay and and it raised the question is okay then who's looking after security so in this environment okay because we are really uh looking putting the developers and devops engineers in the focus of all of these deliveries and and this requires you know a specific skill set a specific understanding of security or understanding what is doing what is good and what is bad for security and and you know usually as a security engineer myself you know originally I can say that nothing is good enough for security okay so therefore we need to we need to understand Beyond and do some prioritization among the bad things so um so I think that that the skill is need to be there and we have to have some kind of an answer of automatizing you know security Parts okay of this delivery process and and and and sometimes you know I I'm telling my friends that that if I'm looking uh into the GitHub actions of of actual projects today I can see that they're they're using uh spell Checkers okay uh in in their data directions to approve new codes into the into their project but they are not using anything so any security tooling so and and I myself was you know for me is good spell checking is really important okay and I get annoyed okay bye by that spelling I can still think think that if I need to prioritize two things I'm I'm going to automatize my security person just after the spell checking right so so I think that that's right that's really interesting right like how they they feel as if they're enabled enough to put a spell checker in but they don't feel as if they're unable enough to put security Tooling in simply right right so yeah so I think that the importance of putting security uh Gates into our processities into this uh uh you know um areas is is really important okay so to to be able to keep up with the velocity why while we are you know thinking or less concerned about security the security aspects of this velocity is is going to be you know um as we are evolving in the cloud native environment and in in these processes is going to be you know a Paramount thing okay otherwise we'll get lost in this part and and you hit it upon something that's really interesting there and I just um updated my my diagram to show it but we've moved from the old world of it being kind of fire and forget to now this idea of continuous deployment right and you'll see lots of diagrams similar to this that kind of look like a a wheel right because it's going round and round but this idea that you can now take something locally and like you say Ben have it in production the same day it's pretty crazy and you know developers are looking at real-time signals from their production environments making tweaks locally and then deploying out and so to my point a bit facetiously here about using servicenow with some other old-school method forgating they're just not adequate and they only compound the fact that security is not the fourth of those of those thoughts so I think it's really really interesting because you know as we start to increase our velocity there are certain industries that just won't participate in continuous deployment until they have a risk profile analysis uh before they deploy into their target volume right whether that's kubernetes or on a VM or a function right they they have certain Regulatory and governance requirements that means they have to do due diligence to make sure they're not regressing or by the data that you suppose you know and some some control that's not being met and so you know that's where you know I was really interested in the stuff that um that the folks at cubescape are doing and partially to facilitate that conversation I just want to take you folks through a really simple example so you know the idea that most people are working in a get Ops pattern is not completely accurate however it does certainly represent the future that a lot of people are trying to move towards to give you an example I've got a really simple repository called cats right it displays pictures of cats right um it's not super super complicated but it is representative of a common pattern where Engineers will build the code in the repository but then they will also have the templates in that repository as well for that code I think a lot of people have tried different patterns such as having your kubernetes manifest in One Directory having it in another having a different repo but I commonly see there is an amalgamation of code and templates for that what is interesting though is that even in this world there is um opportunity to do better because I can cut a release and I can deploy that out very very easily and very rapidly right if I've got permissions for my work let's say I'm in a mega Corp and I can produce a microservice and I have committer permissions what that also means is that it will build an image off that those commits and it's quite easily for me to go to production so with very little thoughts one single error in this cat's repository can be deployed out through a gitops paradigm into my production environment in minutes or if not if not seconds and so I think that only exacerbates the need for not only gating but continuous scanning I mean how how what are your thoughts been on sort of moving towards a github's passage it's obviously a good thing but it does come with some dangers right with great power comes great responsibility right yeah it is you know um this obviously you know the security gating also around githubs and the weights that things are getting into production systems or or not even production systems I can tell you that that chroma research we are seeing that the law of staging and development systems are also public facing in the internet so this means that that attackers can get there so um so production getting into these environments is you know is something that obviously you know attackers are really looking for uh for different reasons okay uh you know we can't talk about you know these reasons for a long time okay but if I really want to you know boil it down to you know um a few things a few points okay attackers are looking for you to take your data attack servers are looking for to destroy you know uh uh uh either your services or your data behind your the services in in in order to you know because you downtime or attackers are simply looking for you know to to take you know your Cloud account and and start to use it for their own on good um and and you know therefore githubs is is has obviously um um you know very some very concerning you know uh um uh Dimensions where where we have to look after okay what is really getting into the our gate if once we had to look in into what was going into uh you know our production system and we did it with you know looking at the actual you know packages you've been telling before okay in the old school that we're looking okay preparing some installation package and and in this installation package okay the security engineer we're looking going through it uh now we need to uh we need to be sure that that that actually the the interface with the production system is not the API server of the production uh kubernetes but actually your interface with their production is is your git repo or or your main branch and therefore this is the place where you have to you know look at where what is getting in there and and honestly you know um your you know uh your drawing made me think okay for another interesting thing that that not just actually what is getting in from a security perspective what is getting into your production but also the time you need to fix it in the old world okay as you saw that we are we were opening uh servicenow ticket or any other ticketing system you're opening up tickets and you know uh pushing your you know your changes through the whole uh whole organization um today you know uh this tooling enables you to find out security issues not just earlier to prevent to to not to let these issues going into your production but you it it can give you also already a feedback very early okay in the production phase it means that as you as a developer you as a devops engineer can get an instant feedback okay about your changes and you can solve it right away okay in your pull requests for example and it also makes in in the one hand these new processes are are are concerning for our security perspective but in in general it can lower your costs okay because it it Go these processes are can make give you feedback much earlier I think there's a lot of wisdom there I was I was making some notes on my document as you can see it's interesting because if you think about it what you've described there is the lens is Shifting isn't it right moving to Here Right Moving to this place on the left over here uh from the right so for those folks who maybe aren't familiar with Git Ops think of it this way again the kubernetes cluster itself runs a process um there are several out there I've got one running here for my for my demo that process synchronizes to your git repository into your artifacts right it pulls them in rather than pushes them from the CI CD but as Ben described what's really interesting is that you can now with the right tooling start to identify things that are going to be a problem later on before they become a problem so you're not actually looking at here anymore so that's kind of where the old pane of glass used to be for security so where security used to be and then where security is moving right it's starting I should say and that's really interesting because I was having a play around with cubescape and if you looked on my screen in the background what I've done here is I've installed through the marketplace on my vs code I just went and grabbed it before this little call and I was like hmm what can I show off and so as a as a previous cluster admin you know in many many roles one of the big problems that people often put on is host networking set true which gives you certain um certain routing capabilities and access to IP address ranges and what's interesting with this is that an engineer might just turn that on because they copy paste it out of a document or a guide they don't really know what's going on and what's really cool is that I get this pop-up that starts to tell me hey you might want to think about not doing that because it's going to inherit the access to the entire host Network and if you if you look further into that it talks about the remediation and I think this is really interesting and I guess this is a question for you Ben but it feels to me a lot like you're coming from a developer experience first perspective on this was that an intentional thing or was that an organic thing did you decide you know as a as a as a project this is something we want to Target because if we make it simple for people to understand and they're more likely to use it so um yeah I have to tell you that that the how we started the cubescape project it was really you know um from a developer perspective and not just developer per se but also operations srvs devops um we really were thinking about you know not the classical security Persona in the organization okay because we understand that that today just as you you know you drill it up here that actually the the the the you know the the way that the world has shifted in into the direction of where the things are happening really and where the things are really happening is is around the code and around what developers and devops are doing therefore when we created this project okay we decided to Target actually both personals okay we are not saying that we are against any security Persona here but but uh uh We've we've really targeted the developers and devops okay and enable them to uh you know with the same engine okay uh as you would you know scan for security issues your cluster you can scan the quietest objects you're creating even before and just as he's shown in the in the vs code plugin example already in the developer in the development phase uh to show you these issues raise these issues and you know going from the developer face to the other gates to the other phases you you would have the same uh same engine okay if we are talking about between Engineers okay you're taking the same engine through the whole process and this enables you a lot of good things um and not just you know showing early these issues but you can synchronize actually your your expectations across you know the whole uh left to right pain yeah I mean I can imagine that if I was to copy this to my security team and they used a different type of scanner then it's almost like you're wasting that effort having to translate one thing to the other and so that was really cool because before this um we set this webinar up I was playing around and I built um an action based off the docs from cubescape to do image scanning um misconfiguration scanning what was really cool well I'll show you what was called I was able to just add it into my workflow as an engineer so if we can imagine that this was my my local directory you know I'm cutting my code and I create that new PR um what was really fun is that it it adds itself in as a check and I can actually see if there's a there's a there's a misconfiguration you know in my code um and what's also interesting is I believe that then there's there's a way to to tailor that isn't it you've got a couple of methods it's like thresholding there's exceptions I mean do you want to speak a little to how how that would work in reality because I know that in a real world no one's going to get rid of every problem yeah yeah so um so actually two you have as you know what we're talking about right now is using cubescape as part of GitHub action in as part of a security gate what kind of code is uh what kind of codes you are accepting into your your cluster okay or sorry not the interior heat triple okay and eventually into your cluster so uh you can have different approaches okay solving okay that what are you doing with these issues you are seeing okay here you can say on the one hand that well okay I'm cubescape is generating uh on overall respore which is um you know we could call the webinar about how this risk score is calculated but the rule of thumb okay you could say that check what is your currency score uh and say well okay I don't want to go below this risk score okay so you would use this score as an as a threshold and you have a common argument in in the GitHub action for for applying that this is one uh one approach another approach is that well you know I'm fine with accepting uh uh you know a low risk issues in tumor into my repo and every like every issue you know cube step is really raises past you know this severity uh uh scoring okay of uh critical high medium low and and you could say that I'm okay accepting low issues but I don't want to see high end critical or and you can also find with medium issues and this was because the the you know the pr to fail if someone introduces a high but I think that that yeah sorry please finish it though so there is another way that that you can with cubescape I think it's very very powerful to to create what we call exceptions like you know you're saying that well cubescape is checking uh whether I I my deployments are doing uh using the Linux hardening capabilities okay which you have you know shown just on the screen before and you can say well I'm fine with with design I'm not really you know uh concerned about this issue and and and you can create exception in a simple Json file and keep it in your the same repository okay so as part of uh you know of the pr processes someone can either solve their issues or can add the specific issues into the exception file and say well this is something I'm okay with and these are the three ways to presume presumably if you're running through a cicd your security team could actually keep their own exceptions repository right so you actually have this ability to have a separate type of persona who's managing exceptions so I can't just go and bypass them without talking to somebody right right so this is I think that this is the most mature mature you know approach okay that's really you know to to split you know the ownership okay in this case and have you know the security team to to to create these exceptions um and but uh but it might turn out that you know actually the security team will also you know manage it as part of the git okay and it's really depending it depends on your organization and your you know the way you're working um but you know cubescape enables you to to handle ing to how you would like to work okay and Define your workflow I suppose what for me as uh an engineer I find most appealing is that because it's built you have the ability to use the action and you've got the local experience you are being told quite clearly several times like there are misconfigurations so developers can no longer claim ignorance right like oh I just put this thing in so that when you get a massive um you know vulnerability report coming at the cluster level you had plenty of opportunities prior to that right and I guess that takes us to the third part of this isn't it is that we've described um how you do a lot of the shift left so you've got the local config that's being checked we also have then the ability to run it in the cicd so cicd in this scenario I've described as sort of my GitHub action you know run um remote checks and then you at this point let's say you've gone through both of those and that's not really the end of the story is it because as an engineer and as a sysadmin I still need to make sure and even as a security expert I need to make sure I have continuous scanning in the cluster like I know that you folks have an offering for that as well talk me through a little bit about how that works because you know I've played with it but I'm not an expert yeah so um so we're really in the cubescape project we are targeting for the whole range okay um you know at the end um and I'm I'm a big fan of githubs okay but at the end okay you need to you know you need to also look into what is actually happening okay in your in your your actual production environment um and therefore you know there are two ways to to use cubescape or you know other tooling okay you can scan Cube API okay with the same CLI Tool uh we are releasing uh uh and and you know see the same issues also in your production environments in case you haven't fixed them uh before uh and the other option is to cubescape as part of of your cluster you can it you have a you know a simple Helm chart installing it as a microservice and in this case the the cubescape microservice will monitor okay uh uh your your production environment it will monitor okay uh your your Cube API and we'll check okay every once in a while that how you know your deployments or kubernetes objects are looking like and it'll also uh um it will also scan you know the vulnerabilities in your your images and and eventually okay we'll as the project our project is progressing okay we'll connect even more data feed uh uh data streams to uh to cubescape to check to make a better prioritization of your issues and maybe find issues with security issues we haven't you know we cannot detect through Cube API or image will not read the scanning and as of today there are two directions okay to take this data from okay one is that you are using it as a standalone uh project in your cluster and in this case you can visualize the results with the with Prometheus uh we we can export the data into Primitives and and from there you can take it into grafana or or to with other Integrations and um and we have our uh almost cubescape Cloud offering okay where we are freely you can push your your data there and you can do the monitoring uh uh view uh from this uh you know from this SAS and you can look into I think yeah that's super well that's what we're talking about right because right what we started when I when I installed the helm shop and I got up and running I instantly realized that it was at that point in time the personas who can have access to this now far exceed kind of just your engineer who's working down in the weeds in the CI CD logs on a local system right right one of the things that I was first drawn to was the ability to have um like stuff like visualization right so you you obviously spend a lot of thought on who these personas are I mean for me one of the things that relating back to my previous experience I would have loved is for other people from other teams to be able to look at this data and notice as well with things like registry scanning and image scanning there are other features that you can leverage as well to make sure that more of your estate is kept in good hygiene and outside of just a one repo right I really like that right um you know from from your perspective you know these are these are kind of things that I think are super useful to have continuously working do you see um do you see kind of like the crcd processes just the beginning and this is more of the of the kind of where the heavy continuous uh workloads are going to go so people will spend more time looking at the results from scanning or is it going to be kind of like as you said before they'll get an alert or a pop-up in you know whatever they use saying hey something's changed like I guess that's a big question right yeah yeah and again you know there this is really really about you know split of ownership some and making our work you know the most effective collectively you know as a as an organization you are trying to deliver not just functionality but deliver the functionality in a secure manner okay um and and you know there are are really two kind of personas here you know one of the dev who is who is in charge of you know delivering the code and the other is the you know I would say the security engineer or or those who are tests with with the with the security okay or of uh of you know the infrastructure and and the whole solution uh because uh because they still need to have a tool where they are seeing the whole system through uh uh um from the security perspective okay and and this this part okay of the of of the solution okay is really more talking to them okay that that the monitoring part okay that's whether something that might have slipped through the cracks or or something that wasn't delivered in through the right channels are are getting into uh into the production uh either you know you can't you cannot say as a security engineer that well um someone was able to to deliver to their production not through the githubs therefore this is not my problem okay obviously the security engineer will look at the actual uh you know production system and he needs to monitor it and but having said that okay and and I I really you know believe in that that even in this case uh when the security engineer identifies some issue in the production system uh we believe that um that that they need to be able to tell the same language as you said before with the developers right so they need to able to point them in the right direction it it has to be a very very short uh uh you know Circuit of discussion okay here to be able so they have the same language they have they see the same issues and and and and and this is you know the direction we are we believing it's interesting and I'm smiling because it's detected that one of my own repositories one of my own uh pieces of code has vulnerabilities in it just really funny um which I'm sure it does so if we go to this is actually a good proof because we go to Watchmen which is a project I've just been writing for cubecon and we go to the go mod um there there is a vulnerability in one of the um one of the libraries that I'm using in here which is the Prometheus client and uh it's quite cool so I think it's this one here this client go line Prometheus yeah um and it's quite cool it's picked that out and it's also identified that it's related to a um a particular cve so you know I had no idea and of course now knowing this I'm going to go do a go get upgrade or I'll go think about what I'm importing into my images so already you know I think even as an individual user it does make me more conscious and I think what's interesting is you know we look at the checks on GitHub and we use this as like a marker of prestige right like all the tests pass everything's beautiful all the linting passes we should be thinking about that about security as well right so that all of the tests uh and on the controls have been tested should have also passed and you should feel good about that right and I think that is the way that we make this work is that we Design This to I don't want to say gamify it but we certainly make it something people feel proud about right that they consider that as a just think about five ten years ago testing was such a hard thing to get people to consider right but now we've had an explosion of quality and testing and now we consider it as a first class you know piece of our our consideration when we're building software it should be the same for security yeah yeah I I think that that's really you pointed in into one of the most beautiful things okay of this that that that once that you know sometimes like 20 years ago okay you are and no people were left thinking okay of of security and and testing as as being uh you know a fancy thing okay and I think that that I always said to myself that as an engineer okay when I felt that I wasn't challenged enough okay I found something to make make ourselves more effective and more interesting of uh for example through automation okay automating okay the way we work and and you know the things which are not challenging okay let's save time on that and then and and make them worse so this is really what's happening today in the sense that that today not just you know unit testing and Company automatic component testing and so on integration testing is is has evolved but also the security tooling the automation has evolved and and you really can optimize very uh like I said boring stuff also uh uh and make them you know interesting and work fast and create a more quality of work as a developer uh as before you've reminded me of a um of a maxim that I once heard and that is create a pit of success you know you want to make it so people fall into it and it's super easy and I think you you folks are are on the right track there and what's awesome is that people can go off and try this right because it's all it's all available on GitHub and you can play around with it and join the community and which reminds me I have a final slide uh so if you are interested in using um cubescape or chat to these folks check the QR code visit their GitHub equally my uh my Twitter or Ben is also equally I'm sure happy to answer questions but I think that's that's a wrap for today right I think that's everything that's real rap I think that and and you know cubescape is an open source project it's a it's a Community Driven project and and you know we are really looking forward for you know for any feedback okay or contributions and joining uh our community Unity I think that we are making something really interesting awesome thanks again bye-bye thank you very much