Help me with industry sign banking massachusetts presentation secure
all right so it looks like it's 250 on my clock welcome everybody my name is Thomas Cameron and I'm the global solutions architect leader at Red Hat today we're gonna be talking a little bit about containers and security this is an introductory session we've got 40 minutes together and so we're not gonna be able to dive terribly deeply into containers and security but what I really want to do is talk a little bit about Who I am a little bit about what Red Hat has been doing with containers containers in general in the industry what are they how do they work what containers are not and then talk about the components that make up containers security including kernel namespaces control groups the docker daemon and how it works and how to secure it Linux kernel capabilities selinux one of my favorite topics and some tips and tricks and some general conclusions so to start off with Who am I I'm Tomas Cameron as I said I'm the Global Solutions Architect leader at Red Hat I've been doing this since about 1993 I have a sort of a cool interest in security because I actually started out my adult life as a police officer I was a corrections officer when I was a teenager I was a police officer when I was 21 when I was 24 I went holy crap I can't afford to do this anymore and so I changed careers in the IT and I've been an IT ever since I have been with Red Hat since 2005 been NIT since 93 got all kinds of Red Hat certifications before that I was a I started out of like most folks NIT I started out Novell NetWare yeah I'm kind of dating myself then I became a Microsoft guy and got Microsoft certified and fell in love with Linux back in about 1995 and have been doing Linux ever since I have spent a lot of time working on security and organizations like banks and manufacturing facilities and e-commerce companies things like that I certainly have learned the longer I've been an IT that I don't know everything but I've certainly got some impressive skaars generally though just a big old nerd working NIT so let's talk a little bit about where I've come from at Red Hat with containers we've actually been working with container technology since before 2010 a lot of folks don't realize that we bought a company called Makara back in 2010 because we saw that we needed to have a platform as a service offering the makar acquisition was eventually rebranded to OpenShift which is our container offering today our past offering today we started doing containers except we called them cartridges using SELinux control groups and kernel namespaces which should sound a little bit familiar if you're working with containers today in about 2013 though docker really started doing some amazing work and in the true spirit of meritocracy and open source we kind of realized that holy cow this docker thing has really taken off we had been doing some contribution to it we really ratcheted up our contributions to docker and last time I checked and I haven't checked in probably a month or so we were the number two contributor behind dock or two upstream the docker project an industry adoption of docker is phenomenal Dockers been through multiple successful venture capital rounds EPS era cisco EMC etcetera etcetera etcetera etc including us have all invested and and worked on standardization of containers with docker even Microsoft has announced that they will support docker containers so what are containers at a at a very high level containerization specifically docker is a technology which allows for applications like web services our database services application services to be run abstracted from and in some in some ways isolated from the underlying operating system so for instance the docker service can launch containers regardless of the underlying Linux distro which is very cool we've had the promise of software abstraction write once run anywhere for a long time and it's it's kind of worked but with containers we're getting a whole lot closer I think containers can enable incredible application density since you don't have the overhead of a full OS like you do with virtualization and Linux control groups allow for really really impressive utilization of the system control groups are not only and I'll talk more about them in a little while control groups are not only about stopping a process from taking over the system but control groups are also about carving the system up into little bite-sized pieces to get the best utilization possible and the same container can run on different versions of Linux boot you can run on Fedora CentOS can run on rel human sacrifice dogs and cats living together mass hysteria or at least really cool things for developers to do to roll their applications out so what are containers not now containers are not a panacea they are not the cure for all that ills yeah or all that ails yeah and they are certainly not yet anyway a fit for every application you know we see folks early go we should run the name in container sometimes it makes sense sometimes it doesn't if you are beholden to third party is vs for instance if you're running big enterprise databases or ERP applications or something like that those vendors may not yet and probably won't yet support those in application and containers so you know like I said it's not necessarily a panacea for everything that you want to do containers are not virtualization you can certainly run containers in virtualized environments you we run containers on bare metal machines which I do all the time so I I do get questions periodically that are you can kind of tell by the way that someone's asking the question that they're kind of thinking in terms of virtualization that's really not what it is so let's talk a little bit about container security containers use several mechanisms for security and it's a layered approach it's kind of the old onion idea you know you got multiple layers of security multiple ways of keeping bad guys out of it and let's face it at the end of the day what we really want is for the bad guy to go this is too hard to target I'm gonna go next door so Linux kernel namespaces linux control groups are c groups the dr. Damon itself has security built into it the Linux kernel our Linux capabilities Lib cap has the ability to limit activities or limit privileges that route processes can run and then other security mechanisms like a farmer or selinux I know SELinux so that's what I will talk about so let's talk about kernel namespaces I've had a lot of conversations with folks when you have sort of the stock conversation but well how did we secure it you talk about kernel namespaces and you got you know mount namespaces and pit namespaces and user namespaces and you get the kind of a blank nod and people are like yes but what does that mean and so we'll talk about that and I'll show you some examples of what these neat what these mean so namespaces are just a way to make a global resource appear to be unique and isolated and the namespaces that Linux kernel manages our mountain namespaces pid' namespaces UTS IPC network and user namespaces and let's talk about how those look so with mount namespaces what this allows a container to do is the the container will think that a directory that it has access to is which is actually mounted from the host OS is the exclusive domain of the container so for instance when you start a container with the dash V the path on the host and then the path inside of the container and optionally read write or read only argument you can mount a directory that exists on the host within the container the container sees that directory in its own mount namespace and doesn't know that it's actually on the host so the cool thing about that is if you have any sort of shared resources that you want multiple containers to have access to especially if you want to make sure that that content is not going to be modified by the containers and it's going to be identical across all the containers instead of trying to copy it a whole bunch of times you just mount it one time or you make it available to the containers so that it's always the same in every container and as an example what I've done here is on the host I cat var dub dub dub HTML index dot HTML app that's my silly web page I used docker run - i.t with the dash V argument I say take the filesystem on the host put it there on the container run an instance of fedora and my executable is bash so now you can see that my prompt changes from my user prompt to a root prompt so I'm now inside of that container and if I cat that VAR WB of HTML index dot HTML file I see the same contents inside of that container now the cool thing is this is just inside of one container if I spin up a hundred containers they're gonna see the same thing and depending on how I mount that whether I write amount that read write or read only and you should do a read only then the content of that file is going to be immutable and within the container you're not going to be able to damage that content so you can make comment it available to containers and make it read-only so that they can't the person operating the container can't do anything about it and because we're on kind of a tight schedule I'm going to move a little bit quickly so the next namespace that I want to talk about is process ID namespaces so pit namespaces really just let the container think that it's its own contained instance of an OS it's its own in a instantiated operating system so when you start a container on a host it's going to get a new process ID pit namespaces enable the container to see the pidz inside of the container as if they were unique as if that it was a new instantiation of the operating system so in the following example I launch a Fedora container running bash and pS ax command and what that looks like is when I run docker run IT fedora with the executable bash and I run PS it thinks that that bash instance is process ID 1 it's the first process it's a it's a self-contained operating system but then if I open up another console on my host and do a PS and actually there's a type of one here I apologize for that I should have gone further up into the docker process because I did where did I do it weird I do it I accidentally copied that process ID number and I shouldn't have it's actually the bash command that we're talking about is that one right there so when I show you that that bash instance on the host that's actually process ID 18 596 that's just that isolation or that abstraction of those process IDs within kernel namespaces so I also want to talk about user namespaces when you start a container assuming you've added your user to the docker group or however you've done it started as your user account so I start the container in this example is my user T Cameron but immediately once it started inside of that container I'm user ID zero because of that name spacing so you can see I run the ID command and I am T Cameron I run docker run - - or with the executable bash but my ID changes to UID 0 now am i root on the entire system no I am only root inside of my container I still need to pay attention to do smart things because I am route inside I'm a container and I can do all the silliness that I want to but this is an example of that UID namespacing where a non privileged user not through USU or anything like that but just through the privileges that are granted from within the container can have elevated privileges inside of the container through that UID namespacing now networking namespacing are Network namespaces pretty cool capability where basically it allows the container to think it's got its own IP address it's not going to be in the network range of whatever your physical interfaces in fact the doctor doctor service will set up IP tables masquerading Rule two make sure the container to get it can get out to the rest of the internet and in this example let me just go to the page I use docker inspect to take a look at the network settings IP address within the container that I've fired up and I get this address is 172 1707 when I do IP add or show on my interface my interface is sitting and connected to a network so I don't have an IP address at all that address that network namespace address gets set up and then the docker daemon is smart enough to set up IP tables masquerading so that inside of my docker container I can get out to the network and pull things down and stuff like that it doesn't necessarily depending on how you set it up it doesn't necessarily grant access to that container from the outside directly because we are doing masquerading but it's going to segregate that container off from the rest of the network stack on the host and make it safer inter-process communication namespacing our IP c-- namespaces same thing with inter-process communications essentially it just abstract them out so that within a container for instance i can do i pcs and nothing inside of the container the thing things I'm my own operating system I don't have any IP CS stuff running or any any processes running or anything really it's not doing anything but if I go to another console see right there I'm the route inside of my container but if I go to my main console on the host and I do I pcs I've actually got page after page after page after page after page of inter process communication mappings going there so again all this kernel namespacing is doing is isolating the container so it thinks its own unique little world and it doesn't know about anything that's going on in the host alright UTS or unix time sharing system namespaces allow the container to think it's it's own separate OS with its own host name own domain name and so on so if i look inside of or if I look on the host and I run the host name command that is my actual fully qualified domain name for my laptop T 540 ptc calm but if I fire up a container you notice that it changes I've got the root prompt there and when I run hostname in there it thinks that it's this randomly generated you guys have all heard the pet's versus cattle analogy this is a cattle name right it's a it's just a serial number almost for the host name for that container it thinks since its own instance its own network name and it's it's segregated from the activities of the host alright so that is kernel namespaces again the whole point of kernel namespace is really just to let the container think it's doing its own thing it's segregated from the rest of the networking and rest of the capabilities of the host so that if anybody does anything bad to it its isolated within that container now control groups allow for some really cool fine-grained control over resource utilization on the host so there's a ton of really good documentation about control groups in the kernel see groups text file and really all it allows you to do is to aggregate and partition sets of tasks and their future children into hierarchical groups with specialized behavior this allows us to put various system resources into control groups and apply limits to it like how much disk IO how much Network IO how much memory you can use how much CPU you can use so you can have some really fine grained control over what's going on in the case of for instance the Red Hat atomic server for instance which is our container specialty container server we actually set up control groups that are very fine-grained for every set of containers that get kicks that gets kicked off and basically what that does is it ensures that even if an individual container is compromised they still whoever takes it takes over that container or even if somebody just has poorly written app and their container just spins out of control you've got some Java job or you know some craziness going on or some idiot there's a fork bomb because someone always has to do that to prove something does something inside of a container we're gonna limit it to just taking down that container so for instance when I run the command system control status docker dot service I get the control group and slice information so you can see when I run again system control status docker there's my control group and it is in its own control group including SELinux we'll still talk about a little while so even if a bad guy does something sill
to it you can't apply rules to that control group that say no more than 10% CPU or no more than 10% network or 5% network or whatever you can look through the sis see group sudo directory to see what resources are allocated to your containers now there are over 8,500 entries in that directory just on my little laptop so it's not practical to be able to dive into the depths of it but essentially you can look inside of there and get information about again memory CPU block device i/o network i/o and so on in that environment I just showed that when you go to the sis FSC group directory and do a find dot pipe WCHL there's like 8500 almost 80 600 inch instances in there alright so we've talked about kernel namespaces we talked about C groups let's talk about the features of the docker daemon itself so the doctor daemon user bin docker is responsible for managing the control groups orchestrating namespaces and so on so that these docker instances can be spun up and secured because of the need to manage kernel functions docker runs with root privileges do you be aware of that now at Red Hat's we we don't enable the ability to do things like if you read the docker documentation they say to create a docker group and add your user to the docker group and once your user is a member the docker group they can run docker from the command line we actually don't enable that we would rather see you actually run things as route and let the docker Daymond drop privileges so there are some considerations for running docker only allow trusted users to run docker I recommend that so let me move back up the docker documentation recommends that you add users to the docker group so they can run the doctor commands that's fine but just be aware that there are some risks associated with that make sure you only delegate that ability to trust and users and remember that they can do things like mount the hosts file systems and potentially you know do bad things to you so again we recommend that you actually only grant privileges to the users that you want to by using a mechanism like su do for instance so you can create your Etsy sudoers file and say that they're only gonna have access to certain docker commands and things like that if you're using the REST API to manage your hosts make sure that you don't have any vulnerabilities exposed in other words keep your systems up-to-date it's kind of common sense right but a lot of times we get something working just the way that we want it to and then we're like don't touch it so don't do that make sure you keep your systems up to date and make sure that you're using strong authentication if you're doing things over rest if you're going to use the REST API over HTTP please make sure that you're using as a cell or TLS don't expose it unless and except by secured networks or maybe by VPNs so Linux kernel capabilities or Lib cab really cool technology which allows you to essentially set limits on root privileges historically the root user has had the ability to do anything anywhere to anyone right once you're authenticated you know I am I am root bow before me but Linux capabilities is a set of fine-grained controls which allow service or even users with root equivalents to be limited in your in their scope so even root users can be cut down on what they're able to do but you can also use Linux capabilities or Lib cap to grant regular users the ability to have elevated privileges without having to do su or anything like that for instance you could be granted a user could be granted the net bind service capability and they can bind a service to a privileged port in other words a port below 1024 even if they're not running as a root user now in containers a lot of the capabilities to manage network and other services not actually needed for instance SSH services cron services file system mounts things like that really not needed from within the container typically except for SSH you never need to run SSH in your container don't use SSH in your container it's dangerous it's silly it's going to get out to date and people are going to do bad see so a lot of these things are not needed by default docker does disallow a lot of root capabilities including the ability to modify logs change networking modify kernel memory and the catch all caps this admin which I'll show you some more about in a little while so if you look at and I'm sorry this is kind of an eye chart for those of you in the back this is the Lib cap page on github and a man that came through horribly badly I'm sorry it looked really good on my screen but basically I'm not gonna go into it but this is a table of all of the capabilities all of the root capabilities which can be managed by Lib cap so I've talked about a lot of them networking capabilities you know changing kernel memory and stuff like that but if you look through the the Lib cap github page it's actually really informative it's really cool to go through and see all of the things that you can limit and this is the this is actually the docker filters which use Lib cap so you can go through and see all of those by looking at that page and again you'll you guys will get this presentation at the end so you're you're welcome to go follow that link alright so one of my favorite topics I actually present about SELinux on a pretty regular basis I really enjoy it I've got a YouTube video called selinux for mere mortals if you don't like SELinux that's cool go watch give me give me 50 minutes go watch SELinux for mere mortals on YouTube and I think I'll probably change your mind or at least I'll make it so you don't hate selinux quite as much selinux is mandatory access control system processes files memories addresses network interfaces and so on all have labels that are maintained by the kernel or on as extended attributes on the file system everything's labeled and there's a policy which is administrative Lee set and fixed so the policy is going to determine how processes can inter interface with or interact with files on the file system how processes can interact with each other network ports and things like that it's a really cool technology it can be a little bit complicated but the thing about Izzy Linux that I tell folks is SELinux is really only about two things labels and type enforcement so for instance if I have the mythical service foo the foo service the executable on disk might have the SELinux label foo underscore exec type or underscore t the startup scripts might be foo underscore config type the log files foo underscore log type the data may be foo underscore data right it's actually fairly intuitive when you're doing SELinux it's all about labeling right when the food process is running it may have the label in memory foo underscore team so that's labeling type enforcement is just the rule that says if I explicitly allow the foo eggs act type for instance to access the foo config type files then when the foo service starts up it can read its config files then when I set a policy it says oh yeah the food exec type can also writes a foo underscore log types again that's fairly intuitive right you want your process to be able to run right to its log files but type enforcement says unless I've explicitly allowed it I'm going to deny it so for instance any other access and less explicitly allowed by the policy is denied so as an example if the fruit foo process running in the food type of context tries to access for instance the directory slash home slash tea Cameron which has a label of user home dirt type even if the permissions are wide open even if I have done Hammad 777 on my home directory right we'll give you the gun and point you to your foot will tell you how to do it but selinux will step in and go no unless it's explicitly been allowed I'm going to deny it so SELinux is really cool it can save your bacon in the event of misconfigurations I've seen it happen when I talk about type enforcement and labeling when I talk about the labels the labels are usually stored or are stored in the format of the SELinux user the SELinux roll the SELinux type and then optionally MLS and MCS labels so for that that mythical food service the full syntax would label the running process might be the user you user the user user object role foo type and then we can have MLS and MCS labels as s0 and c0 now when we're talking about SELinux the default policy for SELinux is the targeted policy in the targeted policy we really actually don't care about the SELinux user or the SELinux role we really care about the label because remember it's all about labeling and type enforcement so we can also ignore the MLS or multi-level security labels since that's really only used in the MLS policy which is usually only used in like Department of Defense or CIA or places like that we really only care about the type and the MCS label so think of the mcs labels really is just extra identifiers and the reason that's important is we can use this in containerized environments to provide very fine-grained control between containers so for instance these are totally different labels I got user u object our food type s 0 C 0 and then down here I've got s 0 C 1 even though those are identical except for just that MC s label from an se Linux perspective they may as well be completely different as black and white or you know whatever type enforcement says that the process with that first label is different from the second one so policy would prevent the two of those from interacting also there's no policy allowing the process running those labels to interact with the file system unless it's labeled a foo config type or foo content type or another predefined label so if one of those processes for instance was compromised and it tried to access a file on the host let's say ed Z shadow which has the label shadow underscore T by default SELinux it is not explicitly allowed it would be denied so on a standalone system running docker for instance all the containers are in the same context by default if you look at OpenShift for instance or atomic platform that's not the case each container actually runs in its own context you can do that on a standalone laptop machine you'll have to tweak your docker come to do it but you can absolutely do it on on a standalone machine but as a for instance I've got three instances running they're all running in the open shift contact or with the open shift label I should say but you've got different contexts here so even if somebody were to access the docker container process on the host even if they compromised the Dockers process and they got into one of your containers they still would not be able to access the other containers on the machine so what I'm going to do is I'm going to show you an example of a simulation of somebody exploiting your docker environment so what happens is on the first line I'm logged in and I have the context let's we really only care about the uncontained type s 0 to s 0 and C 0 to C 10 23 so when I run the ID command you can see that as my selinux context what I'm going to do is I'm logged in as root root is omnipotent I can do anything on the system that I want to as root right but what I'm going to do is I'm going to use run Khan to change my running context and I'm going to change over to the open shift T label with a 0 C 0 and C 1 and I'm gonna run the bash command I am still route all I've done is I've just changed my SELinux context and the funny thing is as soon as I run bash it goes what permissions denied to bash RC because I'm no longer in the right context if I try to cat and see shadow for instance even though I am route permission denied if I try to touch a test file in the root of the filesystem even though I'm still rude I've just changed my selinux context permission denied if I take a look in the home directories even though I'm still route because I'm no longer in that that correct context I have changed over from the unconfined context over to the openshift context and openshift does not have selinux access to home T Cameron I immediately get permission denied I'm route well as route that's easy all I need to do is just disable SELinux right nope if I try to run seven-four-zero because I am no longer in the right context SELinux will see that and go nope permission is denied so set and force failed so selinux is an incredibly powerful capability the things that I've talked about previously obviously all really really important kernel name space is really important control groups for for keeping compromised systems from taking over or compromised containers from taking over your system really important my humble opinion not that I'm biased or anything like that because I've never presented on se Linux or anything but my personal opinion is se Linux is really the linchpin to security and a containerized environment so let's talk about some tix tips and tricks' containers are at the end of the day just processes running on the on the host right I mean containers are not magic they're cool but they're not magic so some of the things that you do want to do in a containerized environment do have a process in place to update your containers follow it it is so easy I get it it is so easy for a developer to come up with something that's like hey I got it to work it's working perfectly we're gonna throw that bad boy out in production and then I'm gonna move on to the next project it happens we know that but have a process in place to update your containers and follow it run services in the containers are the lowest possible privilege drop root privileges as soon as you can whether it's web services database service I don't care bill & ted's excellent service make sure you drop privileges use services that allow you to do that mount filesystems from the host read-only unless you absolutely positively have a good reason not to treat root inside of the container just like you would on the host watch your log files pay attention and don't don't just download any old container you find on the net Bill & Ted's Excellent container repository may have some cool stuff in it but unless you've vetted it and you know what's going on with Bill and Ted you probably want to be real cautious about downloading them don't run SSH inside of the container use the system management tools of the hosts or you know use git or something like that please don't run SSH don't run with root privileges unless there's absolutely positively no other option which case find another container use another piece of software you shouldn't run with root privileges don't disable SELinux if you really think you need to disable SELinux go watch SELinux for mere mortals and then send me an email Thomas at Red Hat comm I am available I will talk to you about SELinux don't disable SELinux don't roll your own containers once and never maintain them again have a policy in place to keep those things up-to-date and you know again don't run production containers on unsupported platforms and it's a shameless little marketing plug from Red Hat there but you really want to have a certified platform are you gonna be able to pick up the phone and go if something bad happens so in conclusion go forth and contain stuff containers are awesome I really I've been doing this for a long time I'm the first one to admit that I'm pretty jaded right you know we always here every year this was the next new big thing and it's gonna be awesome and it's gonna change everything and after about 10 years in the industry you like yeah ok whatever containers are pretty cool I don't know if I'm gonna say that they're gonna be like a tectonic change in everything we do NIT but containers are pretty cool they make app deployment really really easy they leverage some incredible capabilities of the underlying operating system and by design they're pretty secure they can be secure if you maintain them well there are some gotchas though as with every other piece of software out there it requires some care and feeding right you got to take care of your systems but well-maintained containers will absolutely make your business more agile less complex hopefully and if done right safe so thank you very much for coming I appreciate it we'll open it up for any question
if anyone's got any questions I can't see anything right now because I've got these flamethrowers in my face but if you have a question please go up to the microphone and I'd be happy to answer them if I can someone's gonna stop me though it always happens yes sir when you use drunk on to change your context would you have been able to use run con again to change it back nope that was a one-way trip that was just for demonstration purposes hey look here's your foot there's the pistol yes sir hey there I'm thanks to the talk I'm Richard from Claire engineering a box I'm just curious what workers read had done if any around regulations like FedRAMP and PCI and container configuration we've actually done a whole lot of work with with getting the atomic platform certified for you know Common Criteria and things like that in conjunction with the folks in the DoD and and and folks like that like as far as specific projects and stuff I don't have that right here with me but but yes we are absolutely aware of the requirements for that and we're working with the federal government to make sure that we are at least pursuing if we haven't already received a lot of those certifications thank you hey just by the way real quick guys if you need to reach me I don't think I put it on my slide I am Thomas at Red Hat comm you can follow me on Twitter at Thomas D Cameron if you have any questions don't hesitate to follow up and these slides will be available on the website when we get done yes sir what do you think is missing to get better multi-tenant security from two different containers on the same system what is missing that's interesting there are a lot of things that we need to get better about there are a lot of things that we need to get better about around just doing simple stuff like enforcing security enforcing updates within containers I think some of that that that glue the plumbing around that is probably something that the industry in general is weak at from from a kernel standpoint do you think there's any capabilities right now that you're missing or do we still need to run different VMs to keep tenants separated I think that no actually I think that with containerization we're doing a lot better if you look at what docker specifically and what Red Hat as a contributor is doing around Lib cap that is that is changing almost weekly it seems like so I think that what we're really having to do is spend a lot of time taking a look at what capabilities are absolutely positively needed and weeding out the rest of them I think that's probably where I'm seeing the most activity and then also selinux policy and doing things like SELinux segregation on like what we do with open shift I would like I'm lobbying internally at Red Hat to make that available for every since everything every place that we use docker it's not there yet but we're working on it Thanks thank you yes sir so you mentioned more than once to not use SSH so the problem is again SSH is like an open doorway to the world potentially you know if you secure it correctly and use keys and no passwords and stuff like that it's better but the thing is if you need to have access to your systems do it don't have a million instances of SSH running if you need to login to the host SSH into the host and then make whatever changes do you know do a docker attach or whatever but running a cessation side of the host is an invitation for disaster because invariably what's going to happen is you're gonna have an old outdated version that has security holes in it and people are gonna think about ooh I'm running PHP or Java or whatever so I'm gonna pay attention to the application and I'm gonna update the application but they'll forget to do the SSH daemon and you just eventually wind up shooting yourself in the foot yeah I mean yeah if you're not maintaining it the the thing I think the reason that it's it's I don't know if I'd say critical but I think the reason that it's that it's so common in containerized environments is you have app Debs and even if the app devs are being really smart about keeping their Java or nodejs or whatever up-to-date they're not so sad mints right so they don't even think about the SSH daemon the the guy that they got you know they tugged on the sleeve to set that up he's not involved in anymore and so that's where you see that kind of thing slide Thanks ok how are we doing on time do I need to get down to minutes ok hey thank you very much for the presentation thank you to the previous question around selinux and security mm-hmm so what additional security capabilities do you get by running within a VM or or do you believe that containers let's say Linux are truly that they do not require VMs to be truly secure so if you run your containers inside of a VM using something like project atomic or the atomic host or you know whatever obviously you're gonna get the ability to let's say for instance you're doing really really heavily multi-tenant environments and you want to have this customer have you know these 50 containers it may make sense for you to spin a VM up for them so they can spin up those 50 containers you've got your control group set up inside of that VM maybe you're even using control groups and the underlying hypervisor to make sure that that VM doesn't lose its mind you know so potentially there are gonna be cases where it absolutely makes sense to use virtualization in addition to containerization I think as we get more sophisticated with containerization I think don't hold me to it but I think we're gonna see less of a need to have segregation at the VM level I think we're gonna get to the point where you're gonna be able to have these big honking hosts and just spin up zillions of containers and apply security through C groups and SELinux and so on so that you don't have to have that those multiple layers does that make sense yeah thank you very much appreciate it thank you okay one last question is there any development around the isolation of containers from a networking point of view I mean yes you can totally disable the convocation between containers but I haven't seen how I could be more granular let's say I was some communication but not yeah so we're we're looking at it but then there are also some third-party tools like oh crap I was just talking to him last night he's former red header and I'm drawing a blank as to his company name but they're using quagga they're using quagga to do dynamic routing and dynamic networking so that you can get all the way down to the individual container layer and set up like really harsh you know strict rules that say this thing can only get out to the internet and I can only get back in and they can't see each other so it's not just Red Hat it's not just docker there's a ton of folks who are working on that it's it's clearly a gap and there are a lot of people who are who are trying to figure it out open source hopefully the meritocracy will be that the the best one will rise thank you was this helpful was this good okay good thank you very much thank you for coming I appreciate it