Save Attestor Default with airSlate SignNow

Save attestor default

hello can you guys hear me yep okay great everybody so we're going to talk about spiffy inspired today a little bit deeper dive into spire my name is Scott Emmons and I work for Scytale my name is Liliana Birnbaum and I also work for Saito so today we're you know cutting to cover a couple of things I mean the main thing we're going to talk about is deploying spire on kubernetes and Scott's going to give a demo showing that so you know during coop con we've had we've had a couple of talks today Evan and Sean Crampton had a really great talk about zero trust the back and forth can't you know chemistry was amazing and then tomorrow Evans also given a talk on securing multi cloud across clusters with spiffy inspire and then we're we're finishing it off with Tyler Julian and Dan Feldman talking about uber security and how they're building workload identity platform so since last coop con what we've really done it is we went back and concentrated on on getting spire running on kubernetes and that's what a lot of the work that we've been doing for the last I would say four months and some of the things that we did is we created a SATA tester so a SATA tester is using the service account token from from kubernetes and really it's for virgins of kubernetes that don't support that don't have the token volume projection and things like eks or an older version of kubernetes and with the SATA tester it gives you the ability to to use the PKI in kubernetes you don't have to write node attestation policies that are relying on the the infrastructure that you're running on for instance AWS security groups that you could write a test search for that and then the other thing we've done is we actually created another a tester for for peace at tokens and that's really four versions of one 12 and higher and peace at will give you the difference between P sad and sad is that you get you could rotate them you could expire them and you also get some stronger arbok controls around your tokens and Scott will will demonstrate that and the last thing we did is kubernetes bundle notifier so the problem we were trying to solve is how do we get the initial bundle to to the agent when it tries to do its bootstrap back to the spire server so we actually had to create a new class of a plugin for this and the first one we did was for kubernetes and it basically delivers the bundle two to two the agent using the config map and again Scott is gonna show that and demonstrate that so that's really a lot of the work that we've been doing now for like like I said the last last couple of months and all of this now is in the last version of spire that we released actually this week 0.8 so another thing I want to talk about is initio Citadel plugin that we've we have right now and review it hasn't hasn't been merged in but it's an experimental CA module and what we really wanted to do with with this do is have spire be a CA plugin into Citadel so we what we're trying to what we're showing is off mesh expansion off of an SEO mesh so you could have a workload in sto talked to a workload outside of Ischia using spire as a bridge for that so we're not gonna do a demo of that but if anyone's curious we definitely will share that I hadn't put the URL for it but it's out there so I'm gonna just go over how we do this so what we're doing what we're showing here is just initio deployment with a with two workloads you have a task application and a task API that you want to talk to you and the tasks API is is running outside of the ischial mesh and what we do is we deploy a spire server inspired agent outside of that mesh and we're using our spire see a client and the ISTE Onoda tester right do two pieces of code that we've written that allows the issue node agent two to go back and and get a s fit from the spire server and there's a little setup that we have to do first we have to get a secret so we have to get a bearer token out of the kubernetes api and and a bundle to to basically go back and talk to the token review api and again this is our our experimental first shot at this so what happens is that the envoy tries to get his kiss s vid and I kind of skipped over it but the the tasks API is already attested to the spire agent it's already gotten his identity and that's been delivered to the Envoy running in the in the docker container so what uh what the spires basically the this geo node agent will talk to the spire C a plug-in client which will then communicate back to the spire server using our our node API so that's how we could communicate back into the spire server and send the CSR requests back to spire and once we receive that spire will use the the service account token at the bundle that it received to authenticate back to the token review API and then it'll present the token that was delivered from the CA client if that's verified and you know if that's if that's a valid token then we will do the CSR request and then we will deliver the s feedback to the Envoy proxy inside of the mesh and now you could talk the task app could now talk to the tasks API and again we you know we ran this outside but you could easily run the Speier server and all of that inside of the mesh and just have a spire agent outside of the mesh this is all running as one trust domain and again this is a experimental implementation that we did and hopefully it will be merged soon so now I'm gonna I'm gonna hand it off to Scott Scott's gonna do a little more deep dive on how do we deploy spire and kubernetes all right here I'll just use the keyboard yep all right so there's a lot of considerations about deploying spire on kubernetes a lot of decisions to make so I'm just gonna out loan outline just some of the things to think about and some suggestions you'll find that most of the time spire server is going to be deployed as a stateful set it that's not a requirement but it really depends on what configuration you have setup because fire is is pretty adaptable and configurable for different environments for kubernetes most likely you're going to require a persistent volume for the data dated or a volume claim template in stateful set is a really powerful way to let kubernetes just sort of manage that volume so you don't have to provision it separately I know I'll go into a little bit more detail on that in a production environment spire server requires a shared database currently Postgres in my sequel are are supported and scaling out spire services is really fairly easy you just need some sort of load balancer or round robin DNS maybe there'll be more more ways to do this in the future but those those do work today but one thing you do need to be a the certificate store is eventually consistent it's not synchronous and what that means is if you bring up a brand new spire server that's completely new that's not using a shared file from previous or sorry sheriff volume from a previous instance none of the agents are going to know about that server it's not going to trust the bundle so it takes some time for that bundle to get propagated to the existing agents so that they can trust it and so because of that you just need to make sure that you deploy servers and rule them out and I in a kind of slow inconsistent manner because you don't want to strand your your agents for deploying the agent that's a little bit more rigid it's always going to be a daemon set in kubernetes it may require local storage for the data Durr that depends really on the a tester that's in use and I'm gonna go into a little bit more detail on that shortly and then finally we need to expose the agent workload API UNIX domain socket mouthful into the workload and this is it's it's not tricky but there are things a couple things you want to know about first of all make sure that you are exposing in the directory not just the socket file because otherwise what happens is if the agent crashes and restarts you're going to strand all of your workloads on that node because it's not the same file anymore so this is kind of what the configuration looks like for that on the spire agent we have a volume mount with read only false because we want the agent to be able to create that socket and for the type we set it to directory or create so that if that directory doesn't already exist it will be it will be created as a as a host path for each of the workloads this looks exactly the same except for two things we've flipped read-only to true and type two directory so we don't want to work load to be able to create any files in that and we don't want the workload to be able to create the directory right because you wouldn't want a rogue workload masquerading as the spire agent and don't worry even though it's read-only is true the workloads can't connect to that UNIX domain socket and and write to it so I mentioned no data to attestation and kind of how that affects potential persistence of the data der so there are several different noted testers that you can use we have the ones that Emiliano mentioned there that are specific to kubernetes the service account token and projected service account token but even though you're running in kubernetes it doesn't mean you can't necessarily use the cloud provider of testers if you're running a managed kubernetes you're going to be your hands are going to be tied a little bit more if you're just running on your own instances there isn't generally nothing that prevents you from using the cloud provider specific testers but one thing you need to be aware of is you probably are going to need a local data Durr and buy local data doreen local to the node right so a host path type of type of storage and the reason for that is that the cloud provider our testers once an agent has attested you can never attest again as that agent you need some persistent state and the reason for that if you think about how an identity document works say in AWS or Google you don't want someone to be able to steal that identity and then masquerade as your node and then be able to register as a server so for the kubernetes of testers how do we verify that the agent is who the agent says is to the to the server there's two methods that are available we can verify with a token certificate or or public key or use the token review API if you have a new enough version of kubernetes and the token review API is all that's available if you're running on google kubernetes engine for example both both of the attest errs support either method as of 0.8 and then as Emiliana mentioned the new kubernetes bundle notifier plugin so with this with this plugin we can replace both the bootstrap certificate and the upstream CA so the bootstrap certificate is a static certificate that the agent uses the first time it starts up to connect to a spire server to to get the TLS handshake started an upstream CA is an option it's not a requirement it's never been a requirement it was an option where you can use another signing Authority and instead of just having that being a self-signed certificate with the bundle notifier you can kind of replace both of those just with the notifier and what happens is that the bundle is now stored in a config map and that's completely managed by the spire server via kubernetes api so that actually is never even mounted into spy on spire server as a config map it just acts a spire server just modifies it directly through the kubernetes api their version using kubernetes object versioning so we can make sure that that it doesn't get corrupted that way that config map is mounted read-only into the spire agents and if you are using this this plugin the spire server data dirt does need to be on a persistent volume because there's some state that's that's really important there talk a little bit about registration entries for for for spiffy right so you want to put some thought into how your workload nodes will be grouped when I when I first started playing around with spire it really wasn't clear to me how do I organize my spiffy identities how do I organize things so that everything gets attested what's the difference between node attestation and workload at the station and how do i how do I do all of that so I found that creating hierarchies and groupings that are kind of natural for your workload works works pretty well for this and just kind of a final note if you find yourself creating manual entries that are using very specific node or specific information like a node UID that's kind of a good indicator that your might want to take a different approach because if you're creating many manual entries on something that's like that massive thing that probably won't be around very long it's probably not not the way you want to go now if you're doing automated type of registration then this is a this is this is a different situation right because if you use the automated registration it handles all of that for you so let's just take a really simple example of two workloads and we know that those workloads are gonna be part of this kind of set of nodes so the next thing we'll do is map that into our spiffy namespace so we'll create something called my cluster IO nodes that will represent all of the nodes that that the workload will be running on and then for each of the workloads we have my cluster do slash workload slash a in workload slash B so that kind of creates our hierarchy and our groupings and then that directly translates into these registration entries so for the for the nodes entry we're gonna we create a node entry it doesn't have a parent we specify this 50 ID that we just do we just mentioned nodes 1 and then set up a selectors for our piece at a tester which we'll be using in in the example and similarly for the workloads we'll we can translate those directly in the parent is going to be our my cluster do nodes entry these 50 IDs gonna clearly be whatever our workload is and then we're going to have some set of selectors that are appropriate for for the workload all right it's withdrew right yep ok so we got out I'm we got to move the machine so give us one second so well you know while we wait what Scott's gonna show us is the piece at a tester and the bundle notifier that we were talking about so just give it one second and yes it worked all right right well that's half of the demo right there just plug into the oh yeah whoops there's no typos there okay so let's take a look at our configuration so now that customized is part of cube CTL 1.14 I've started to use it everywhere I I think I've gotten my head around it and so let's take a look at the customization yeah Mille that we're going to use so we're going to inject a config map for the spire servers configuration and a config map for the agents configuration and then an empty config map with with absolutely no content and this is going to be used by the bundle notifier so that spire server will inject that that bundle there and then don't worry I'm not going to go through every ammo file because that would be very painful so let's take a look at the server deployment so I just in case you're not familiar with stateful sets and persistent volume claims they're fairly new in kubernetes what this entry does is whenever a node or pod for spire server comes up it will automatically allocate a volume let's take a look at the spire server configuration so here we are setting up all of the configuration for the piece at a tester so we give the cluster a name demo cluster and then here we specify what what service account and namespace is allowed or we're willing to accept agents for and if you've looked at most of the other examples you'll find an upstream CA here and you'll see we've replaced that with the Bundle notifier which we'll take a look a little closer look at here shortly and now let's take a look at the spire agent deployment so the first thing to note is you saw this a few a few slides ago this is setting up the socket that is used for the workload API and here is the entry to inject the projected service account token so you'll see that it we've set it to expire after 7,200 seconds and then as we mentioned we have a little bit more controls over a kind of traditional service account token and that we can specify an audience of who that token is valid for and then finally we'll bring in that empty config map here and just to kind of see where that is so it makes more sense when we look at the configs so we're gonna mount that in at runs fire bundle whoops and let's take a look at the agent config so this entry right here trust bundle path again if you look at older examples you're gonna see the bootstrap certificate here right this is a certificate that that spire agent uses to on its first connection to spire server and now we're gonna use the bundle notifier for that and then here just as the server-side had its configuration for that piece at a tester spire agent has its configuration as well and we just use the same cluster name and then this is the path where that projected service account token gets injected into all right so let's apply this so first let's get this running so this is just shows that we're running in mini cube and then it'll show our posit or deployed in the spire namespace as well as the default namespace and wait for it wait for it now it's the bigger fun everything flows off a little bit no it's okay up there okay okay so now we have a running Speier agent and server just great let us look at that config map that was empty so you'll see now that it has kind of format funny anyway anyway now you can see that there's a bundle CRT in that config map so spire server has injected in the bundle certificate and and made that available to the agents now I mentioned that scaling spire is is easy so let's do it all right now now we have two Spyder servers running now I'm lying a little bit I'm not using a database remember earlier I said that in a production environment we need a shared database so I haven't done that here so let's go ahead and kill that let's so I want to show one thing if you're not familiar with the way persistent volume claims work you'll see that even though we've scaled down to only one instance of spire server there's still two volumes here and with the persistent volume claims in a save Ville set if I was to scale that back up to two two servers it wouldn't create a new volume it would use one of the existing wolf the existing volume in the ordinal wall in the ordinal number for that pod right which in this case would be would be one so it's just something you need to be aware of it's a good thing for for the purposes of spire okay let us look at a client so now we want to deploy a client workload to see that we're able to get an S FID I'm kind of lazy here so we're just going to use the spire agent container only instead of running the spire agent we're just going to run sleep so that we can exec into it and and run some commands so let us apply that guy just a little bit of typing here oops oops yeah ice messed up can't see P I watch okay so we're getting an error here no identity issued and that's really simple this is expected because we haven't had created any registration entries all right this workloads not allowed to run it's not allowed to get an S fit and in fact we haven't even created our note entry so our agent really isn't even attesting correctly so let's look at what that registration entries will look like that will create if you remember my earlier diagram with the tree these are exactly those really one one side of that tree so we created a note entry for the cluster we create a selector so again if you remember within the spire agent and server configurations I had demo cluster in there so this this matches that we've attested to be a member of that cluster and then for the workload it's very similar we're going to have a parent which is the previous entry the cluster nodes we're going to give it this spiffy ID and then we're going to use this selector this is a terrible selector to use by the way don't ever use this except in a demo right because this is basically a testing that anything running in the default namespace a service account is a tested but it's only a demo so it's okay so let us run that and in a few seconds yay so we've now attested the workload so we are able to be issued an S vid and you know you can see our spiffy ID there that's it for the demo yep so that's it for our presentation so do want to throw up the other slide yeah yeah let me let me just throw up our URLs one second one more to Teru okay so yep so let's see if that yep so you could find all this you could find us at it github also our our slack check our slack is spiffy IO and that's it for us so up thank you Scott thanks any questions no questions all right well we answered everything Wow find that hard to believe it was good did really good job all right all right thanks everybody thanks and we'll post a slides later today [Applause]