Streamline Your Document Workflows with airSlate SignNow's Pipeline Crm System in IT Architecture Documentation

How to create outlook signature

all right guys in today's session we'll be designing a ci cd pipeline now with the cicd pipeline developers can make changes to code that are then automatically tested and pushed out of delivery and deployment the acronym ci cd has a few different meanings the ci and ci cd always refers to continuous integration now continuous integration is the practice of integrating all your code changes into the main branch of a shared source code repository and automatically testing each change when you commit or merge them and automatically kicking off the build once code has been tested and built as part of the ci process cd or continuous delivery takes over during the final stages to ensure that it can be deployed packaged with everything it needs to deploy to any environment at any time cd or continuous delivery can cover everything from provisioning the infrastructure to deploying the application to the testing or production environment cicd is a huge topic and it's really important for you to make sure you are able to filter down or clarify all the core requirements your interviewer is looking for during the system design round and this doesn't just apply to this system design but any system design interview you are going for having said that for this specific system design round we'll be building a simple ci cd pipeline that takes the code builds it into a binary and deploys the code globally we are not really worried about testing here so once the code is merged to let's say the master branch we want engineers to be able to trigger the build and to clarify we are not designing a system to submit and review code but a system which can simply build and deploy the code now it's very important to organize ourselves and to lay out a clear plan regarding how we are going to tackle our design what are the major factors which are driving our design now it seems like this system can actually be very simply divided into two clear subsystems a build system that builds our code into binaries and a deployment system that deploys binaries to our machines across the world now these subsystems will of course have many components themselves but this is a very straightforward initial way to approach a problem and so that is pretty much our functional requirements that is build code and deploy code where engineers can write code commit code and deploy and then they can refer to the specific version of the code through the sha which is basically the pointer to the specific comment as far as the system requirements are concerned typically during any system design scaling scale is one of the factors and so we need to ensure that the system we are building is highly available highly scalable most of the time and so in this case also that is the assumption so think of this ci cicd pipeline is something we are building for a huge company at a scale of google or facebook or amazon so at a minimum we are going for five regions and let's assume there are about 100 000 machines we have we have about maybe the deployment time takes at least 30 minutes per deployment and we would at least need two to three nines of availability because even though it's a ci cd is most of the time used internally by the companies but cicd pipelines drive the amount of defects that it can solve so coming to the high level design typically when engineers commit their code as the commit comes in the build jobs are triggered the build is processed by some servers and finally a binary is produced the binary can be a docker image it can be a jar file and this binary is then finally stored into certain location most of the time people use things like artifactory and yeah so this is basically what is going on at a high level now one important thing to note here is that all these jobs are run in an orderly manner that is first in first out basis so whenever an engineer commits a code their code will be processed first and then the subsequent comment committed code will be processed later and so we would need to implement some sort of a queue here to make sure that the order is maintained so once the build is starting to process these builds are being processed by for surefy some servers and these servers are let's call them worker nodes and these workers are responsible to process the build that is to compile the build and whatnot and then finally store the build package the build and store them into certain repository like i was saying typically we use engineer use artifactory but in this case let's go for amazon s3 which is a blob store you can either choose google or seo blob stores as well but for this system design since i'm comfortable with s3 i'm going with s3 plop store so this is what is haptic at a high level engineers trigger the build or any comet triggers the build they are processed by these set of workers these workers process the build in a first in first out basis and then these wheels are being stored in s3 now since availability is really important in our case we also need to ensure that we have some sort of health check service to monitor the health of all these worker servers that is if they go down we need to ensure they are auto scalable and if they go down if any of the servers go down we bring them up in some way so definitely we need some sort of auto scaling going on here we don't want all all some or all the servers going down at the same time and which is what i have uh these health check servers here to monitor the health of the service now as far as implementing the queue is concerned one of the most easiest and efficient way to do that is to represent the queue in a table in a tabular format where every job is a record in the table it could be a as simple as a mysql database table where we store um all the jobs so i have the stable build jobs here with various attributes from id name the sha that is the pointer to the commit the time the build was created the status of the bill and the last scene heartbeat of the worker servers because remember we have this health check to ensure that the servers are up and healthy so it basically sends a heartbeat to the database making sure that the last scene time is not big enough to trigger an alarm that there is a problem with the server so let's say the last scene of any of the build servers is any of the worker threads is more than say 30 minutes in that case there will be we can probably trigger an alarm making sure the auto scaling group kicks in and checks if the servers are up and running if not then increase the number of servers so using this tabular mechanism to implement a queue we can implement the actual dequeueing mechanism by looking at the oldest creation timestamp with a queue let's say with a qt status and this means that we'll likely want to index our table on both created add and status column now as you can imagine this table will be used by hundreds if not thousands of workers and since we are using a mysql database uh it guarantees asset transactions that will make it safe for potentially all these hundreds of workers to grab jobs of the queue without unintentionally running the same job twice and basically avoiding any kind of race condition so overall our actual transaction will look something like this so a worker note when it is available it will go to this table and look for a java status queued basically the job is just not running and it also ensures this query also ensures that the job is one of the first job that was queued because remember we have to follow first in first out order and which is why we have this order by 4 and then once the worker node picks up the job it will update the job and status status to running to ensure no other job no other job picks it up no other worker node picks it up and basically commits it and this entire thing is in one transaction so there's a beginning end to the transaction which ensures that there is no risk condition no other worker node is trying to execute the same code with same job id and and that's that's what it is and if at all there is nothing cured uh there is no job available for the worker node to run it's gonna basically roll back so now let's try to estimate the minimum number of workers we would need to run our ci cd system without any hiccups for this large company say if we have about 5000 bills per day and let's say each worker can process up to 100 bills a day so the total number of workers we will potentially need is 500 which is at least 50 workers and one thing you should remember here is that the bills are not consistently done across throughout the day because uh it might be the case that engineers are usually running the bills towards the end of the day or maybe in the beginning of the day what it means is the build traffic or the number of worker nodes required will not be consistent throughout the day there might be times of day let's say maybe during the evening hours where where developers are making more comments there might be more build jobs which are triggered and so we might end up needing more uh worker nodes maybe we have to vertically or most likely horizontally scale the number of working nodes so depending upon the scenario we might we should always have this option to scale the number of uh worker nodes meaning we should be able to automatically add or remove the number of workers whenever the load requires it and we can we can in fact also scale our system vertically by making our workers more powerful and reducing our build time all together so once the working node completes a build now it can store our binary into amazon s3 before updating the relevant row in the jobs table this will ensure that the binary has been persisted before its relevant job is mark succeeded in our sql table in our relational database table and since we are going to deploy our binaries to machines spread across the world it is likely to make sense to have some sort of regional storage rather than just a single global blob store and and we can design our system based on regional clusters around the world uh in our five or ten global regions and each region can have a blob store a a regional that is a regional s3 bucket and once a worker is successfully stores a binary in our main blob store the worker is released and can run another job while the main blob store performs some asynchronous replication to store the binary in all the regional gcs buckets in all the regional s3 buckets and given there are five to ten regions and say 10 gb files this step should take no more than five to ten minutes and which which basically brings our build and deploy duration to roughly 20 to 25 minutes that is 15 minutes for a build and 5 to 10 minutes for global replication of the binary which is most likely be asynchronous now our actual deployment system will need to allow for the very fast distribution of all these 10 gb binaries to hundreds if not thousands of machines across all our global regions now we can update the table with status successful only when the binaries are replicated across all the regions because if we do it before that and the replication fails for whatever reason the system can become inconsistent and so most likely we'll also need an additional service that tells us that when a binary has been replicated in all the regions so here we have a deployment system where we can have a global service that continuously checks all the regional s3 buckets and aggregates the replication status for successful builds and in other words uh it checks that a given binary in the main blob store has been replicated across all regions and once a binary has been replicated across all regions this service updates a separate sql database with rows containing let's say the name of the binary and the replication status and once the binary has a complete replication status it is officially deployable and for our blob distribution since we are going to deploy 10 gbs to hundreds if not thousands of machines even with our regional clusters having each machine download a 10gb file one after the other from a regional blob store is going to be extremely slow and so we need to take a peer-to-peer network approach here which will be much faster and will allow us to head the 30 minutes time frame for our deployments now let's describe what happens when an engineer presses a button on the same on let's say some internal ui that says deploy build or binary to every machine globally this is the action that triggers the binary downloads on all the regional peer-to-peer networks now to simplify this process and to support having multiple builds getting deployed concurrently we can design this in a goal state oriented manner the goal state will be desired build version at any point in time will look something like this a current build b1 and this can be stored in some dynamic configuration say a key value store like zookeeper which will have a global goal state as well as a regional goal state so each regional cluster will have a key value store that holds configuration for that cluster about what builds should be running on that cluster and will also have a global key value store so when an engineer clicks the deploy p1 button our global key value store build version will get updated and the regional key value stores will be continuously polling the global key value stores say every 10 or 15 seconds for updates to the build version and will update themselves ingly now machines in the cluster regions will be polling the relevant regions key value store and when the build version changes they will try to fetch that build from the peer-to-peer network and run the binary so guys i have used a lot of terminologies in today's system design exercise uh it's auto scaling indexing transactions or peer-to-peer networks and this is something which i've covered in my system design playlist the links are in the description [Music] you