Add Employee Matters Agreement Mark with airSlate SignNow

Add Employee Matters Agreement mark

hello everyone okay great to have you all here um my name is mark coleman and i'll be speaking to you today about design patterns for error handling in c plus plus programs using parallel algorithms and executors and uh before i get started um this talk will be i'll take for about 25 minutes um if you could please um i'll hold answering questions until the end but uh please feel free during the talk to type in remote in the q a section to add questions there and if you could include um the slide number or title when you ask the question that would help me answer them i'll answer questions at the end immediately after the talk there will be an ama ask me anything um that will happen in the same room and i will just continue with the same stream and keep talking um so just if you're interested in attending the ama please just stay in the same room keep connected and we'll continue there thank you and my name is pronounced um i included some ipa there and my pronouns are he and him um so i have about 10 years experience after my phd are writing parallel c plus for science and engineering applications my background is in parallel algorithms for solving large linear algebra problems i'm really really new to see standardization processes my first standard committee meeting was at the end of 2017 and i just started a new job at stellar science in march of this year um today i'll tell you about how first how parallelism makes it harder to do error handling to deal with errors and specifically c plus parallel algorithms and tasks make it harder to handle errors i'll then talk about the message passing interface for parallel programming in a distributed memory sense which has been around for about three decades mpi um experiences from mpi teach us design patterns to detect and handle recoverable errors and i'll outline some of those towards the end of the talk when i say parallel i mean using multiple hardware resources nodes cores vector units cmd units to accomplish more than one work item at the same time in order to improve performance performance could mean latency throughput or responsiveness parallelism hinders error handling and that's because parallelism relaxes execution order and it does so deliberately to improve performance i o to daisy holman whom you might know the distinction between parallelism and concurrency parallelism relaxes execution order to go faster concurrency constrains execution order in order to make it easier to reason about what's happening when an error happens it interrupts the normal flow of execution and in order to handle the error it constrains the order of execution you have to change the control flow errors could actually lead to deadlock which means waiting forever for example if one parallel worker drops out before a collective synchronization between multiple parallel workers the other workers may wait forever and the correct handling of errors requires communication when i say communication i mean either data movement or synchronization they're really the same thing data movement implies causality which implies synchronization and we might need to communicate for a couple reasons for example we might need to stop other workers from waiting forever on us or or we might need to propagate and combine error information from other workers now there's really no free lunch there's no zero overhead solution to this error handling requires communication communication is expensive it costs a lot especially on modern computer architectures making c plus plus do it for you automagically will not be free and so as c plus developers often want a zero overhead solution if that's what you want you the coder will need to handle errors now standard c plus plus offers a few different ways to get parallelism just in the standard first are the parallel algorithms that are in c plus 17 these include algorithms like for each reduce transform and sort the first argument of these parallel algorithms is an execution policy and that specifies the permitted changes in execution order from the normal sequential order and for all of the execution policies currently in the standard if you throw in your parallel loop body in the element access function that you give to the parallel algorithm terminate will be called there are also asynchronous tasks for example c plus 11 has async and when you launch an asynchronous task uncaught exceptions in your task get captured and waiting on a result throws the past along exception in the future and there are also proposals to add um more general asynchronous tasks for example there's p0443 executors and it's companion paper the synchronous algorithms paper p1897 and executors have a separate path of the so-called error channel for handling an ancestor task on caught exception there's also an algorithm when all and when all allows a task to express a dependency on more than one task so a task can depend on two or more or parents and when more than one parent task throws the when all algorithm captures only one of those exceptions it drops the rest so as you can see two examples of of how exceptions can cause trouble in your parallel code i mean a parallel algorithm if you throw in your loop body instead of the exception propagating as it does normally in a sequential algorithm terminate is called and in when all if more than one parent throws all but one of the exceptions are draw so you may argue well if i'm writing parallel code i care a lot about performance and so i shouldn't be having exceptions in my parallel code i should have written my code not to throw and that's an argument we can think about um exceptions are for recoverable errors really if we have non-recoverable errors maybe terminate is good but let's think about the kinds of code the code characteristics that can lead to us wanting to handle errors that can lead to us wanting to deal with exceptions i'll read this diagram this big chart from the bottom up and on the left side i see what is the status of exceptions in that level of code and on the right side we see the typical kinds of things that level of code might do so at the very lowest level of parallel code exceptions are not allowed even if we catch them you're just not allowed to throw or catch or do anything and that that happens in code that like explicit cmd intrinsics or dialects of c plus plus for running on gpus above there there's a level of abstraction where exceptions usually indicate bugs contract violations that kind of code looks like tight well-optimized loops i've thought really hard about how to make the code go fast and so any exception that i might encounter there is means i made a mistake i mean a bug but above that level of code we often see code where exceptions that are not bugs that are not contract violations and other kinds of recoverable errors are more likely this kind of code might call third party libraries of unknown provenance it might do input and output operations it might do speculative computations and i'll show you an example of code like that that wants to live in a parallel loop in the next slide this example is an algorithm called domain decomposition it's from my background in solving large sparse linear algebra problems domain decomposition is an algorithm for solving a big system of linear equations ax equals b and in domain decomposition we do that by decomposing the big linear system into many small linear systems called subdomains we solve those small systems independently and then we combine the results domain decomposition is an approximation and so we may have to repeat this process until we converge until we get close to the right answer now domain composition is not foolproof it might fail and it might fail for any of various reasons the entire problem might have no solution mathematically or it might take too many iterations to get to an accurate solution or some of the small systems may not have a solution at all mathematically even though the whole problem does or we might fail to solve the small systems for some reason other than their mathematical problems for example we may run out of resources now there are other algorithms that we can use to solve big linear systems these fallback algorithms like a sparse lu factorization can take a lot more memory and time so we would prefer for domain decomposition to work but still we do have a fallback on the left i show a sketch of how we might implement a domain decomposition in the non-parallel case so the the pre-parallelized example and we just wrap the whole thing in a try catch we set up for all the subdomain solves we do a for loop over the subdomain solves we solve each one in turn and then we clean up afterwards to combine all the results and no matter what solve throws if any of the if the any of the sub domain solves fail we know that we need to fall back to a different solver and so we just say that we fall back to a different solver now on the right i'm showing how we might parallelize this algorithm now we've done two things to parallelize and to optimize the first thing that we did is to use a fixed size memory pool this is because um memory allocation inside of a parallel loop can sometimes be slow and it kind of needs to synchronize if you think about it and so here i've used a fixed size memory pool to speed up that allocation process and i've also replaced the range for loop on the left with a c plus plus parallel algorithm called four each and i'm using the par unseek execution policy that means parallel and unsequenced so i'm allowed to execute the loop iterations in any order in our domain decomposition in our parallel domain decomposition algorithm we need to be able to distinguish between two different possible errors first it could be that the memory pool is too small if the memory pool is too small well we need to figure out how much additional space we need in the pool we need to reallocate the pool and then we need to retry domain decomposition but there's another kind of error and that could be the error in which it is mathematically impossible to solve one or more of the subdomain problems in that case we need to give up on domain decomposition and fall back to the slower solver however we have a bit of a problem inside of the parallel algorithm any kind of throw causes terminate to be called and that means that we can't have a try catch that distinguishes between the different types of errors by catching different exceptions so what do we do about this we can't distinguish between the kinds of errors so how do we tell what happened and recover from it hold that thought and i'll talk to you a little bit about the message passing interface the message passing interface or mpi is a c and a fortran interface for writing distributed memory parallel code it's a library interface so you call functions there might be macros involved in the c interface mpi has been a standard for three decades it unified divergent interfaces for distributed memory parallelism around the early 90s the first a version of the mpi standard was published in 1994 but it's an ongoing evolving standard in 2015 uh version 3.1 came out and version 4.0 is pending people use mpi to solve enormous huge problems on enormous computers you can get millions away parallelism with mpi npi is a stable interface code from the 90s just works it has modest hardware requirements you really just need some kind of computers hooked up to a network you can run it on those on a raspberry pi cluster you can run on all sorts of computers and mpi cooperates with other programming models for example threads or for gpus now mpi is a distributed memory parallelism model and the idea is that you start with p parallel processes they have a fixed location in hardware there's a fixed number of them p and so it's a little bit like your entire program is running in a parallel for each over the integer range from zero to p minus one the processes are distributed they do not share memory at all and so in order for the processes to communicate you as the programmer need to do something explicitly and the main way that you do that in mpi is by messages these are explicit function calls you can call a function to send something to receive something or to communicate something altogether and the calls are two-sided all of the entities all of the processes that participate must call the function so if i'm sending i need to call something and you the receiver need to call another function that corresponds to my send messages can come in point-to-point forms send and receive or they can come in collective operations like reduces all reduces scatters and gathers and on the right side of this slide i show a diagram illustrating four mpi processes and they're sending and receiving messages the messages do not synchronize with respect to the collective communication and you can see that process three when it sends process zero receives process zero and process three both need to do something and all of the processes that participate in the collective need to call a function as well mpi is hostile to error handling i would call it a bit more hostile especially for c plus developers than even c plus parallel algorithms are hostile mpi in fact deliberately punted on error handling compared to competing programming models such as pvm the parallel virtual machine mpi's goal was really to portability and performance mpi wanted to run faster and it wanted to do so in a wider range of computer systems especially the the early distributed memory parallel computers were were very primitive in terms of their capabilities in terms of the kinds of things that their network and the processors could do and so the goal was to run run faster and to be able to run on all of those systems as well in fact the mpi equivalent of terminate in c plus plus is only best effort so if you call mpi abort on one process hopefully it stops all the other processes but there's no promise that it does so and if you have a c plus exception that's thrown and you don't catch it and it percolates all the way up to the top um that's undefined behavior for an mpi program and often in my experience is that if you have an uncaught exception on one process it causes deadlock because the other processes are waiting for that process to do something that it never does there are some ways around that you can get the terminate handler to call mpi aboard but it's again it's only best effort and so because mpi is such a hostile environment for error handling we have library and application developers who use mpi have learned some lessons from that and i think those lessons are good for sql plus parallel algorithms and tasks so i'll present those lessons to you now the first is to turn exceptions into values before they risk breaking control flow or percolating up without being caught the second is to turn a parallel four whose loop iterations might fail into a reduction and you can reduce on did everybody succeed and you can also reduce on other information that you can use to help with reporting the error and or recovering from it the third lesson is to prevent synchronization related deadlock if you find yourself needing to synchronize in a parallel algorithm or task you could use that as an opportunity to communicate error state because synchronization and communication are really the same thing and finally a lesson from mpi is to exploit out-of-band error reporting the first lesson is to turn exceptions to values and reduce over work items in c plus 17 parallel algorithms there's no good way to get an uncaught exception back to the caller it just calls terminate so you can't propagate that information well what do you do you turn that exception into a value like an error code by catching it and returning a value you then reduce over those return values to tell whether all the work items succeeded and you might do that by changing the algorithm so instead of calling for each for example you might call reduce or transform reduce now there are some algorithms that are not easily convertible into reductions sort is a good example of that and i'll explain later on how to handle algorithms like that now if you're using executors tasks with a when all algorithm that combines the results from multiple parent tasks the when all algorithm drops all but one exception from the parent tasks so if you need information from the other tasks exceptions what you need to do is turn those exceptions into values for example via the let error algorithm that's described in 18 97 and then your next task will reduce over those values the transformation of those exceptions into values so it's really just like it's just like the parallel algorithms case i'll describe here how i can take the first attempt at a parallel domain decomposition in my previous slides and i can use this reduce over error information technique to get it to recover from errors so in this algorithm i change it first so that instead of the solves throwing they return a result result is a struct that has two fields and first there's a status field that's a bit field that explains why the solve failed and the second value in the struct is the number of bytes that we needed from the memory pool whether we got those bytes or not depends on whether the solving succeeded of course so the solves return error information and then we reduce over those error information over the solved results so i've transformed my for each algorithm into a transform reduce algorithm and i include um in the large red box i include this reduction that takes the two results and combines their bit fields using bitwise and and combines the number of bytes needed using using addition and so if the solve failed because we ran out of memory now we have the number of bytes needed total and so now we have the new required tool size we can try again now deadlock getting processes to wait forever is really easy in an mpi program that's because the entire program is a parallel for each and so you find yourself needing to communicate and synchronize more often than in c plus plus where you have parallel algorithms that are separate and have a four joint semantic so in mpi the entire program is a parallel for each you find yourself needing to communicate more each involved process must participate in the communication operation so that in the example diagram on the right i show the sequence four processes running process three sends to process zero if process zero throws an exception before it can post the corresponding receive operation and process zero will not participate in the send and receive that could make process three wait forever on the send the send will wait until the receive actually runs there's a bit of a subtlety there that i can explain in the ama later and so process three could wait forever there also process zero could wait forever because if process zero catches the throw but fails to post the receive and then process zero participates in the barrier process zero will wait for process three to wait on the send for process zero deadlock and parallel c plus plus as i mentioned is perhaps less likely because the way that you write code is more small parallel region after small parallel region versus mpi where everything is a parallel region but it's still possible so for example if you're if you're using a thread barrier where every thread must reach this barrier before any of the threads may proceed that's a way that you can have deadlock and parallel c plus plus an example of a thread barrier in c plus plus 20 is std latch or std barrier those can be implemented with an atomic counter where you start with the number of agents of execution and each agent of execution decrements by one that counter and when the counter reaches zero all the threads know that it's safe to go on a good use for a thread barrier is in managing a shared resource for example you may want to make sure that all execution agents have stopped using the shared resource so that a coordinator agent can release the resource or reallocate it make it bigger and so in this example on the right agent zero throws before it could participate in the thread barrier and so all the other threads forever and the way that we can avoid deadlock in code like this is to is by going through the motions so we can think of a parallel loop body as a sequence of local blocks punctuated by synchronization always participate in synchronization so don't throw and prevent yourself from participating in synchronization catch it participate in the synchronization but give each of those local blocks a bypass if there's an error do nothing harmlessly and pass along the running error state this looks a lot like the error channel in p043 executors and there's a good reason for that you can kind of think of it that way you can also treat synchronization as an opportunity for error reporting for example if you are implementing a thread barrier using an atomic counter starting from zero instead of the s2d latch example each execution agent that encounters an error can add a big number instead of just one to that counter and so the final count is bigger than the number of agents that would tell you that some error occurred the final technique that we've learned from mpi for handling errors is something that i call out of band error reporting here when i say out of band i mean asynchronous or not synchronizing with respect to whatever else you're doing mpi has this interesting function called a non-blocking barrier mpi barrier that's kind of a funny phrase non-blocking barrier what does that mean it means something like you check in and when you're ready to wait on everyone else then you can wait so it's kind of a two-phase process now collectives in mti are not ordered with respect to other kinds of communication so messages it's not like a memory fence it's different messages can cross over the barrier um also when mpi says non-blocking it's really important to distinguish that from makes progress in the background makes asynchronous progress mpi does not promise anything about making progress in the background communication doesn't have to happen until you call a function that says wait or test until the thing is done so it's a little bit like in the c plus plus standard async if you don't specify a launch policy to async the standard could decide not to do anything until you call get on the resulting future however you can poll to force progress in fti so you can start the barrier and when you're ready to wait on the barrier you can call test test task test until it succeeds and mpi is required to make progress on that and so in this case we can use this non-blocking barrier technique to test whether an api process through or dropped out of communication i show that here in the diagram on the right so the way that that works is first each mpi process runs some risky local work some local work that might throw on the right i show this as this diagonally shaded blue diagram it shows up a little funny on the street view but it's kind of this blue region and those blue regions aren't synchronized across the processes that's why you can see that they end at different times and when each process finishes that local work it checks in by calling this non-blocking barrier function and so the different processes checking at different times and then the processes wait on whether the other processes have finished and they do that by spinning on the mpi test function with the timeout process can also do other work speculatively while spinning if there's a timeout you assume that the other process some other process may have died so you call npi abort to stop the program if if the test succeeds you know that all the processes have checked in so they all made it safely through the risky work in c plus um the equivalent of out-of-band communication or the generalization of that idea is atomic updates in c-plus-plus lock-free atomic updates do not block so you can use them anywhere in parallel algorithms or tasks and you could stash away some error information later and then read it when you're done and this is a good thing to do in sort or other reduction or other algorithms that are not easy to transform into reductions you can also use it if you don't want to pay for the reduction so if errors are rare it's it's not really a zero overhead abstraction to to call it a a reduction um it's true that atomic updates may hinder some kinds of compiler optimizations but um that may only be a concern if the loop is really too optimized for recoverable exceptions so it's less of a big deal so just to summarize um c plus parallel algorithms and asynchronous tasks make it harder to do and like error handling make it harder to discover errors that errors have occurred and make it harder to recover from them and so we can deal with that by using design patterns that mpi parallel programmers have developed over time for example we can turn exceptions into values we can reduce over error information we can avoid deadlock due to synchronization and use synchronization as an opportunity to communicate error information and we can exploit out-of-band communication such as atomic updates to detect and report errors and i'd like to thank my new employer stellar science for funding my efforts and giving me the opportunity to give you this talk and we're hiring modern c plus software engineers and uh thank you all for for visiting and just uh i'm going to answer questions now let's see if you have any questions do you have questions oh thank you yes please post your questions in the q a tab thank you well they don't have any questions aren't you curious about the spelling of meep meep they're not curious about the spelling of i'm going to tell the story anyway so paul julian was the background artist for the roadrunner cartoon with wiley coyote and the picture you see on the screen is actually a real road runner that's what they look like um i live in albuquerque we we have these in our yard they come and visit us in fact one of them raised a chick in our front yard because there was lots of grass in which you could hide and in the cartoon um the the background artist for the roadrunner cartoon spells the sound that the road runner in the cartoon makes says h-m-e-e-p now you know awesome and if you have any detailed questions i'm just going to stick around for the next half hour and i'll i'm not going to change anything i'll i'll keep connected in the same way and stay in the same room oh good you have a question how would one write a custom executor um uh i would recommend reading um the executor's paper for that um i'm not gonna give a tutorial on writing an executor um that's a little bit that's a little bit out of out of scope for this talk but it's a really interesting question i'm more interested as a as a parallel as someone who writes parallel algorithms i'm more interested in the programming model that's presented to me rather than implementing the programming model i think that's also a very interesting question but that's a little bit out of my expertise where is stellar science located um stellar science uh has two main offices um our biggest uh and our home office is here in albuquerque new mexico and we have another smaller office in virginia and um we have a lot of remote workers um i we and we've been hiring through this whole time and we're still hiring and so if you're interested uh get in touch with me i think we have a little over 100 employees right now but we've been around for about 20 years okay well um looks like the end of my half hour um as i said i'm going to stick around here for the ama um thank you all for attending um if you want to go to the next talk and i want to stop you but feel free to stick around and ask me questions thank you you