Save Heterogenous Initials with airSlate SignNow

Save heterogenous initials

okay so I'm going to start off this morning session talking about some where we are and also where we'd like to go both from the material science perspective in the computer science perspective in materials modeling and particularly multiscale or scale bridging algorithms as we look ahead to the type of architectures the Jeff federal described in his talk a little bit later so first i never thought i'd say this besides to come to DC for the fresh air we've had a couple months of smoke from arizona and even more closely recently so things are finally calming down there but it is also nice to come back to this conference and having been at the first few csuf conferences back to long ago it's really good to be back and to interact with all to you so first I'll kind of set the stage of where we are the workload the workhorse of most computational materials science research are still in single scale materials applications several of these have been among the first applications ported to to each new type of machine so as blue jean and Roadrunner and very sorry protectors came along some of the first applications there were came from these single scale applications so it's kind of instructive to review some of these and also kind of for those of you those of you who were at the last year's HPC workshop by kind of remind you some of the stuff for it strikes it covered in his talk so you kind of be careful a lot of times people will refer to multiscale materials modeling but what they're referring to is a kind of a more traditional sequential approach where you use lower link scale models and calculations to develop models constituent models and parameters that then you feed into higher length scale models and there's a really beautiful example of this and a paper recently published by Nathan Barton and colleagues from Livermore on this multiscale strength model that used exactly such an approach so starting at the lowest scale where our ab initio calculations are used to solve the Schrodinger equation for materials under various pressure and temperature and strain conditions to develop parameters like the the force model that you might use in a molecular dynamic simulation the equation of state that you might use in a continuum calculation so how the material energy and stress depends on the strain and temperature various parameters like that so these types of codes that the ab initio scale I'll say a little bit more next slide but there you may be familiar that they're typically solving showing your equation using a basis set for for periodic systems like materials typically with a periodic basis set of plane waves this involves a lot of Fourier transforms and various numerical algebra that I'll talk about in a minute so that the next scale MD is classical MD is now we're going from from the quantum scale of nanometers and picoseconds with md we jump up to two lengths of time scales with kind of pushing it on on some of the largest machines up two microns in nanoseconds if need be and here using these force fields that have been developed from the ab initio modeling we study things like the defect nucleation and growth interface mobility the instability development and various things and this of course molecular dynamics is just evolving particles so now said I'm Ashley we have set of particles that we propagate it's a very simple algorithm so it actually gives us the opportunity to to explore things like Fred talked about last year of how we do things such as various load balancing strategies some some resilience strategies recovering from parity errors and then also with students we're typically we may be doing billions to tens or even hundreds of billions of atoms this also pushes the limits on visual visualization and how if we have these massive datasets in such you how do we do analysis while we have the data how do we do visualization and kind of cool down the amount of information just some data reduction because it's just becoming increasingly impractical to check point say a trillion adam system that this really stresses the i/o system and you don't want to spend all of your computation time doing i oh so more and more Institute work is needed so again using using the md simulations you come up with things such as for metals where the the mechanical response is determined by the line defects called dislocations so when the crystal planes slip over each other it leaves a dislocation those are kind of the fundamental properties in a metal or you may have the material changing its crystalline phase as an iron so at a meso scale now jumping up to say hundreds of microns and nano seconds depending on what problem you're interested in you may use phase fields to a monk to model evolving phase transformations or in simple and pure metallic systems dislocation dynamics to model the evolution of dislocations and whichever one of those use these kind of determine things like the the phase fraction growth the the stress-strain relationship and then those finally are fed into the continuum level models where you need a constitutive stress-strain response to drive the mechanical behavior and then kind of closing loop here we're now back in algorithms that are driven by things like FFTs dominated by things like FFTs in the case of some of the poly crystal plasticity models so again this paper kind of carried through that this entire sequence for the case of some tantalum and vanadium metals under shock loading and directly compared going all the way from the ab initio scale up to the continuum scale modeling the experiments okay so just a little bit more in each these fundamental algorithms and I realize I pulled most of these from Livermore so we'll get to some Los Alamos work in a second so down at the electronic structure scale from again if if we're for from materials unlike chemical systems where in chemistry is simply gas phase and we use localized orbitals and materials metals looking at periodic systems so plane wave basis sets are used we're still strong solving the Schrodinger equation now this plane wave set using an effective potential so that we don't have to solve for all of the electrons and then you know it's on its face a fairly simple equation that we're solving but already it kind of points out an issue that that's pretty general for other algorithms and also architectures ended the different parts of this this equation have ideal representations that make their solution simples so in this case the kinetic and potential terms they're sparse in either the momentum or real space so instead of sticking to one or the other you go back and forth all the time with with the ffff TS and so this type of optimal data representation and layout is also an issues will see with Md in a minute when you consider hybrid architectures where you have different types of processors and there may be well it turns out there are two different layouts and organizations that you would like for different parts of an algorithm as simple as MD let's say there's also some complexity here and just maintain the orthogonality that also adds some linear algebra and then one other point is some of the nice work that they did into a Gordon Bell price was showing that the optimal decomposition strategy wasn't really obvious ahead of time you might do a simple default spatial decomposition it gives a reasonable performance then something like well you it's it makes sense you may want to minimize the surface to volume ratio to minimize the amount of communication that actually leads to a slightly worse performance and these more odd looking decomposition strategies actually lead to the optimal performance so the point of this is that sometimes it well in generally it pays to kind of keep your mind open and have the flexibility to optimize things other than just the mathematical colonel so again as Jeff is going to talk about with with the architectures we're moving towards the era where it's not the math it's not the flops that are limiting asst it's more data communication it's a lot harder to optimize things like this than it is simple mathematical kernels okay so jumping up to the next scale the classical md this is just a brief reminder of some of what Fred talked about last last year with the DD CMD code there are various ways of doing a decomposition from molecular dynamics you can do a spatial decomposition is in the code I'll talk about in a minute in this case it's a particle-based decomposition when you go to more complex force fields st. biological systems then sometimes you go to a bond based decomposition what were you you can even have more processors than particles so the examples like that or the the work of de shaw with the Anton machine and and their md code there so again as I mentioned there are the fairly simple algorithm although you can make it more complex with more complicated potential functions but if you look at things like embedded Adam method potentials are essentially pairwise interactions it's a simple enough algorithm that you can explore things like how do you how do you do parity error recovery in this case they showed that the just by periodically storing in memory snapshots of the the current system state the atomic positions and velocities then if there's there's an unpaired hierro detective you can just back up and do an in-memory restart so speak and this is also possible because the the memory footprint of these md simulations is quite small yeah ok so finally I'm going up to the mesoscale and one more examples so I mentioned in the case of metals that the mechanical response is driven by the these line defects dislocations so there instead of modeling the entire atomic system you can just focus on the dislocations and the interactions between them and their evolution and nucleation and growth and reaction processes so paradise is kind of the state of the art well is the state of the art of these types of simulations where you discretize these line defects into into either nodes or segments to a time step algorithm where you compute the forces the each of these line segments exerts on the other and propagate forward in the case of dislocations they're complicated by the what happens when dislocations interact and how they can have different reactions and this leads to some complicated Junction formation as you start out with a simple few dislocation lines and material and stress the system they multiply and grow form these junctions and eventually get some complicated mess and actually this is what limits how far you can push these simulations is just the dislocation density in your system that you need to resolve as you notice is these grow you tend to get a fairly in homogeneous distribution of these in space so how to actually balance these among processors is a challenge so this was some of the early work I run blue jean for this going from excuse me four thousand to eight thousand processors hmm there's a fairly good speed up 1.8 nearly two but then doubling that again the added gain began to diminish so it's difficult to push these these much forward in part in large part because of this challenge of the load balancing issue and the the evolution of this dislocation structure okay so jumpy back now to md i'm going to use the code that we've been working on los alamos called spasm as an example in large part because of the experience we had taking this porting it through a number of evolutions and architectures and then finally coming to the radically new types of architectures as a road runner where instead of just homogeneous collection of processors now we have to consider different processor types a more hierarchical structure and also great change since the days when this code was originally developed from the met the ratio the relative importance of computation and communication so then here it's the simple md algorithm instead of decomposing setting up your list of particles and divine data processors we divide space among processors which in this case is reasonable because we're usually looking at fairly homogeneous system so we a simple spatial decomposition is it is good and then to to do a rapid search to find the atoms the interacting atoms with each neighbor you further subdivide space into these sub sub domains that are the size of the interaction distance of the potential so this has been used over many many architectures and pushed up two trillion atoms again because the failed the very small memory footprint of md simulations just the particle the positions of loss ease of each atom role that need to be be stored so the types of things that we've studied with this or are a lot of high deformation loading so in this case a shock wave propagating through an iron poly crystal so the atoms here are again the atoms are colored by the local orientation or local coordination so the gray atoms you're in a body centered cubic lattice structure as the shock wave compresses them you can see that this blue region where the it's just the BCC lattice compressed as the shot goes along but then there's a transformation to a new a close-packed phase HCP which are the red atoms and then this can occur in two different variants so so we have these these twin boundaries between the two variants so things like this are used to to study the the mechanism of the phase transformation that kinetics for feeding into higher link scale models of the transformation the properties say the mechanical properties of the new phase and other things such as these one thing I want to point out here is the importance of again visualization and sits you both for seeing what's going on physically trying to identify new mechanisms and evaluate them but it's also been important for for debugging code in a lot of cases a lot of times there may be bugs that the rarely pop up that you can do things to identify sits a in some cases we found problems along particular processor boundaries by using visualization techniques to identify where the numerical problem originally occurs using these visualizations and they're also good for and communicating the work and especially just experimental asst we've collaborated with on some of this and describing the the work so how big is big enough for the case of materials like this in the case of polycrystalline materials like i just showed there's well-known behavior where the the strength of the material depends on the grain size so in the kind of large engineering scale limit it's been known for over 50 years that is you decrease the grain size of the material that these mechanical strength goes up is the inverse square root of that grain size so that's led the drive towards more nanostructured nano scale materials but eventually that reaches some limit because if you go down to a grain size of a single atom that's just an amorphous material and snow and amorphous materials are fairly weak so there has to be some turnover region here on it's been kind of predicted by models and then shown by both experiments and simulations that this turnover is actually in the region of a few nanometers to a few tens of nanometers so if we want to model a system like this completely atomistic alee say we need at least 100 grains diameter 50 nanometers that puts us in the billion adam link scale if all you want to do is propagate a sound wave through that just sound or shockwave through that sample it takes at least or on the order of a nanosecond time scale but typically you need to you're interested in a longer time phenomena so timescales maybe nanoseconds two microseconds with Md that puts us in the millions of time steps to perhaps a billion time steps so these are kind of the limits of what can be done with a single scale application so finally a on the single scale aspect I was going to just kind of walk you through our experience taking a simple application such as this and porting is was kind of the naive word but trying to rewrite that for for new architectures how we do this and the motivation of course is that over the years there's been a great growth in the performance of GPUs and other types of accelerators relative to to the confessional CPUs so on chips they've been adding more and more processors the CPUs but then there's also the opportunity of exploiting the GPUs for additional acceleration these in the case of Road Runner the choice was made to use the the Cell processor that's in the that was in the PlayStation it's actually somewhat resembles in some ways the old Connection Machine the CM 5 from back when I was working in the csuf program where the CM 5 had these these 8 vector units and I guess performance of a whole one entire Giga flop as we jump forward to the cell it's a hundred gigaflops with these eight so-called synergistic processing elements so again a main powerpc processor and these eight sbes and how you get those who work together is the question actually in the case a road runner since the Cell processor for the the PlayStation didn't really need a lot of didn't use this power pc very much it was stripped down a lot of things branch prediction and a lot of the power of the PowerPC was taken away so it's essentially worthless from a scientific computing standpoint so for Road Runner in addition to the cells the op some hosts nodes opteron processors were added so Road Runner which goes the first petaflop machine is this kind of hybrid cluster of clusters so a number of these so-called connected units were assembled together in the spherical network within each connected unit there's a one-to-one mapping between opteron cores and sell processors so how do we get those how do we take our algorithm which is originally written for something looking like this without the accelerated the cell accelerators and optimize that for this this now hybrid architecture so the idealized picture of doing this and again coming from some of the gaming and graphics communities is you have a number of tasks that you need to do on the on the cell and you if you can write these in such a way that you can overlap the data flow they to feed down to the cell and then up to the to the or down to the sb's and up to the power p the PPE or in our case all the way up to the cpu if you can overlap that communication with the computation that's going on down on the SBS then you get it you can amortize it and don't have to worry about that communication time so ideally you have these overlapped direct memory address instructions and computations so you can double or triple buffer the incoming data the data you're working on and the outgoing data and again hide this this computation time when it actually comes to taking that idealized picture and then applying it to to your real scientific code it became much more of a challenge because as I mentioned in this case we have not only the two types of processors on the cell also these opteron so that means three different compilers different communication libraries between each of them how you synchronize the this this data transfer is critical and actually two different types of bite or drains on top of this so when you look at roadrunner a roadrunner here you start a sort of feel like the victim here but actually turned out to be not so bad so walking you through this the this code that i mentioned spasm it was originally written back 20 years ago for the connection machine back then memory and computation were the bottlenecks so we had a hole 32 megabytes / / spark node on the other hand communication was cheap so this is something actually for the for the original connection machine that Richard Fineman had helped out on and then for the CM 5 and take advantage of work for fro MIT so there's a lot of work on network topologies and structures in the case of the CM 5 it was a fat tree the t3d which was slightly later had a three-dimensional torus which for the type of 3d spatial decomposition of md is exactly the network that you want nearest neighbor communication to each of the six nearest neighbors so the algorithm that was developed to minimize the amount of memory used was to just at any one time only have the particles within a processor in memory plus one of these little sub domains from from the adjacent processor and then stepped just march through and lockstep each of these subdomains computing interactions between pairs of between the particles within a sub domain or immediately adjacent subdomains and then doing synchronous send and receives to communicate it as needed along the way so in pseudocode this is just looping over subdomains computing the interactions between pairs of particles within one and then with each of the neighbors using these synchronous center receives inside that this loop so of course as we go forward and make the computation faster and faster and communication latency slower this overhead begins to to dominate and you have to turn things inside out so as we move forward to kind of the past five to ten years memory has become there's been more and more memory added for is driven by law the other applications before md we have memory despair so you can use it for this in in sets you checkpointing for the fault tolerance you can use it to buffer the entire boundaries cells of atoms and this is known as ghost cells or a halo exchange where you bring in all of the the neighboring cells ahead of time this leads to some redundant calculations so computing interactions the span of processor boundary those pairwise interactions are now computed on twice once on each processor but again since the computation communication trade-off it is as change that's worth the payoff so looking at an algorithm like this trying to think how we can accelerate this on a with a hybrid architecture you're spending the vast majority your time ninety percent ninety-five percent computing forces so it makes sense you have this force algorithm you just accelerate that down on the cell or the GPU so the original approach was to to just take that single subroutine accelerated on the cell as we march along we get the particle positions send them down communicate the forces send the forces back up integrate time steps to check pointing re re sort particles do whatever whatever other bookkeeping we need to and march forward and this is a typical type of interaction this embedded Adam method mentioned which is essentially a pairwise potential twice as complex there's a yeah well okay so there's a pairwise interaction that's essentially the in the simple picture the inter nuclear repulsion and then an attractive one for metals with an electron density around each atom and then a functional of that leads to the attractive bonding when you place a nuclei with in the background electron density of its neighbors so that's the simple simplest and the most widely used interaction for metals so anyway when we look at this algorithm implemented it it actually only gave about a two and a half times speedup over the base coat and the reason for this is that again when you look at the potential performance on these different processing elements the vast majority is is down on the accelerator so to optimally use it you really need to have the accelerated busy busy as much as possible any time its end the idle you're just killing performance so this kind of back and forth trade off of work between opteron and cell is damaging performance so then you turn around look at it from a cell eccentric accelerator centric perspective it turns out now that when we're focused on the computation we're going to focus on the work on the cell again the data layout that we want changes whether you have an array of structures so you in going back we make all the way here these interactions and communicating particles across processor boundaries it's most convenient to have each atom contiguous in memory so an array of these particle data structures is optimal from a communication perspective when it gets down to doing the the math down on the cell it's better to have the structure of arrays so the all the positions and he direction aligned with each other so you can just stream those through and calculate those and vectorize those down on the cell so trying this around we looked at this from from the cell sorry perspective as far as data layout put all of the work that we could down on the cell and then hid the the data transfer time with with work that we could do with with data is art that was already local so start your overlap local calculations with data transfer to hide this transfer time doing this I should leads to now a lot of dead time on the opteron which is not so bad it's actually useful for for things like doing institue analysis visualization checkpointing we can again because the memory left over here we can have checkpointing going on while we're continuing the time stepping down on the cell so we can overlap these doing this then got roughly a ten times speedup twenty-eight percent of peak overall and about fifty percent of peak in the actual colonel on on the cell here so this is where kind of the state is from for single scale materials applications on hybrid architectures now finally after this long introduction getting to multi scale and why we need to do this so again one more md example if we're looking at something like the development of the fluid instability where we we have a heavy fluid on top of light fluid in this case or the kelvin-helmholtz instability Fred mentioned last year we're interested in the evolution of this interface layer which is a fairly small volume fraction of the materials so of all these in this case what's seven billion atoms it's really only the interface which it early times is a very very small fraction and even late times is is fairly modest it's only the interface where we need atomic resolution in the for the fluid away from the interface this homogeneous this heavy red fluid on on top here we could model that just as well with a continuum of finite finite element or some other model we don't need a time of resolution there it's the same thing for for the materials problems where we in in that case in the case of iron that I showed when we have the the shock wave propagating through the material we don't need a time resolution ahead of the shock wave when the materials just sitting there or after its transform it and is saying there we really only need resolution in a fairly small amount and this is becoming more and more apparent as we go to these larger and larger simulations so then also if we kind of look where we're headed as far as what we can do now in terms of time and length scales for a single scale md application since we're resolving the time of vibrations our time step is down on the order of a femtosecond of course we're resolving it the individual atoms so is we the amount of total amount of memory that we have in the system determines how many atoms week we can just we fit in the computer so this is up currently to as i mentioned we're figure out a trillion atoms on the latest machines but it's it's pretty useless to do a trillion atoms for a few times steps not and even of town time for one into the sample to propagate a sound wave to the other side and for them to realize the size of the sudden the entire size of the sample so typically we trade off this system size for for speed and simulation time so with short range potentials where we can do these types of spatial decompositions it's a simple order n algorithm you have this linear trade-off between size and time until eventually you hit some limit communication limit where you're where you then memory or communication bound and just the the bookkeeping overhead for doing individual time steps sets how fast each time step can be so when you look at this on current machines you can push out to millions perhaps billions steps so it gets you to tens of nanoseconds if you really push you can go longer but there's essentially a time wall here that we're facing as we look forward with the projections toward exascale by definition exascale is a thousand times faster than petascale so this this linear scaling performance region goes up by three orders of magnitude the current projects is our memory may go up by two orders of magnitude but communication and especially the the the clock speeds and bandwidth and especially relative to the number of processing elements per per per node isn't going up very much it may actually diminish so this time scale problems not not increasing so either have to get over this by algorithms either by extending the time scale of md algorithms or by coupling scales and i'll talk about both of these and particularly the kind of where we are in in the state of coupling scales between these applications so i mentioned these first of all the sequential multiscale techniques where you just do an information transfer it's kind of a the scientist is doing that the the coupling developing the physical models that you feed into higher link scale models on the other hand so called concurrent techniques in concurrent techniques the coupling is done in situ and in the computer so you're linking different different scale MA bowls different types i showed the beginning so again sequential multiscale works if you have an idea ahead of time or can discover what the relevant processes are and develop models for those and if there is a separation of time and length scales between each of these so that makes sense to say we have electrons determining the interaction between nuclei we can develop an inter atomic potential for that feed that into the next length scale as user born-oppenheimer approximation and then the atoms in the material lead to a mechanical response that's dominated by these dislocation line defects so we can use the line defects as our fundamental element in the next scale that that works when that's true the sequential multiscale works great in a lot of cases turbulence is one there's a strong coupling between different these different length scales length and time scales so we can't simply integrate out the small scale the subscale degrees of freedom and feed them in the end of the course of model so we really need a coupling it's also been ported out this is a good review of years ago by Lou and kicks eros on various techniques that these models aren't just useful for this in and of themselves but also for kind of developing ways to to do the data reduction so to determine and work out how do you identify what's the essential data at the at the fine scale that is really important so an example here again it is in doing checkpoint restart so in an md simulation with billions to a trillion atoms you don't want to have to check point to write out all the positions of velocities of all those atoms at each checkpoint when it's only it may only be the interface atoms that matter away from there you just have a either a perfect solid or a perfect fluid that you can just write out the kind of the average state of the system there and then reconstruct the atomic configuration that matches that so this is one of the major challenges as we go forward is data reduction and it couples nicely into these these scale bridging techniques so the first type of methods I'll talk about there's actually a nice the I know it's the winning essay in the magazine now is about refer to some of these onion methods where there's kind of an embedding of different scales a separation of models within the problem so say a classic example has been fracture where we have a crack propagating it into a material away from the crack front we just we made table you just have an elastic solid where we can use a continuum model but at the crack tip the crack is propagating forward by individual bond breaking events so we really need atomic resolution at the crack tip so in this case that there's a continuum region an atomistic region and the challenge is how do you couple these so one common ways is to shown on the bottom here within the atomistic region have ghost atoms that are kind of come from that are whose positions are determined by the finite element in the boundary region and then the finite element region similarly have kind of these ghost cells that come from the MD so that you can do each type of each scale of simulation and then couple them through through these boundary regions in in some cases this may be carried on not just coupling to scales but multiple scales so again in the in the case of a crack you may actually need to describe these bond breaking events with a quantum method or type binding in this case so there may be these multiple scales okay so I'm just kind of briefly describe the of some of the more common more widely known techniques and actually some of the historical once that have kind of played an important role in the evolution of this one is so-called causing continuum method again it's coupling a finite element and atomic regions where the idea here is that in regions say where we have a crack or dislocations on indenter driving into a material we have a fully a domestic representation away from that we then only need if so in the center here is kind of showing this defect region that's fully atomistic further away where it's just an elastic material we don't need full a domestic resolutions so the dark atoms here on the left or kind of Representative atoms these rep atoms that kind of get the local just describe the local elastic response to the material there and then as the simulation moves forward that you these boundary regions may evolve the number of typically the the number of fully atomistic the size of the fully atomistic region grows until it overwhelms the computer and simulation ends there the common theme for several these early techniques is that it's somewhat simple to do this in static situation so if you're just finding the minimum energy configuration the zero temperature static solution this can be done adding temperature or doing dynamics and this is is a challenge I'll say more about this in a minute one early case where dynamics was done is the so-called mad approach again this is the the three scale coupling that I mentioned earlier where other or finite element md and tight binding regions ah in this case the type binding ahead of the crack-tip right where bonds are breaking md around it and then finite element farthest away in this case it's it's written as a hamiltonian where you have terms for each of these three single scale regions and then the two overlap or handshaking regions are really where the challenge comes in and how you do this in particular for quantum systems how you do hi do calculation of the energy of this configuration and treat the dangling bonds between with the with atoms that you've now cut out of the the type binding model in this case for silicon as possible do this for covalent system just by adding fake atoms that that make solve the coordination satisfy the coordination of the silicon it's more difficult to do for metals and this is still an open question coupling quantum it have an issue for metals again for the case of covalent systems coupling quantum and classical regions is is widely used in chemistry and the so called Quantum q mmm quantum like quantum mechanical molecular model simulations where their quantum and molecular regions but in metals it's much more challenging so there's I think this is the only example where well this it's possible for silicon so going forward from this coupling the different regions there in studying these the behavior of these scale bridging techniques one major challenge was in this coupling region of having a consistent propagation of waves or avoiding spurious i should say reflection or introduction of waves from this interface region so as you go from an atomistic region and then course on that too to a finite element region if you have a sound wave with a wavelength that is less than one of the finite element cells what happens is it goes from the atomistic region where it can be supported into the finite element region where it's not having this consistent transfer across boundaries is really a big challenge so this is something that there have been elegant solutions for how to do this but the problem is they're very expensive they involve some particle history memory and then they also involve kind of a their non-local not only in time but also in space so they don't scale well so you can either have a an approximate algorithm that scales well but has this spurious wave reflection or an algorithm that solves the wave reflection but doesn't scale so one approach kind of advanced in this is this so-called coarse-grained md where there's a consistent transfer between the between scales so this is written in such a way that there's a coarse graining of the it's thinking of the system atomically but then coarse graining in regions where you can to to transition smoothly to a continuum mesh so in this case there's a smooth transition propagation of elastic waist between the regions and treatment of these phonon modes and other advantages this since it's focused purely on the MD system you don't need to worry it is in the case back here about how do you have a consistent say in the case of silicon here how do you have a good system description of silicon in a tight-binding picture versus amalek and md picture versus a finite element picture you would need a type binding potential and an MD potential and finite element description Constitution model that are exactly compatible so that's all in this approach where you are looking to everything from the from the atomistic scale however it's a very complicated intricate algorithm it's been shown to be successful in some test simulations in various test cases but not really applied or extended beyond that so all of those are the so-called energy based methods where we're coupling we're the idea is to write down a hamiltonian that describes the system including these different scales the challenge here is that these are trying to do everything at once and we're coupling time scales between regions so this means that whatever our finest scale region is that's the time step that we're stuck with so if we have md finite element where our time step is going to be an MD time step so we're still have that time problem where we're it propagating our system forward in steps of a femto second or a few femtoseconds we're not going to get beyond picoseconds to nanoseconds as i mentioned these matching additions that the boundaries are very challenging and because of both of these of these factors it's very difficult to do actual dynamical simulations at finite temperature so there's another approach by ease group at Princeton based on so-called heterogeneous multiscale method where instead of trying to write down and do the coupling from an energy perspective and MD up it's focused top level downs so your system is driven forward by a macroscale solver so so it's maybe a fine of all volume or element method and the point of the sub scale model such as MD is to give you as needed the information that you need to drive this or to improve that the model driving the macroscale solver forward so this has been broken classified in their work is say two different problems Taipei where you have isolated defects in the sample and do so called adaptive model refinement where around the defects you use md to compute the properties and the evolution of these defects or type B where it's just the constitutive response of the material that we need that's computed on the fly okay so that's the state of save kind of these scale bridging materials up there's a nice review comparing actually there are many more than that are all similar different regards just considering a coupling atomistic continuum as a nursery comparing fourteen of these different methods and their performance on this axis is kind of did the the speed up that was seen and then on the vertical axis here the the accuracy of the method a few of these the causing tanam that I method mentioned earlier were kind of found to be the moat the optimal both in terms of speed and accuracy but as they point out none of these has really been pushed to scale so there's still a question of how they how they transition how they what their limitations are at large scale and how this is done so let me skip over the time okay so finally from the computational perspective and this kind of these points will be brought up more in Jeff's talk but but the issues that we're confronting or the the architectures like roadrunner eyes I showed or becoming more and more heterogeneous and hierarchical it's now we have more and more flops available relative to the bites and it's more of a communication dominated regime so we need algorithms we need programming models and tools that are also heterogeneous and hierarchical we can't have single scale bulk synchronous parallelism did the effort it takes two to even just do a goal synchronization across a billion cores is enormous on top of that resiliency is more and more problem we can't guarantee we can't assume that we're going to have those billion cores from one time step to the next again power is also an issue because of this and as I mentioned long the checkpoint restart issue is also important so finally summing up the fault tolerance and then analysis and visualization how do you do data reduction to avoid to kind of mitigate these the IO IO challenges so feeding back to this hmm method that I mentioned on the the approach that we're taking it is again based on this refining as needed to give you the consist of response using subscale models such as md finer scale models to drive forward the macro scale model from a computational perspective that this the excuse me one advantage of this is that is a task based approach where you have independent relatively independent work units so say for each of these cells as you need to compute locally the constituent response that may be an md simulation or phase field that we're we're spawning off to compute that that's a fairly well defined and contained task so we can fire that off its independent as far as it maps the header Jade is heterogeneous because we have these different scales since we have these multiple tasks it kind of addresses the concurrency challenged by instead of trying to have a billion way parallel single Scout code we may have a thousand million way parallel which is kind of which is doable where it's where we're at now and then firm the resiliency side if one of those thousand million way parallel jobs or tasks fails we can just reissue it so that's where we're trying to go let's see so yeah we're coupling these different scales there's a whole similar set of pictures if we look purely in the time domain instead of coupling different link scales you can consider for safe remodeling the evolution of radiation damage their approaches such as kinetic monte carlo where we r we have models for the evolution of defects within the material and this relies on knowing what those what the defect structures and their migration mechanisms and rates are this on the fly connect Monte Carlo is kind of accumulating this library building it up by firing off these smaller scale tasks to use ab initio or md to compute that the possible events and rates along the way so let me let me just briefly that this is as i mentioned again related to some of the work this idea of building up a database populating a database as you go along by coupling scales has been shown by Livermore and seeing those this kind of call to arms for task parallelism is addressing this so finally how do we get there the two current the way things are currently done it is just unsustainable that they're currently we not in all cases in this program is one of the exceptions but but for a lot of kind of code development projects their set of application people who have kind of a fuzzy view of what the hardware and the operating system look like a very clear picture what the problem we're trying to solve is and what the possible algorithms are for that on the other side of the computer science people who you know may be developing the architectures and the middleware have typically perhaps an outdated view of what the applications are there may be an old set of benchmarks that they're targeting their kind of a approaching from the other side this isn't really sustainable so the way to move forward here is the so-called co-design process where we actually have the entire set of people from computer science applied math for the algorithms and domain science for the applications working together to kind of optimized Co optimize these different aspects the algorithms the architectures and the applications it's times limited I'll kind of leave this just mention it it's it's not really knew that there's back 50 60 years ago the case the maniac it really was a kind of a co-design machine and in this case since was one of the first machines and it was it was built over a number of years there was the opportunity for this close collaboration between the applications people and the designers and in some cases the lines blurred so optimizing this over a number of years what was was possible so this is where we're headed since time is limited i think i'll just stop there and take any questions