Send Assorted Validated with airSlate SignNow

Send assorted validated

[Music] the second speaker in today's session is Ruben buttock women okay thank you good morning everyone I'm happy to use are still here for the last day of the conference so my name is Ramon Bartok and this is a joint work with Audrey and Maillard who was a postdoc and Rafael Cardozo and it's about validating hierarchic plans using attribute grammars so I believe that the task is quite clear so one addition means that we are given a plan a sequence of actions and we are given an HTN model and the question is if this plan complies with the HTN modern meaning there is some path in the HTN model sides that if we decompose the task we got exactly that plan so this is the task we are sewing so what's the motivation why this problem is important well maybe eventually HTM planners will find we're back to the IPC so then we need to validate what the planners are producing and even if it's not part of the competition if anybody is developing a new HTM planner it would be good to have an independent tool that can validate if the plan generated by the planner is is actually is actually correct but we can also look at the validation from the other perspective so you may know that your plan is definitely correct and you have an HTM model which may not be complete and you may be asking is this HTN model corresponding to the plan is it possible to generate it plan using the HTN model and if not maybe we need to modify we need to verify the HTN model obviously this validation techniques are somehow the first step to in general plan recognition or NGO recognition so you read you recognize a sub sequence of the actions and you are asking what's the task that that agent is going to solve and you may say okay so what's new there I mean this is not a new topic there are people trying to do brand recognition and go go again and goal recognition but when you look at the papers you will see that they aren't actually covering full HTM it's at least I believe it's almost always a subset of HD and something something is missing so I believe this is the first approach they do relay a complete validation so you may have all the constraints in HTN you can validate the plans against them and we use the concept of attribute grammars so what is an attribute grammar I guess you know from the undergraduate course the notion of context-free grammar which is very nice very simple an attribute grammar is an extension of context-free grammar and this extension is based on adding attributes to symbols non terminals and terminals so these attributes can be used to communicate information between different branches of the decomposition and they may be also connected or restricted by additional additional constraints so let me show an example of the power of attribute grammars so this is a famous context-free grammar for generating or recognizing words starting with letters aids and following letters B and then following at the C obviously we don't need a context-free grammar it can be done using finite state automata or regular grammar but it looks nice using a context-free grammar and you know that if we do a small modification in that language meaning if we require that all these letters appear in the same number so it's a ^ + b ^ + c to 2 power n we need to go away from from context-free languages we need to go to two context sensitive languages but if you use attribute grammars we can just slightly modify the grammar to generate this specific language and this modification is based on adding attributes that are counting the occurrences of the letters so we say okay generate an a lattice of a and B of letters of B and C of address of C and all these numbers should be equal okay so this is the constraint and then we just generate independent letters a and we count them and the same 4b and 4c so those of you who know how the context-sensitive grammar loops for this language it's not that natural but if we do it in it's attribute grammars we keep the same the same simplicity of context-free languages so what we suggested is using attribute grammars to model HTN so we already heard about HTN so it's a natural candidate to be described in using like context-free language is because of the hierarchical structure so that's nice but we have additional constraints in HT ends and these constraints cannot be covered by context-free languages and we know it formally so formally we know that the language generated by HT n is not context-free so we need something more and now we know that attribute grammars are enough to model to model this now this is an example I'm going to use you don't need to look into details but basically we are moving containers between locations and we have one task that is called task move move to containers which consists of two subtasks move one container and move another container and moving the container means moving two robot to some location loading the container to the robot moving the container to the other location and unload it okay and so this is the tacit I will be using and it's very natural to encode this using grammars again we are not the first ones who did it so there are many approaches that are using grammars to describe HTN so I said okay I believe it would be easy to use attribute grammars but then look at the papers and we realize that actually nobody is really covering HTN in full complexity right so that's the question maybe you will tell me that yes a rat rom as it can do it and one of the major problems we notice that these grammar approaches are not covering is what we call task interleaving action action interleaving what does it mean okay this is an example of this transfer to containers tasks and you can see it's decomposed into two sub tasks transfer one container transfer yellow container and this sub task is decomposed to task loader robot move the robot and unload the container and this one can now be decomposed just into two tasks log and unload with not need to move action sir because we can use the move action from their task but if you want to do it we need to interleave tasks degenerated from one branch of the grammar with stars with the other one okay and this is what's not context-free so we need some things something stronger there and using attribute grammars we can do it we use the concept of timeline how to do it okay so this is the task interleaving what I call activity activity interleaving so just to briefly describe how we are we in cold HT ends as attribute grammars it's actually been very natural so if we want to say that the transfer one task is decomposed to these three load move rot unload drop primitive primitive tasks we can do it using this River I think rule obviously there are always some constraints so you can see that this task is about robot R which is the same robot here and the same robot here so there are always some some equality constraints but there are some other constraint that we can express like partial ordering constraints so we have precedence constraints loading must be before moving moving must be before unloading which is we know sequence in this case and then we have a some type of causal constraints like the before and between so the before constraint tell us that before this action actually not to the second estas something must be true in the state so when we decompose the task before the first action of the decomposition this predicate this atom must be true which is not that complicated is like a precondition but the between constraints are more interesting so we can specify that between two tasks some condition must be preserved so this is especially important if we insert something key between so it should not destroy this precondition okay so we can say that between loading and moving this predicate must be true whatever we insert in between them in the in the in the final in the final plan okay so it's very natural to encode H States like one to one translation of HT ends to to attribute grammars okay so let's go to what we did in this paper so we are doing koala Dacian of plans and we use parsing so parsing is an approach to recognize whether you are given whether given sentence belongs to a given language described in this case by a grammar it's done in a bottom one bottom-up way so we basically collect terminals in this case actions together and we form tasks so we are so building the hierarchical structure from the bottom to the top okay so this is for the higher vehicle structure but what about the constraints what we do is that for each task we keep what we call the timeline which is a sequence of slots describing the states that are somehow generated when we are executing when we are executing this this task so these slots contain actions from the original plan and they also contain description of state right before the action and effects of the action which is useful to check to check the the cause the cause of constraints and the slots might be empty if we don't know we add the action because we will need to interleave something from another browser so we may use an empty and this is okay so the idea is that we have the rules we have the the set of set of actions corresponding to the initial plan and we are trying to group them so we try to find a rule such that the right hand side of the rule is matching to some subset of activities in this set of activities we have and if it's possible and all the tasks are satisfied we can group them together and we can generate a task corresponding to this okay and we need to obviously to merge the timelines and so on so I guess the best way how to describe the algorithm is to to show an example how is it working so let's start with this simple plan to load operations and the move operation and then to unload operations so the first step is to generate the initial primitive task where's the with the slot so we have five of them there is always the name of the task these indexes indicate to which activity is this task span so this is just one activity at position two so that's why it's two and two and you can see it's like it's four and four and this is a precondition of the task and this is the effect of the task so we can fill them to the slot because we know that the state before the action must contain this this terms and the state after mass or must not contain these phantoms and now we try to group them so let's use this rule that I presented before so it has three tasks load move and unload so we can find them here it's like this load this move and this unload obviously the names of of objects needs to march but there's also another trip like this one right so we can all use this load this move and this unload okay so we can combine everything which is possible so I just selected the first rigid I showed and we combine them together and we generate or we prepare a new task transfer one and you can see that now this task spans from one to four because this is the one three and four so we fill the slots for this task bought the slots to is empty we don't know what will be there yet okay so this is the first part but this is not enough we need to check the constraints that are behind the decomposition so we merge the timelines and now we have the constraints so we are checking them simply by looking at the timeline so if the constraint says okay before loading something must be true so we look before loading or it's true so it's verified this between is more interesting so between loading and moving something must be true okay so actually it's true at this state but we don't know if it's true here so what we do is we put it there right there for future to know that this must be true too in future and we do it with all the tasks obviously obviously the the precedence constraints can be verified as well by checking the indexes okay and the next step is actually using the information that the plan is flowing meaning from one state to another state we go by reactions so this is the classical condition that the next state is calculated from the previous state by removing negative effects and adding the positive effects so we can actually propagate this information so this is a description of this classical relation between the states and using this we can propagate information between between the states and we can go in both directions from left to right or right to life okay so for example in this case we know that the predicate add is deleted so what we do is that remember in the next state that it should not be served there just in case that in future we will try to do ad it okay so we can propagate the information through the timeline from both directions left to right and right to light the same is happening here so we deleted some saying so we should remember in the next stage that it should not be sir if we try to add it later okay so this is the the principle of the algorithm if you asking is there really some algorithm or is it just about pictures yes there is an algorithm you can find it in the in the paper I have like one minute so I will go one by line to explain it completely because it's not just this algorithms there are many auxillary procedures route like merging the slots and and so on well I guess it's more interesting to see if it's really bulking so yes it's working in theory but we already implemented it and we compared with the only validator for instance which is part of the Panda system this violator is based on translation the validation problem into SOT and it is being presented last year at AI caps okay so panda can do the validation as far as we believe it's not complete it does not cover this between constraints as far as I know and some of the constraints are compiled away so we are actually not validating the original plan but the plan after the after the compilation and we use just two planning domains to do the comparison and we measured run time both for valid and for in valid invalid plans okay so this is the result for the supplier domain it's it's a run time in in seconds it's logarithmic scale okay so this is the pondo system you can see not big difference between vert and embolic plans the same for our system there is slight difference for panda it's more complicated to to verify valid plans for our systems obviously it's more complicated to validate in voluntee plans because we are generating many many other tasks there and it looks like we are one order of magnitude faster even in the run times are now now much better and for the transport domain it is smaller as small as the same so two conclusions now we believe we have the first validator for complete HTN models with all the constraints or whatever constraints has been introduced in HTN which be able to model to model them sometimes it's more interesting to look what what to do next so obviously right now the answer is yes or no if it's valid it's perfect but if it's not valid we just say it's not valid so it might be more appropriate to return where is the problem and the problem can be in the plan but the problem can be also in in the model we can go to partial plans right now we require a complete plan in the input so it would be interesting if it's possible to extend this algorithm to partial plans like give me the prefix and I will use what tasks are you going to do maybe the partial plan maybe even not correct so there might be missing actions or there might be wrongly recognized sections or actions that are actually not belonging to this domain at all and based on this obviously if you know the task we can predict what's the next section which I believe it's very important for any security agencies and so on to predict what you are going to do but for us it's one more to think to look at the other direction and this is what should be modified in the model to comply with the plan so we know the plan we have a partial model but we would like to know what needs to be added to the model such that we can cover that plan so I believe this is more interesting kara for from the Norwich engineering perspective and that's it thank you for attention time for question the talk and for the example inspired by a boat trip yesterday can you hear me now yes yes okay thanks for the talk the correspondence between the attribute grammar is a complex result and your algorithm is a huge algorithm so do you think there's a role for mechanization and actually implementing this in tools like Isabelle and caulk and Hull so that we have some trust in plan validation tools and in the equivalences between the various formalisms actually what we proposed originally was using attribute grammars to model workflows and we hope that it could be some of unifying framework to modeling many techniques it's quite natural to describe classical strips like domains in attribute grammars now we can describe HT ends we can describe workflows so we can use it as a tool that can translate maybe between between different formalisms and obviously the idea is that if you if you design algorithm for this unified model and you want to verify whatever you want in your formalism is just enough to write the translation and you can do verification through this so that's somehow the whole to have a unifying framework such that we can translate between between different representations we can also the reason for grammars is we can learn something from the linguistics and these guys that are using grammars or use it and now they use deep learning a lot but when they use the grammars they develop a lot of technique so hopefully we can learn something from them for other errors like planning so that's that's my ambition for this [Applause] and they have a video too and the third paper in the session is a journal track paper thank you morning I'm Jose Carlos Gonzalez and I'm going to present you a joint work with Fernando Fernandez which is my thesis advisor and Jose Carlos Pulido this is part of the journal track or five cups and I have the opportunity to show you the now or now therapist project so at first the first thing the first thing that I want to show you is it's a video to see how or architecture works so here we go so this is what we have we have a humanoid robot and the aim of this robot is to perform rehabilitation sessions autonomously for children with prop with problems with their upper limbs so we have a 3d sensor in that Budhan supports a platform that we have and the robot says please do you have to put this position with your arms and then the kid tries to imitate him it and the children can doesn't have exactly the same position with the with his arms with their arms then we can change we can correct the position with our 3d sensor and the robot say say do you have to you have to correct your right arm or your left hand for example and a second attempt or any kind of attempt in fact and the robot can imitate the the Chile the child so it says a I've seen you that you have your arms in this position so you have to put them in this other position the motivation of this work is that this kind of children have problems like several policy and brachial plexus palsy and they have to perform a lot of different rehabilitation sessions they are very hard boring and they have to do in a very stall in a very small time window during their lives so it's very important to maintain them they're motivated and to reinforce a commitment with their treatment so this kind of of projects are very useful for therapists because of the motivation but we also are gathering all kind of information from from the 3d sensor so the therapist can see their evolution in in allanville all sessions so at when we started with this project we contacted with a hospital with a Spanish Hospital and they have this therapeutic procedure for this kind of of children so the physician first performs the diagnosis of this of these patients and after that primary evaluation then determine the objectives of the therapy for example if they have to train more bmail and be manual exercises or you remain well exercises for course brained things like that and then a therapist has to design the therapy and the therapy consists consists in several sessions with a lot of exercises and each exercises are suitable for different therapeutic objectives and then after refining each session they have to execute it so they just execute these sessions with the children and they make they can change also the the previously scheduled exercises depending on the evolution of the of the patient so we took this into account by creating a three level architecture so in the main the most important part of our architecture is that it's it's it's all based in planning in this high level training we have a therapy designer in which the physician just say just say I want this to get this to this gate to be trained with that in the manual exercises or things like that and then develop the therapy designer just the output of that module is the for the whole therapy with all of the sessions with the with its his exercises and then in a step B we have the medium level planning which is the actual execution of those sessions that we have the plan before so in that decision soup we have a decision support there we gather all the information from the environment from a 3d sensor and also from the robot and then we transform it to low-level actions that the robot can understand and that and of course we have there are low-level planning which is which is transparent for us because it's just a path planning for example to move one arm from two from one place to other place so I'm going to explain the high level first in the medium level and the low I'm going to say just a few insights about the low level because it's not very important for our work so for the therapy designer which is the high level of planning we have sessions and I've told you before which have exercises that have a maximum duration and a minimum duration and they have also faces so they can be a suitable for a warming up phase then a training phase or for a cooling down phase that must be expected in all of the sessions and then all the exercises also have poses and a pose we can say that it's a cycle of post set by the robot then the perception of the pose of the children and evaluation if the post is correct or not and then the correction if needed so also we have a variability and patient constraints the exercises can't be a RHIB can reappear in one session to Inquisition to avoid the foreignness of the session and also the exercise distribution should be asserted two sessions so that's why we want very variable settings among all the therapy and also we should avoid group of exercises that the children couldn't couldn't perform so this is the output of our high-level planning we have all of these sessions and on the exercises that some of them are suitable for our for the warm-up phase order for a training phase and for the cooldown phase and how we did how I did that so how did it how can we guarantee the therapeutic objectives that the physician inserts into the system well for that we defined the therapeutic objectives communities level that numbers there which is B main well finally manual course in the manual things like that are indicated by the physician so he say he says for example I want 15 points of the manual 13 points of five new numerator and 5 points of course mmm oh okay then each exercise has an adequacy level for okay for each one of those therapeutic effectives so we can have an exercise which has for example 1 point 4 remain all 5 points for fine uni manor and things like that it's exercise also has a duration and a value of intensity and difficulty which is determined by the but each therapist and of course the group of the exercise to to avoid them if needed if they if I can perform them so the goal of the problem is to reach the respective TOC L by assuming all the adequacy levels for a for all the exercise of each session so if we have for example 3 exercises in a session with five points of liminal then the mineral part of the session will be accomplished so we can guarantee that the kid is going to train accordingly to to the guides the guides that efficacy and gave to the system so yeah we have a planner and the planner has to choose with a with an exercise database that we have previously per set and then the output as I told you before is the plan sessions with also respecting all of the warm-up face training face and things like that of course reaching the day well accomplishing the TCL's reachability property so for that we created an HT n we try to do it with P real but we found that we we made a heavy use of numerical fluent so so the n HT n works much better in in in our case so first we generate the therapy then each a.cian then the a chair then the exercise for warm-up training and Coulomb figure and cool face and of course each its exercises the pay that the exercise that our plan for for a centering face are suitable for them so we can achieve that curve that it's shown there with a low and and soft difficulty at first and the end of the of the therapy so this did what that was for the high the high level planning and I want to show to explain it the medium level planning in which is each plant session that we that we had that we managed to plan with with the previous level are going to be executed so we are using classical planning here with replenish and each of these sections have some example of actions that we have in our pddl domain so we have a welcome stage in way in which the robot says hello and presents itself and then start training and in the training stage at first it introduces the exercise that is good that they are going to do and they are started with a cycle of place set the pose and then the correct deposit needed and so on and then finish they finish the training so this is or Morial that will or the schedule of of the system that we have for the medium level of planning we have the executive that gathers all information from the environment from the 3d sensor that we have and also from the robot and extrapolate some exogenous predicates like correct pose detector patient and things like that so then it completes the owl state of the world and sense sends it to Polly Apple is a sub architecture that we have for planning and learning and and root planing that we develop in our group and it's a very fresh it's a very major framework to work with now so when the state of the world reaches our decision support then the monitoring module checks if it is comparable with the rest of our planned actions and if it's not compatible then then it sends again to decision support and we plan a new claim so we now we are currently using metric FF and these are in the bottom part with these are or medium level actions that we have told you before and then when the decisions report sense since the next action it's they compose into low level instructions for a robot like say player leo files it angles from vision things like things like that and as an insight for our love letting its independent from the robotic in a platform because they are generic local elections so they can be interpreted by similar robots of course they should have arms but also for virtual avatars or thing systems like that and we have tested with the no robot with the ulcers robot which is a Houston micro customized robot from from Spanish University and also the rim robot from pal robotics so the journal paper conclusions is that we had an autonomous an autonomous system that can drive therapeutic sections previously planned by itself and the control was addressed in 3 SF suction levels with planning the high level was a therapy plan a therapy this designer the middle level in which each session was controlled and the low level which was transparent for us but we did their pack planning tasks for for the robot and we had evaluated this with a large group of children a large group we did and 2015 we did our initial tests of 120 healthy children in schools and three real patients in a hospital so that was our first interaction test only one session and children of course like at the the system because because it was very interesting for them to see that the first time but we thought that we would shoot the front long-term test too to see if they can be engaged with the treatment so then in 2016 we performed this long tank which shall be with 12 patients in a hospital two times per week for four months and yeah the children were motivated but we felt that our sessions were very repetitive nevertheless therapies were very interested in the project so in 2017 where we change we created a lot of new activities to perform with the robot and we also found that the therapies were very interested in in this one and the children were liked it a lot because it was very different each session was not really different we had a storytelling activity so the robot tells a different chapter of its own story it's new session and of course different games so things have changed in in in those ears so we made some improvements over the paper that I am presenting here especially in the high level in which now we can replay for high level Evans also the high level Evan that I told you before was planning an offline phase and also in the medium level now we have a fairly declarative mechanism for our decision support so we can for the execution and monitoring of the actions and also for the refinement of the actions so we could let ultimately with with a graphical user interface something like that the therapies to create new activities for for the robot and it will work and we also make interruption of actions in the middle of the execution and as I told you before new games and activities and now we are independent also from the 3d sensor not only from the robotic platform so as a future work we think we want to develop a few generic multi level control architecture because we find that each that not not only with three levels or doesn't win an arbitrary level of offer of hurricane and also compare it with with our control systems we are now currently working in a system like that so these are all references or some of the words that we have full list and that's all for my part thank you [Applause] I was just wondering how you because he said that in LA the third phase of his study you ensured that the robot always just different exercises every day to have a little bit of how do you ensure that so how do you ensure that the exercises you do on the second day are different from from how can we ensure that the children where I'm more committed with their treatment yeah it's difficult it's with with questionnaires and also asking me with their with their therapist in the third evaluation that what there was the bigger one each of the children has a has its own therapist and they were completely different they they don't know each other and after all of the of the tests we we found that they they'd like it to go to to the it wasn't a hospital it wasn't a university it was a revelation summer camp but we found that they were they were very interesting the problem here is that we can we can assure that our system is apart from motivation is good for the children because this kind of for children has to do a lot of different difficult exercises like sport for example so we can't we can't isolate our system and said yet definitely this works for this kind of shelter so the only things that we are measuring here are the motivation and of course the the usefulness for the for the therapists with with with out with our system I created a graphical user interface for them and so on since I injured my back recently could I schedule an appointment sorry I said since I injured my back recently may I schedule an appointment yeah you cou we are also thinking on use this system not only for for physical rehabilitation but also for other kinds of rehabilitation also for other types of patients but we have focused on children because of course the the way they interact with the robot is much more natural than old people but but we are trying to focus also on on out our and autistic children because some of them are more it's easier for them to interact with robots instead of people maybe because of their condition I'm not an expert with that therapies say that stangs speaking again that ends the session