Signature Travel Information Made Easy

Signature travel information

all right great excellent so welcome back everybody this is our next stop now and it is my distinct pleasure to introduce so dr zarlul is a senior investigator in the functional and restorative neurosurgery unit at the national institute of neurological disorders and stroke dr zanu received his bachelor degree from mit his md and phd degrees from the university of pennsylvania and his graduate work focused on developing silicon models of visual processing in the mammalian retina he also completed the residency in neurological surgery and post-doctoral research investigating the neural correlates of human memory encoding decision and reward his lab is focused on investigating the neural mechanisms underlying human cognitive function and i'm very happy that we were able to have dr zalu with us today and so the title of his talk here is neural signatures of memory and information in the human brain so without any further ado i will just uh please join me in welcoming doctors online great great uh thank you for the very nice introduction yeah but there's a bit of an echo do you have two crowd cast windows open maybe you need to close one of them yeah yes okay and are you still seeing my shirt screen yeah okay great uh very nice introduction kid even sorry for the technical difficulty uh thank you for inviting me to speak and thank you for giving us the opportunity to present our work um i'm actually very excited about this meeting and i'm looking forward to the rest of the talks and to the panel discussion so uh our work in our lab is uh largely focused on understanding the neural mechanisms that underlie human episodic memory formation um obviously we study memory because we think memory is both quite interesting and quite important we rely upon our memories uh for our everyday experiences not only to form our own personal identities but we also rely upon our memories to guide and to plan our future actions uh but one of our other motivations for studying memory is that we believe by studying memory we can actually gain insights into how the brain actually represents processes and encodes information and so what i thought i would do today is share with you some of our work related to memory and understanding these signals in the brain underlying memory uh and argue that we can use this to begin to begin to gain sort of some knowledge about how information is represented in the brain so of course there are a lot of different types of memory that one could study the memory that we focus on in our lab is a specific type of memory known as episodic memory so this is the memory of episodes or events that you'd experience at a particular time or place in the past so this would be the memory for example of your first date or your last vacation and it's argued that the study of episodic memory is really best accomplished in humans humans are unique in their ability to use language and so they can use language to explicitly describe a recollection which is one of the key requirements in the study of episodic memory now the study of episodic memory of course is an active area of research and it seems to involve a complex interaction among multiple structures in the brain so when you first experience an episode or an event you process information about the episode itself as well as the surrounding time and place during which that event occurred in these cortical sensory areas and these cortical association areas this information about the particular contents of an episode as well as its surrounding context is conveyed down to medial temporal lobe structures such as the neuronal cortex and toronto cortex and the peripheral gyrus and then ultimately to the hippocampus and one of the main purposes of the hippocampus as james just talked about was actually establishing these relational associations between these different elements to comprise every event or every memory that we're trying to encode the hippocampus internally communicates back with these medial temporal lobe structures with these cortical association areas and also with the prefrontal cortex and all of this allows us to [Music] encode new events into memory and to organize these memories now there's some evidence that suggests that when we actually retrieve a memory we actually replay some of these patterns of neural activity and that's what i'm going to talk about uh today so the framework that we use to guide our work is pretty simple it's captured by this very simple schematic uh so what i'm showing you here is this very simplified representation of the state of your brain and you can think of these brain states as exhibiting these slowly changing fluctuations and these fluctuations are affected by who you're seeing or what you're uh where you're walking for instance or even what you're thinking about they form both a spatial context and a temporal context but they also form an internal context now suppose that you and i engage uh in a great conversation in a restaurant you want to remember this conversation well if you're going to remember that conversation you're actually going to embed the memory of that conversation within this context now suppose at a later point in time you're walking down the street and you happen to walk into the same restaurant and when you walk into this restaurant it triggers you to remember this conversation that we had well there's substantial behavioral and psychological evidence that suggests that when this occurs you actually mentally and internally jump back in time and you replay that conversation in your head but you also replay the surrounding context you might think about what you were wearing at the time where you're going to next and what you were thinking about and so this idea of mentally going back and replaying an episode it was coined as mental time travel by tolfing in the 1970s and it presents us with an immediate accessible hypothesis which is that if there is a pattern of activity that is present in your brain when you first experience an episode then when you go back in time and you retrieve that episode you should actually replay a very similar pattern of activity and so one of the first aims in our lab or one of the first areas that we focus on our lab is actually trying to understand the mechanisms that underlie this process of replaying these memories now what we also know is that this brain state has a significant impact on your ability to actually form and retrieve these memories for instance you might be more sleepy you might be more awake you might be paying attention you might be paying less attention and so another area of focus in our lab is actually understanding how these different brain states actually affect our ability to form and retrieve memories and then finally as i said one of the main purposes of having a memory is to actually use these memories to guide our future actions and so one of the other areas that we focus on our labs trying to understand how this actually occurs in the human brain so today i'm largely focused on this first area this idea of understanding these mechanisms that underlie human replay and the experimental platform that we use to actually test these questions is actually taking advantage of the opportunities that we have uh through the clinical care that we provide in patients with drug-resistant epilepsy so i'm not gonna talk about epilepsy very much but as some of you probably know patients who have drug resistant epilepsy continue to have seizures despite best medical therapy and in those cases we can actually offer them a surgical resection to actually remove that part of the brain that's causing seizures but to do so it critically relies upon our ability to actually localize exactly where those electrodes are in the brain and if we can't do this non-invasively we actually have to place these electro contacts directly in and on the brain to be able to do that so i'm going to show you a picture of a brain of the next slide so just brace yourself uh what i'm showing you here is a typical uh implant that we have and so what we're looking at here is a temporal lobe in one of our patients and you can see that we place these electrodes directly on the surface of the brain there are also electrodes that lie underneath the brain there are electrodes that go uh to the frontal lobes and so on and so forth and we use a variety of different electro combinations depending on the actual clinical focus that we're trying to localize well it turns out that this gives us a very uh good opportunity from research perspective because while these patients are being monitored they are awake and they're interactive and so we can ask them to participate in different types of tasks or games and we record neural activity and ask how this activity correlates with the behaviors that we observe and so from these electrodes we can get a variety of different signals so for example from any one of these contacts we can capture the net aggregate activity over a small patch of cortex just generally reflects the net dendritic inputs over a hundred or hundreds of thousands of neurons and what we get from this signal is something that we call an intracranial eg signal and what you'll notice about these intracranial eg signals is that they are characterized by the presence of these fluctuations or these oscillations these oscillations come in a variety of different frequencies there has been a substantial literature that's linked different frequencies to different types of behaviors and so this is one of the main signals that we observe what we can also do is we can actually ask how signals might be correlated or coordinated across different brain regions and now this begins to give us a sense of what's happening at a larger network scale how information is moving across from one brain region to another and then finally with uh more recent technologies and more recent electrodes we can now actually capture the activity of individual neurons we can actually capture spiking activity and ask how individual neurons how population spiking activity might be related to this process of forming and retrieving memories so like i said what i'm going to focus on today is what are these mechanisms that we've been able to uncover that mediate our ability to actually go back and replay these memories i want to show you first is some previous work that we've shown where we show where we've seen that there are patterns of activity in the brain both at these larger spatial scales as well as a smaller spatial scales that are reinstated when you retrieve these memories and then we're going to ask how these uh patterns are actually reinstated the mechanisms that underlie that and i'm going to show you some data that tells us that there are actually sequences of spiking activity in individual neurons that are actually being replayed and then finally i'm going to make the argument that what this is telling us then is that these packets of information are really captured by these sequences of spiking activity so the experimental paradigm that we use for the set of tasks is what's known as a parrot associates verbal episodic memory task so in this task what we do is we present our subjects with these pairs of words these are just regular nouns uh we then ask them to make these novel associations between them uh they then form they then perform some simple math problems as a way of distraction and then later on during testing we cue them with one word and their job is to remember what the associated word is and we do this we use this paradigm because it offers us two major advantages so first of all most of the electrodes that we place are actually in and around the temporal lobes and these are regions of the brain that we know are heavily involved in semantic processing in order for you to actually complete this task it requires you to draw upon the meanings of these words to make these associations and so what that suggests then is that the patterns of activity that we see in the neural data might be related to semantically processing the meanings of these words and therefore might be relevant to the experimental questions that we're trying to ask but the second more important advantage is that this task gives us direct control over what a word pairs we're trying to remember or trying to encode its memory and importantly what we're they're trying to remember and when they're trying to remember it and so what that means is that we can look at what the activity looks like when people are forming these memories and we can ask whether similar patterns of activity are actually being replayed when people retrieve these memories so in earlier work this is exactly what we did so in the first one in one of our first studies what we did is we looked at these patterns of oscillatory activity across these clinical surface electrodes uh and so from each of these electrodes we can extract out spectral power within different frequencies we can aggregate all this together across all of our spatial locations and we can generate a feature vector and we can ask how similar these feature vectors are between when you encode a memory and when you retrieve it and what we found across all our subjects is that we can create what we call these similarity maps and so just to walk you through this what we're looking at on the x-axis is every single time point during retrieval until the time that they vocalize their response on the y-axis is every single time point during encoding beginning with when the word pairs come on the screen and the color of each pixel tells us the extent to which that feature vector is similar between that encoding time and that retrieval time so if we see a big red blob what that tells us then is that there is a pattern of activity that was present immediately before people vocalized a response that was quite similar to the patterns that were present when they were encoding these words into memory now if we look what happens during the incorrect trials we find that this similarity largely uh disappears and there's a significant difference between the two now it's going to work we actually extend this down to the level of individual neurons and so to do this we actually use these special microelectrode arrays these are colloquially known as utah electrodes they are arrays of approximately 96 electrodes and they extend about one millimeter into the cortex and what we do is we place these arrays directly into our uh the anterior part of the temporal lobe we place it in this region because this is largely an area that we anticipate we're going to resect each of these electrodes can capture the activity of individual neurons or action potentials and now we can ask what this population of neurons is doing as people participate in this task so just to give you an example here is a population of neurons and their responses in during the encoding period you can see that when the word pairs come on the screen at time zero there are neurons that show increases in the responses and red and other neurons that show decreases in their responses in blue now if you look at what these same neurons are during during doing during retrieval what you find is that they exhibit a very similar pattern of activity that the neurons that were increasing their activity during encoding are also increasing their activity during retrieval those that were decreasing their activity during encoding are also decreasing their activity during retrieval and so once again we can construct these feature vectors now we can populate this feature vector with the population spiking activity we could generate these similarity maps and again across all our participants what we see is that there's significantly greater similarity during correct retrieval compared to incorrect regal so together what these data tell us then is that at both the local scale as well as at the neuronal scale that when you retrieve these memories you actually reinstate these patterns of activity that are particular to the individual item that you're trying to retrieve from memory and so the question that's motivated us recently then is how does this actually work what is the mechanism that underlies this process and so to do this we actually drew a lot of inspiration and motivation from the literature on spatial navigation and rodents of course everyone here is mostly familiar with this literature but this is a quick summary of that work uh and what i'm showing you here is a phenomenon uh that has been uh described in literature for about 30 40 years this is the presence of a specific type of cell in the medial temporal lobe known as a place cell as was mentioned earlier in the earlier attacks earlier talks these are cells that respond to specific locations such that if a rat is moving along a track what you'll see is a sequence of firing along a sequence of firing of these cells as the rats moving along the track now remarkably when the rodent is asleep and perhaps consolidating their memory or when they're awake and maybe thinking about different runs on the track what we find is that there are different that these the same sequence of uh spiking activities actually replay suggesting that there's some element of memory that's actually involved in this process of replaying this sequence of spiking activity now what's been left clear is what happens in other regions of the brain like in the cortex and certainly what we have not known is how such spike replay might be relevant for awake human memory retrieval now what we've also seen in these rodent studies is that when we see these replay events they're often accompanied by another physiological signal known as a sharp wave ripple so these are fast oscillations that are observed in the local field potential signal they're called sharp waves because they oftentimes sit on the deflection of these sharp transients in the filter trace you can see that these are these fast oscillations and these ripples have really come to represent or be seen as a biomarker for these replay events now again these replay these ripples have largely been studied in the medial temporal lobe oftentimes during sleep less frequently we've seen and we've observed data showing how these ripples might be present in the cortex and certainly there's been less data examining how these ripples might be relevant for awake human memory retrieval so we decided to tackle these two questions so the first study is a study that's carried out by alex vaz who's an ascended graduate student in our lab and what alex wanted to first ask is whether we can see such evidence for these ripples in our human recordings so i'm showing you here on the left is a schematic from one of our implants you can see that we have electrodes lying underneath the brain right along the medial temporal lobe and we have other electrodes sitting on the lateral temporal cortex and if we look at one of these medial temporal lobe electrodes what we can see in the raw trace above and in the filter trace below is the presence of these ripples now if we look at the same time one of these recordings from the uh lateral temporal cortex in this case the middle temporal gyrus we can also see on occasion that we see these ripple oscillations and that time these ripples are actually happening at the same time between these two structures so we call these coupled and at times they're happening in one region and not another so we call these uncoupled and so we can see these ripples all the time so now the question is how whether these ripples are actually relevant for this process of memory retrieval so what i'll show you here is the data from one participant across all their trials we're looking at what happens during retrieval and what we're doing is we're placing a tick mark every time we see one of these ripples that's what we can see in the medial temporal lobe is that during the correct trials there's a clear increase in the number of ripples that we see immediately before people vocalize their response this is much less during incorrect trials but importantly what we also see is that there's a significantly greater number of coupled ripples during the correct trials compared to the incorrect trials and of course if we look at other parts of the brain like the primary motor cortex or the primary sensory cortex we don't see these changes and we'd expect that since this task doesn't really involve those regions of the brain so this is providing us some evidence then that these ripples in particular these couple variables might be relevant for this process of memory retrieval now what we also did is we wanted to tie these data to the data that we had seen earlier regarding reinstatement and so what we did is we locked our retrieval data to every time we saw one of these couple of ripples and what we find is that if we lock the data in this manner we can see that these ripples particularly these coupled ripples actually precede this increase in reinstatement that we observe across the cortex whereas we pick random times during the retrieval period we see that that reinstatement and that locking largely disappears so together these data really show us then that these ripples and this interaction that we know exists between the medial temporal lobe and the lateral tropical cortex might be playing an important role in this process of retrieving memories and reinstating these patterns of activity now what we want to do next then is actually ask what's happening during these ripples and in particular what's happening with the spiking activity underlying it and again we can do this because we have this ability to record from these populations of spiking activity uh through these microelectroarrays and so this is a second study that alex carried it out what we wanted to know is what neurons are doing in the presence of these ripples so i'm showing you up top are these ripples that we've recorded on these electrodes in the middle temporal gyrus so these are electrodes that are sitting on the cortex and then we can also see that there is a ripple that happens slightly earlier in the medial temporal lobe and if we look at what this population of neurons is doing during this ripple what we find is that there is a clear burst of spiking activity and in fact if we look at the individual microelectro channels we can see that there are small little micro ripples on each of these lfp traces and the extent to which we see these ripples on the local field potential signal correlates with how much spiking activity that we see and we see this all the time every time we see these ripples we see these bursts of spiking activity and so in the process of characterizing this work uh so this is work carried out now by another graduate student along in the lab uh ifong tong and what she's doing is she's simply asking what do what does the relationship look like between these ripples that we observe and the underlying spiking activity and you can see over this segment of time that whenever we see these ripples both at the macro scale as well as the micro scale the local field potential signal we see these bursts of spiking activity as i mentioned this correlation is quite strong the extent to which we see ripple activity correlates quite strongly with how much spiking activity both in the local field potential signal as well as in the intracranial eg signal okay so now what's happening during our paired associates task so what i'm showing you here is an example from one individual trial in this trial we are presenting the word pear cake fox you can see that there is a burst of spiking activity and what we can do is we can actually ask which neurons fire earlier in that burst and which neurons fire later we're going to color those neurons yellow and blue respectively well it turns out that when we present these word pairs we actually see multiple bursts fighting and what's remarkable is that the spiraling the temporal order in which these neurons fire is relatively preserved throughout each of these individual bursts the yellow neurons always fire first and the blue neurons tend to fire later what's also remarkable is that that particular sequence of firing is specific to that individual trial so what i'm showing you here is another trial so in this trial the subject was studying the word pair steam seal we can see that the yellow neurons fire first and the blue neurons fire later and now we're going to ask what these same neurons are doing when we look at a different trial so this is a different trial we've preserved the coloring and what you can see is that now some of the yellow neurons are firing later and some of the blue neurons are firing earlier the sequence has changed and so we wanted to know whether we can actually sort of quantify this and we can use any metric to do this but the metric that we decided to use was a metric known as the matching index so the magic index simply tells us how similar two sequences are to one another and it's a bounded metric so if the sequences are identical the matching index takes on a value of one if the sequences are in the exact opposite order it takes on a value of negative one and so what we're going to do is we're going to ask whether uh how similar sequences are within every trial compared to how sequences are when we compare across different trials and so if we look at the trials that patients uh that people eventually encoded correctly what we find is that there is significantly greater similarity within the trial compared to across different trials and we find that this relationship is actually largely absent in the trials that they got incorrect and so what these data tell us then is that this process of successfully encoding these word pairs into memory involves exhibiting these precise temporal sequences of spiking activity that are preserved and consistent within each individual trial okay so now the important question of course is what happens during memory retrieval so what i'm showing you here is another example again in this example we are showing you the word pair jeep crow we have this burst of activity in which the yellow neurons fire first and the blue neurons fire later now we're going to ask what these same neurons preserving the same color scheme are doing during retrieval and so during retrieval we present them with the word paired with the word jeep and they say the word crow and what we see is that we evoke a burst of activity that has a very similar temporal sequence the yellow neurons are firing earlier and the blue neurons are firing later and again we can quantify the extent to which this happens using this metric the matching index and so what we find across all of our trials and across all of our participants is that as we get closer and closer to vocalization at time zero we have these bursts of spiking activity that look very similar to the spiking activity and the sequence of spiking activity that were present during encoding and this bar graph on the right here just sort of captures that difference during that last 500 milliseconds now we also find is that when this is happening we also see overall decreases in spiking rates and so what these data suggests then is that this process of encoding items into memory and retrieving items from memory involves sparse yet temporally precise sequences of spiking activity so of course we're quite excited about this work and we're moving this work in multiple different directions and so one of the areas that we're focusing on of course is how this might reflect representations of context of time and i'm not gonna get a chance to talk about that today uh but another thing that we're quite interested in is how this also might be relevant for how the brain represents individual items or content of memory if we can get a handle on how the brain represents individual items then we can potentially track how these items are actually encoded into memory and how these items are potentially removed from membrane and the sequences that we see have motivated us to think about how the brain represents its information in a slightly different way so i'm going to show you uh our two studies to get at that point of how the brain represents information using these bursa spiking activity and so in the first one this is a study that's carried out by john widdick who's a staff scientist in our lab and what we want to know simply is how does the brain represent individual items and so to do this we looked both at intracranial eg signals as well as local field potential signals as well as spiking activity as individuals simply looked at pictures or read words that corresponded to different items so they could be places they could be people they could be tools they could be animals and their job in this task was simply to tell us what it is which which category they're looking at whether it's a person place animal and so on and so forth it's very similar to the localizer tasks that you see in fmri studies now the additional way that you can do this and the way that we have done this is that you can look at a population of neurons and look at the population spiking activity and construct these feature vectors and you can assign each of these feature vectors to each of these stimuli and then you can build a classifier that can allow you to decode which feature vectors or what patterns are associated with with which different categories and we've done this and this works quite well but again looking at this work earlier regarding our sequences we've been motivated to look at this in a different way and what we find is actually something that's actually quite interesting for us and potentially gives us some insight into how the brain represents information so i'm showing you here is a typical example so if i show you for example a picture of michael jordan what we find is after we present the picture we evoke a burst of spiking activity that has a clear temporal sequence the yellow neurons fire first and the blue neurons fire later and again when we show these images we actually generally evoke multiple bursts of activity and what we find is that the sequence of activity tends to be pretty consistent that the yellow neurons tend to fire earlier and the blue neurons amplifier later and so once again we can use this matching index and we can ask how similar this sequence of activity is within trials compared to when we compare these birds across different trials and what we find in our preliminary data and we've actually sent this now to six subjects is that there is significantly greater similarity when we look within trials compared to when we look across trials and this is true when people are paying attention when they correctly categorize the item that they're looking at it's largely not true when they actually fail to do so so this is actually quite exciting for us because it tells us potentially that this might be very important for how the brain might be representing information from a computational perspective this makes sense and aligns with some theoretical ideas regarding computational efficiency because you have a much richer combinatorial palette if that if you can encode individual items using these sequences of activity so finally um the spinal so i'm going to tell you then is related to the third uh sort of area of focus in our lab which is how memories guide behavior and we have a couple of projects where we are sort of working in this that we are pursuing in this area uh and so we have uh studies where we examine how prior knowledge and how prior memories might guide our ability to actually form memories and might also guide our ability to search for memories but the story i'm going to tell you about is how memories actually set our expectations so imagine for example you're going back into this restaurant where we have this conversation and you might notice that things have changed you might notice for example in this case that the restaurant is now black and white and the reason that you've noticed this is that you had this previous expectation for what the restaurant should look like and now what you're actually seeing is actually different than that expectation and so the story that i'm going to tell you about actually builds on this emerging hypothesis in the computational literature that the brain is really optimized to maximize information and the way it does this is it constantly generates an internal model for the way the world should look like and then it compares how the world actually looks like to this internal model and if there are any discrepancies then those discrepancies actually carry information we call those discrepancies prediction errors and those prediction errors are quite informative for the brain so just give you a single example imagine that you're walking along in this very nice field you see a lot of grass you see a lot of bushes you see a lot of trees you don't really think much about the grass and the bushes and the trees other than they might be beautiful but you don't really think much about them because these are the things that you'd expect to see in a field now suppose out of nowhere up pop up pops a television well for you this should be actually quite surprising and quite unexpected and that unexpected event actually violates your expectations for this field it is quite surprising it is a prediction error and is therefore quite informative this is of course the hypothesis of predictive coding and this hypothesis of course has gained a lot of empiric support in the study of perception in particular vision so we know that you generate these internal models for how the visual world should look like and you learn these internal models over a lifetime of observing experiencing the natural statistics of the world but what we're going to argue is that the brain is constantly making these internal models and in fact uses memories to generate these internal models as well and so whether we tested this is in a task that we in a study that we just completed so this is carried out by rafi haku's another outstanding graduate student in the lab and what raffi did is presented our subjects with these images these are images of these complex scenes they have these associations between different items in the scene and we present them these images once and then later on during testing we show them the same image and we ask them whether the image is identical to the image that they've previously seen and so in many cases the images are identical but in some cases we've actually manipulated the images so in some cases we've added items to the image so in this case for example we've added this bird in other cases we've actually removed items from the image in this case we've removed this other bird and the subject's job is to figure out whether the image that they're looking at is the same as their previous one that they've observed or whether it's been manipulated and if it's been manipulated we then ask them to point to where that manipulation took place and we do this also at the same time by tracking their eye movement so we can see what exactly is that they're looking at and so actually to do this task it requires you to actually draw upon your memory and to make a comparison between your memory of the scene this internal model and the scene that you're currently looking at and so what we want to know is that if these uh two are in conflict if there's a conflict between your internal model or your memory and your current visual experience do we see evidence for a prediction error signal and where does that actually occur so let's look at what happens so we focus largely on high frequency activity in these electrode contacts and if we look at these different brain regions what we see is an increase in high frequency activity that starts in the primary and secondary visual cortices and then extends anteriorly to the visual association cortices and then ultimately to the medial temporal lobe and this is exactly what we expect this is the ventral strain pathway of visual processing you have this progression activity from posterior to anterior parts of the brain so this is what happens when we see an image that's identical to the image that we've seen before so now what happens if we look at an image that's been manipulated so if we look first in the primary areas of the visual cortex we find that the patterns look quite similar however once we start looking in the association cortices either in the temporal lobe or on the parietal lobe what we see is that there's a clear difference in the activity there is this elevated and prolonged activity in this high frequency band of activity and then this activity this change actually then emerges in the medial temporal lobe and so for us this actually looks like a signal that represents this difference this prediction error now if we look at the trials where people actually had an image that was manipulated but they failed to identify that manipulation we see the activity looks very similar to the activity that was present when they're looking at an exactly identical uh image now let me also say that this is true whether we add an image to the item uh whether we add an item to the image or whether we remove an item from the image so it's not a property of the actual visual it's not it's not an artifact of the visual properties itself so this is actually quite remarkable so you have seen an image in one just one time you have a single exposure and this single exposure is sufficient for you to establish an internal model throughout your visual hierarchy about what this image should look like and now you use that internal model to make a comparison to this new image that you're seeing and what's also remarkable to us is that this change this difference that we observe arises first in these association coordinates before it arises in the mtl we have a lot of data in our manuscripts here that sort of demonstrates these temporal differences now what i'm also going to show you is that we can observe this difference when we look at this power but we can also see this if we look at these discrete packets of information so once again we can filter our raw traces in this band this ripple band that we observed earlier we can identify these discrete time points where we see these bursts of activity these packets of information and so if you look at the activity of for example one of these electrodes during this task what we find is that when we present these images that have been manipulated and the subject was able to correctly identify we see a clear increase in the number of these packets of information these little bursts of activity and this is much greater than the number of packets that we see during the repeated trials or during the trials that were manipulated and people fail to identify correctly so putting them all together what does this all mean so what we think is going on is that when you first experience an episode or an event as we said you process information along this visual stream pathway there are these interactions that we know exists between your visual association cortex and your medial temporal lobe and it establishes a pattern of activity that represents that particular element or that item or that memory now suppose at a later time you are presented with a similar but not identical image uh you will first process this information again along the visual stream and then through these interactions with the visual cortex and the mtl you're actually going to pattern complete you're going to reinstate those patterns of activity to retrieve that memory but in these cases what you'll find is that the patterns that you reinstated are actually slightly different than the patterns that you're observing and so this generates what we call a prediction error and this prediction area first emerges in these association cortices and then is conveyed down to the ideal temporal lobe perhaps to update your memory for future experiences okay so just to conclude as i said we use this general framework to study episodic memory i showed you a number of different studies examining how we can use our approach to actually understand the mechanisms that underlie uh this ability to retrieve memories and to reinstate patterns of activity and to replay specific sequences of spiking activity in the human brain i didn't talk very much about how brain states affect memory formation i'm happy to do so if anybody has any interesting any questions that they're interested in exploring uh but finally i want to show you at the end how we can use this information to now understand how we use our memories to guide our future behaviors and primarily i showed you our work where we describe how we use our memories to generate an internal model for the world and we use that model to make decisions about what we've seen what we haven't seen previously and so finally i want to thank of course all of the members of my lab they're they're all outstanding these are the current members on the left and the former members in the middle and all are very close collaborators on the epilepsy service and in the surgical neurology branch at nih uh and of course uh we are always open to postdocs and we're always recruiting people so if anyone's interested please feel free to reach out thank you [Music] okay thank you so much uh korean for this fantastic talk um there are many questions in the question tab um it seems that maybe not very it's a conflict of interest but the upvoted one is mine i'm just going to go ahead and read it quickly so similarities in the activity patterns between encoding and retrieval um are obviously visible with the spike rate activities that you've shown us but also they're visible in the population level oscillatory activity right in the lfb signals and so just curious um whether uh what relationships or similarities you found between the spike rates are the more similar to gamma range uh population activity or more similar to theta or other frequencies in the different experiments you've seen how do these two levels of observation relate to one another as it relates to these memory tasks yeah that's a that's a very good question so that's actually the work that we're trying to finish right now where we are trying to characterize the relationship between the versus viking activity and the overlying changes that we see in the signal i'm trying to bring up that slide right now so i can show you but basically the short story is that every time we see these birds of spiking activity they're accompanied by these um these ripples these uh sort of fast oscillations in the 80 to 120 hertz band you can see an example of that right here so you can see here on the bottom left we have these clear bursts these increases in spiking activity every time we see that both in the local fuel potential channel so on each one of these microelectrodes in purple as well as the overlying intracranial eg signal these larger clinical contacts we see these fast oscillations so we think there's a clear relationship between this relatively fast frequency band uh in the eeg signal and these underlying bursts of spiking activity and in fact the contention that we are probably going to make is that this these changes that we see in these high frequencies are actually punctate sort of discrete points of of change they are discrete time points that underlie or that reflect this underlying burst of spiking activity okay yeah that i think that um also agrees with some other data out published in literature showing these uh close correlations with the higher frequency rather than with the lower frequencies right so the the broadband but also as you see here the ripple activity more specifically so that's interesting we don't have much time there's a final discussion afterwards i'm just going to take one last question um by this is a question by shahab uh uh and so his question is how can one distinguish between episodic memory and short-term memory in the paired association task used in the study uh the follow-up is is it even necessary to dissociate different types of memories that might be involved in this task yeah so that's short-term versus epidemic yeah so those are two very good questions so number one to answer the second question i would actually make that same argument which to say that uh while we are framing most of our work in the context of episodic memory and longer term memory it probably is relevant for shorter term memory and it's probably involving similar processes in that we will likely see replay of activity uh when you are retrieving a particular memory the reason we think that this is not necessarily in the domain of working memory uh is for two reasons number one uh when people perform this task they have this encoding period where they study these pairs of words and then that's followed by a period of time where they actually solve these little math problems and that lasts for about 30 seconds and so no longer are these words no longer should these words be the focus of their attention they should be focusing on the math problems and so that serves as a sort of a form of distraction that then allows us to conclude that this is no longer in working memory the other reason that we don't necessarily think that this is mis relate to working memory is because there are limits uh to the capacity of items that one can retain in working memory and in our task we actually present uh these six word pairs so that comprises 12 words and so that should exceed the limits or the no or sort of what we believe are the limits of working memory capacity [Music] okay thank you so much for that answer and uh obviously there'll be more time for discussion in the panel so uh with this would like to thank you once again for this outstanding talk um and thank everybody else here so what we're going to do now is we're going to have a shorter break than planned just it's going to be a seven minute break uh biological break rest break tea break whatever you feel like and we'll be back at 10 45 um hopefully that's okay with uh dr nicole russ it's going to be a lecture up so please join us at 10 45 that's in about seven minutes for the keynote lecture when we call rest and sorry oh yes and just just one quick mention gather town space is open so you can walk around with your avatar go dance a little bit if you want to just have a small break on gather town before coming back um and there's a button at the bottom of the screen if you see there's a green button says gather town so feel free to go click on that it will take you right to the space and then you can go and explore but don't get too excited don't enjoy it too much we do want you to come back here at 10 45 for the keynote lecture uh the space will stay open throughout the day and we'll have more activities later okay thanks once again thanks kareem again for this outstanding talk um and uh we'll continue this in a few minutes