Save Heterogenous Ordered with airSlate SignNow

Save heterogenous ordered

okay alright so we previously had data we're actually I'm going to run the first a little bit because it's just reading in the data as it was last time and all so the data situation is going to be the same as last time you have a bunch of observations within each condition for a bunch of people so there's a1 and a2 I have this fake sex column and we've got a model it's going to be nearly identical line for line as the original hierarchical mixed Gaussian mall data is coming in the exact same you give it the subject number number give it a number of all the things the all the data the within-subjects contrast matrix in the between-subject contrast matrix and labels for each person transform all this is the same as before the new thing is that instead of having a single observation a single estimate of noise we're going to have an estimate of noise for each subject so we create a vector called noise that is n subjects items long when we want a prior on that noise vector we just can use the same syntax as we had when disabilities parameter was a single object a real single value so we just express it as weevil this sampling statement for how subject coefficients are distributed is the exact same as before so the only thing that changes is this last little bit here when we're saying for every observation why we're sampling why is from a normal distribution with a mean corresponding to the mean source for that subject given that set of within-subject predictors that that row with corresponds to but we're also now saying that with the magnitude of noise corresponding to that person's noise value so this X thing again is to subject the list of subjects from 1 through n such so this just grabs out for if this entry in the subject label vector is for describes out the fourth element from the noise factor so for fourth person noise so this is a little bit more realistic that you'd have variability from one person to another and the magnitude measurement noise that you're going to get particularly context as psychological research where the majority of measurement noise isn't the instruments were using but it's the thing that we're measuring this variable from moment to moment so and I've observed quite a bit that there's a strong variability and across people in the magnitude of noise that they manifest at least in my research where I have people playing games and they're making responses on how fast they can respond so yeah this is just one simple extension you can do I've actually shown you way back when we started talking about heterogeneity of variance breaking or models that can account for heterogeneity of variance you could even extend this model to say that the within-subject predictors actually or any of your predictors can influence the magnitude of Norway just like the predictors we're already modeling that the predictors are influencing this parameter of the normal distribution ie the mean you can similarly model that any of those predictors either between or the within or both are influencing the amount of noise that manifests that starts to get to become much larger model so i wasn't going to actually show it to you guys but if you had good theoretical justification for expecting a given predictor variable to influence how much noise is manifesting then you can actually model that in your mouth so for example if you expect that older individuals will be more noisy in how they respond on your test or whatever you could actually have like age and the predictor of the magnitude of noise we actually kind of covered how to do that in the release tab just between subjects data and I introduced that a little bit if you remember if you do the inference on the log scale of noise and then you exponentiate remember there's a little bit of confusion about that at the time assistant tweak your memory for something that you can look back at ya I said wasn't going to run this example because actually the data that I generated was was homogeneous for the magnitude of noise so this wouldn't reveal yield any different results but what I once also points out is that we are modeling the data as normal you already have an example of how to model the data as binomial if it were a binomial outcomes you actually be one less perimeter you taking off the noise we'd be representing the outcome as burner logit you also wouldn't have you'd have to change about things about how things were scaled but if you had other ideas for how the data were distributed so maybe you thought the data where log normal which is fairly common the log normal distribution still has two parameters it's actually just saying that the data are normally distributed on the log scale with some mean in some standard deviation so Stan lets you just say that you can convey that by the log normal distribution similarly if you had some other idea like maybe it's a weeble with some so that's another common thing to actually see if you wanted to reflect that we talked about counts models before so plus distribution so if you have within each person a number of observations for some sort of count so they're doing some sort of sharing task and you ask them to share settings or could we use a number of items that are certain enough yeah yeah exactly yeah then you can holiday discount if you're getting Likert items for sure I'm so the same person and their number of different questions but you kind of want to lump all the same questions within a given condition you can model that as ordered low just as we had before so any of the non-gaussian things that we've already covered you can do within a context of a hierarchical model other part of model to different yes so if you had for example non-hierarchical rossall we'd already covered that couple weeks ago for a non-hierarchical legit yeah so and one is hierarchical when you have more than one observation for a person in a condition what is it like but existing type of observation yeah and it's different than us multivariate ah yeah we're actually going to get into that momentarily I think if you for example had a scenario where there's multiple items on say a questionnaire you can just ignore that there are well we'll get ya we'll get into that and that is an example of a higher article and yeah well actually that's not one of the examples I have for today but it's one that I that I intended to cover well for this next time one thing that while we're still tough talking about Gaussian distributed variables one thing that can be useful is to count for the idea that there might be some outliers in your data at the level of like the measurements sometimes people excuse one of the reasons that you're measuring the same person many times within a given condition is presumably because you know that people are very error-prone in their responses sometimes people are so error-prone that like value is very extreme relative to the rest of their values and that gets into a whole realm of subjectivity with regards to you okay ahead of time trying to exclude values like that you typical rule as per minus three standard deviations call that an outlier but one way to make your model more robust to a amidst outliers without actually removing them and applying some sort of arbitrary rule for removing them it's actually model the data as heavy tail so I think I showed you guys the coffee distribution before where it's a student the student T distribution with degrees of freedom one basically the normal distribution with like the details not tapering off so quickly so the consequence of that is that to the degree that there are outliers those outliers won't affect the estimates for the standard deviation and mean as much as they would if you send it to expect normal data and if saw data coming in that have these big sur had a couple outliers in the tails so you can make your inference robust to outliers that the sort of scale of measurement noise by changing normal Takashi there that's a really common thing to do transform revival of days early oh yeah so that's a well I remember your scenario and you have yeah so on the x-axis there is like the number of shared items yeah yeah yeah so that's driven by the fact that you've got in ordered logit type scale where people can only say your your first shows 0 to 10 but I have another 10 without yeah yeah as you move from ordered logit to with many many sort of possible outcomes so a thousand possible you start getting into it's better to maybe characterize it as counts data in which case you can have more peaked at zero kind of skewed data like that and you might get what looks like they're like uh sort of well with you bonzers like a whole sugar another time yeah so you might you might have a mixture or like that yeah the if you think of like the smoothness you might have non smoothness just induced by the fact that you wanted your variables done to influence at it if you're showing what the overall data is as that distribution well but if you imagine this you've got even a galaxy an outcome mins there's an effect of phase between groups it's going to look by modal two degrees X there's a larger effect there so it's about like what the residuals look like from the model and so yeah there that's probably count I stated on to get one the actual all of you yeah first good nurser in one book should like that yeah what's up yeah we talked about it a couple weeks ago three weeks till now yeah yeah very fishy yeah ah what's OMG age all right um so I showed you before I'm just going to close the HMG aitken freedom we won't actually go through i showed you that you can with the mixed version of the code if you didn't have a between-subject predictor you still use the mixed version of the code it'd be better to go back and use the with high rock pull within code because that'll be slightly faster but if you didn't want to keep lots of Stan file time you're around you can still use the livid the mixed version of the code when you don't have a between-subjects protector visas for that between-subjects model you just say till the one what the same is not the case if you had between subjects but didn't have any within-subjects predictor and i'll show you why in a second love me hbg hierarchal between gaussian yeah what yeah like what kinds of distributions are up there and yeah that has a number of different distributions now that's going to do like maximum likelihood estimation which is not what we do but it might be a good place to look at different distributions or different shape yeah yeah ah I not today's I keep like off for a day that we don't have enough to cover I have a lecture on looking a little peering into the black box a little bit and that will explain the distinction between maximum likelihood and bayesian but yeah not what we're talk about today so what do you do when you have a hierarchical data set lots of observation within each person but no within-subject manipulation so it's not a one may seem certain person you just have a bunch of observations for a given person but lots of people and maybe some between subjects manipulations you can't use the mixed hierarchical mixed and just pass in an intercept column for the within because this bit down here when we try to estimate a correlation matrix so if we pass in just an intercept column as a matrix for the within-subjects predictor matrix it's got one column and you can't do a correlation matrix on one column span we'll just throw a better in which case you need to not use this HMG switch to hbg I mean this is such a rare kind of model that I hadn't even thought of it until this morning that I should maybe get me this code in case you encounter the scenario but it's fairly similar to how we have the data before except that of course we're not telling it anything about within subjects contrast matrix in the data coming in just between subjects contrast matrix still giving it a list of labels for each outcome what subject is that label correspond to and we have instead of a matrix where it was the number of between by the number of within it's just a vector of between public coefficients and since we only have one sort of within-subject coefficient an intercept each person has their own beam we only have one SD so this is no longer a vector and i called it instead of scales co FS things i call it scales in festus intercept intercept each subject will get coefficients because they're part of a group so they'll get their group coefficients for any of the group affect their first thing is going to be there intercept which is going to vary from person to person and here I've expressed that sort of model of noise where it doesn't vary from person or anything like that you can obviously change that when you express this model in terms of the subject we're no longer saying the subjects are multi very normal door did not you you get a observant everybody you're yeah there's no since there's only one like mean per person there's only one observation per person for their mean it's pushed around by what group they're in but otherwise it's you don't have multiple observations there's not multivariate normal there's no correlation structure at all it's just normal with a mean as specified by what group they're in and whatever their coefficients there are for the effective groups and across people distributed as having a variability specified by this scaled intercept ESPYs parameter finally we just say given up given and observation get the lien corresponding to that person and there's noise so it's actually a simpler started do this again I've maybe introduce the hierarchal between first Wow now because the hierarchal within version is very consistent with the repeated measure stuff that we previously done I just wanted to have this sort of here for you if you happen to ever encounter this it's relatively rare you guys can you think of maybe with the animal stuff you usually tend to be manipulating things between groups rather within groups maybe a incredible like research good initiative like they're over time like me and you need to measure the same thing mm-hmm all right um yeah yeah well the last thing I want to talk about today which will actually continue talking about the subsequent days because it's a very powerful tool hmdx I reckful mixed designs Gaussian error X implying crossed random effects H hierarchical em mixed G Gaussian error x is for Christ's random effects yeah so I mentioned last time that there's some different terminology used by different folks mixed effects model is what people often will use instead of hierarchical models I prefer the hierarchical model nomenclature within that mixed effects model terminology they distinguish between fixed effects which are the things that we typically talk about as the things we're interested in what's the difference between what effect does condition have what effect does group at that's the thing they distinguish that from random effect fixed effects as distinguished from random effects random effects in turn talk about some variable I mean the generic expression would be some variable in your data whereby levels of that variable are relatively consistent within themselves that varies from one level to the other in their outcome and you're not super interested in that variability you want to just allow your model to take that variability into account so it is not seen as error variable so it's not seen as measurement noise so an example that we're all familiar with if people in a given task if it's particularly if it's within subject people are going to vary from one to another and we tend to observe hi consistency within a person relative to the amount of variability we see between people so this is the graphic that we had previously where we've had a 12 and a bunch of lines per person there's large variability across the PB the time point there's large variability across the people relative to the amount of variability there is like from one condition to another another kind of way of expressing that is like if you actually had hierarchical data and there were observations that there's still a large variability between people relative to the amount of measurement noise more important to the things that we're interested in that is the effect of being in one Commissioner the other is how big that effect is across people relative to how people like how big the intercept variability is whenever you have an intercept to variability across people that's much bigger than the effect variability of cross people then you get a benefit in power to actually letting the model see that there's these different people and they differ in their intercept in order to see the fixed effect of one group versus the other so standard it very common practice and they'd we've been doing it all along and I just haven't phrase it this way to treat participants as random effects you actually get in some context people actually letting participants have a random effect on the intercept so saying that people have variability in what where their intercept is but not having a random effect on the slope educate any variability in the slopes that just seem as noise those I mean there's no cost to going to the full model that I've been proposing you guys do where there's variability across people in their intercept and variability trust people in their slope and the potential for correlation amongst those two things you're going to get more accurate influences to degree that there is that variability and potential for correlation but people aren't the only things that could be treated as a random effect in our experience even when we have people in our experiments so this is actually one of the reasons why I got introduced to well I think it's one of the reasons why mixed effects models started becoming very popular in psychology or flex how psychologists started to learn about them is because for the longest time researchers in linguistics had an issue whereby there was a lot of variability amongst people as they as is convention but they also knew their stimuli had a lot of variability so let's take sure of the simple lexical decision task so a word is presented on the screen and you have to decide as quickly as you can whether the word or not work well there are a variety of properties of words that will influence if it's a real word how quickly you're able to determine that it's a real words the word frequency is one thing concreteness whether it's abstract or concrete word maybe ding I'm not sure about that one but there's a variety of properties that influence the degree to which you're going to respond quickly and words are going to have like a given word will have its own word frequency and so if you the idea is that if you show a given word to a bunch of people that word will have a very consistent performance relative to other words so some words most people respond relatively slowly do it somewhere most people will respond relatively quickly too so you in the same way that you there's consistency across people in or there's variability across people relatively the amount of consistency that you observe within a person there's variability across words across stimulus examples relative to the amount of there's about the consistency that you get across people in the responding to that word so this is where the linguistics folks have been battling with a whole bunch of different ways of trying to solve this using like frequent dislike tests because they do one an OVA where it's the standard what we've if there were say two different conditions them and the greatest stimuli versus non degrees deny and looking at the effect of word frequency on your ability to do the exclusive judgment so look at two by two new analysis and you still had to have within your word frequency groups a bunch of exemplars and those of exemplars may be relatively matched and Kurt work for concedes the low word frequencies may all be may all have relatively low and we'll all the highs we have high but they still need to choose something so people would try to like maxim on all the other linguistic property as much as possible but you can't control for everything and there might be some residual variability amongst the words on things that you weren't measuring or things that the properties that you didn't know influence the data so in which case they would often do it do one a nova where they would treat participant as like the repeated measures thing and do a sender repeated measures and then they rerun the out of a-- and treat words as the repeated measures thing in which case they then get out to x values and they try to combine them in weird ways to actually estimate out the true effect of condition regardless of these two sources of variability person and word so that was a lot of well it certainly wasn't the very elegant approach and when mixed effects models game they became very quickly dominance in the mystic literature to the reason that it's much easier to express a model that simply says that there's variability associated with person there's variability associated risk word and these just combine together to talk about the meaning for a given person a given word and it's getting condition so h MZ x gives a little demonstration of this idea so it's called X crossed random effects because typically each person gets measured at each level of the words and its work its measured buddies person so they're completely crossed you can actually have non crops and this lot the same model will work just fine let's open up the project and I'm going to continue this from our next day as well so I'm just going to go through it at a fairly high level open up just like that in terms of what the data or what the stand model looks like and then we'll actually look through a few examples with data set so the data coming in is very similar as before with the exception of this first little bit and this last little bit so we have the number of subjects number of outcomes number of within-subjects predictors some of the special subjects predictors the outcomes within subjects predictors between fifty predictors and subject labels but we also have the number of words like a number of words and then what word goes with what outcome with each outcome we're still scaling the data and we're still going to have a homogeneous measurement noise model so there's only one estimate for measurement always across people and condition the expression of the effect of people vary from one to another is going to be the same and so that includes the idea that people are going to vary in the magnitude which they have an effective cross manifesting across conditions for this simple model that we're going to deal with we're just we're not going to have the different conditions of the experiment effect given words like the effects that a word has on performance so we're just the equivalent of that is just saying that words have a random effect on the intercept ie words vary from one to another in just the overall way in which they affect the outcome but that overall effect in which the affect the outcome isn't influenced by the condition so the given word will have the same effect across the conditions of the experiment will elaborate some more involved scenarios this is maybe one of the because often it's the case that a given word is only presented in like one condition so if it's a high-frequency word it won't be also in a little frequency word condition so you can think of the same way that we have between subjects manipulations and within subjects manipulations you can have between word manipulations and within word manipulation so if the same word were presented though in two conditions of like stimulus degradation that would be a written word manipulation so we'll get some more complicated models where we have within word effect here this is just a between word effect and real model and it's actually only an intercept model so each word is the its own sort of overall rate at which it pushes performance one way or the other and we're just going to let that we're just we're not actually interested in that as like thing that we will help us drive our theory but it by telling it that okay here's the different words and the words have this variability from one to another it allows the model to absorb what is otherwise would have seen as simply measurement error and will therefore be more accurate to my respects so the model structure is the exact same all the way through creating the subject coefficients a new little bit is expressing that okay we've got variability in the amongst the work and so there's variability amongst the word so that's what the word SD is and this scaled word debs is just word deviation so it's for each word what is the amount by which it causes deviation of the outcome so for each word there is one of those and will express that is just a normal distribution you'd want to maybe look at the histogram of I guess outcomes aggregated so it's mean forward just to double check that like it's a normal distribution looking maybe it's log normal or something like that but we'll have it as normal for now so with the expression for what the truth that there's a distribution of deviations across words we can then express that the Y outcome is exactly as before is a normal with some amount of measurement noise plus the subjects mean for the given condition that that outcome comes from plus there's extra a little bit now it's whatever the current word is for this outcome the word I we get the deviation associate with that word we add that deviation so if it's a word that has a that is cause the performance to be larger than average whatever this is say why is the speed so if it causes or the latency so reaction time it causes slower reaction time this will be a positive value the causes faster reaction time this will be a negative value so it will push around what otherwise would be regardless of word the prediction for this coast condition given the person in condition that's in so this would be like the simplest possible model of crossed random effects where you've got boobs having a separate edition of variability to the outcome that is your structured variability a given word has a consistent effect across people now one thing that I'll say still through maintained at the high level is that as you have more data about like a given unit that you're otherwise treating as a random effect you should expect less and less benefit of treating that thing as a random effect so let's take for example an experiment where you're not I gave the experiments moment ago where you had degraded and clear stimuli and vs and another variable in the experiment was high versus low frequency let's say you didn't have the high versus low you've just an experiment where it's lexical decision and degraded versus clear and you actually had stimuli as varying word frequency well one solution to that idea that there's variability in word frequency and there and we have a pretty good theoretical expectation that word frequency is going to influence the outcome so that you're going to be much faster at responding to high frequencies and low frequency you could just again absorb that variability associated with words and their word frequency by treating word as a random effect on the intercept in which case it would absorb that variability but you do better to actually add word frequency as a predictor in the model rather than just say that there's this random variability attributable to each word similarly if you even have the scenario where you've got high versus low board frequency used to decide to binarize it like that but there's some other properties you can add those other plus concreteness and saying lots of things and the more these sort of descriptors that you have for a given things the less variability will then be attributable to the random variability of the word so the inverse of that realization is realizing that that treating something as having a random effect on things it's like saying there's noise associated with this that I'm just going to leave as sort of labeled but Unser's unmeasured or I I don't know I don't have a structured way of putting numbers to why this word would be faster or slower than the other words so to degree that you do have labels beneficial to add those into the model the same thing applies to people so if you expect people across your population to contribute variability if you see that people in your experiment have their ability from one to another you can just add that in person as a random effect the intercepts and also maybe the effect but if you have some hypotheses about things that you've measured about the people that might be contributing to that very Billy like age or sex or all these other things your model will be better if you actually add those in as predictors obviously that requires you to have thought to measure those with words of these because words are things that we can measure at any time with people you had to have sought to measure the thing when they were that in front of you so it's partly why you get kind of simpler models on subjects as a random effect because you don't tend to get a ton of measures per person but then when you've got when you have something like lexical some sort of stimulus category you can go back and really reframe your model according to the properties of that thing so something to think about next day we'll talk about again more things you can do with the hierarchal bottle is included and having multiple cloak effect you can actually well talking about the idea that you might have nested random effects so this comes up a lot when you for example are doing research on couples and you're asking each person of the couple to respond on questionnaire you would expect that maybe there's a difference between the people amongst a couple but you also expect couples to sort of cluster together to be similar to one another so you can have nested random effect another sort of scenario would be students within schools and schools within different sort of districts so you'd expect students be responding on some suppose measure relatively consistent within a school and then you'd expect sort of districts to be relatively consistent so you can have hierarchy of how you're representing the data as having consistency at different levels from one inch with labeled group to another so we'll get into bills next class what you had 12 a scenario what was your scenario it's like way to track the nested random effects with couples yeah yes yes yep so siblings for example say would be something where you want to be able to serve label siblings as sort of a unit of random effect and then you can talk about like those two different from one another by some random amounts that's all right