Signed Test Made Easy

Test e sign

hi guys it's Justin zeltser here from Zed statistics dot-com for the final video in a series on statistical inference this one is to do with non parametric testing now you don't need to see the other videos to understand this one it kind of stands alone in that regard but if you'd like to they're up on the website or if you're watching from university each of the videos will be up on their respective module home pages but in this video we're gonna be dealing with three particular nonparametric tests the scientist the wilcoxon signed-rank test and also the mann-whitney u test which is also called the wilcoxon rank-sum test just to be confusing we're gonna start with a definition of what a non parametric method actually is have a little look at the history of nonparametric methods in medical research and then i'm going to go on to three examples or an example each for these particular tests now as you've noticed there are two particular pathways we can be going down depending on the type of data that we have so if you just have a single sample then you're in this top arm or we can do a scientist or a wilcoxon signed-rank test so in that case you might have a single numerical variable measured across a single set of respondents alternatively if we have matched pairs will also be on this top arm up here now an example of a matched pair scenario might be where you've sampled the same people before and after some kind of intervention so you technically have two samples of data but because they're matched you can actually construct the differences between the two so say the improvement in some numerical measure and in that way you're constructing almost a single sample of differences so hopefully you can see that that would be very similar to just having a single sample in the first place so that's why either of these scenarios are technically the same and we can deal with with them this way however if you have two independent samples so say you've got males versus females on some measure that would definitely be two independent samples will use the mann-whitney u test or the wilcoxon rank-sum anyway I've kept the theoretical content for this video to a minimum and we're just going to be diving straight into three separate examples which I think will be quite helpful to for you to get your head around the tests themselves and also be able to use this video as a template potentially for answering your own questions so what's a non parametric method actually mean here I've written that non parametric methods allow statistical inference without making the assumption that the sample has been taken from a particular distribution in this case I've said here for example a normal distribution and you'll notice that in previous videos or in your studies generally you'll have dealt with means and standard deviations and both of those are parameters and you might also have recognized that in these testing that in the testing we've done up to this point we're very quick to jump to a normal distribution table to assess our inferences and in this case we're going to be trying to deal without our trusty normal distribution table well almost it kind of sneaks in towards the end but for the most part we'll be trying to deal without these parameters now the history of nonparametric methods is quite interesting there's a dude here that I've highlighted John Arbuthnot or our birth no I don't know and he's an English guy and in his paper an argument for divine providence from 1710 he was trying to assess whether there's a difference in the proportion of males and females being born in London over the period of about 80 years and in trying to do so it's actually been credited as being the first paper in inferential statistics and in the paper he uses the scientist which were just about to learn about in this video in doing so he showed that more males were being born and females that was the sort of prime conclusion for this paper and interestingly a secondary conclusion was that polygamy is contrary to the law of nature and justice which may have been a little less data-driven than perhaps the first conclusion I think it's a sign of the times of the paper I would suggest but this guy John Arbuthnot is quite interesting apparently is a quite a polymath was well known for his poetry and other stuff as well seemingly and interesting dude anyway let's get on to having a look at some of these actual examples of these nonparametric tests the first of which is the sign test itself so as I said this is where you have a single sample or matched pairs so here's the example hemoglobin levels in grams per deciliter were sampled from 10 female vegetarians to assess the prevalence of anaemia and here are those ten observations in grams per deciliter we're asked here to assess whether the median hemoglobin level for female vegetarians is less than 13 point Oh grams per deciliter now the reason why I've written median here is that as we said we're not really on terms of these parameters the mean and standard deviation and all that so we're technically using a nonparametric measure of the central location of this population so we can't use the mean so we'll have to use the median here which itself is a non parametric measure and I've used a different symbol the Greek letter eater here instead of MU you might have seen before it's almost like an upside down new isn't it but this is going to be our null and alternate hypothesis the null hypothesis being that the median is 13 grams per deciliter and the alternate being that the median is less than 13 grams per deciliter now if you've done hypothesis testing before you'll know that whatever we're trying to seek evidence for goes in our alternate hypothesis so we're seeking evidence that the median hemoglobin level for female veg O's is less than 13 and the null hypothesis turns out to be the converse of that so you could write here greater than or equal to 13 if you like but I was putting equal to 13 it's a convention thing it doesn't actually quite matter how you phrase that so the first question to you is that if the null hypothesis is true so say it is in fact equal to 13 the median that is how many observations in the sample would you expect to have a hemoglobin level under 13 hmm well we have 10 observations and if the median was in fact 13 you'd be correct in thinking there might be five observations below 13 and about five above 13 you'd allow for some random variation but that's what you would be expecting from your sample what did we get well I've highlighted the observations that are lesser than 13 here and they're all in pink so we get seven observations that are less than 13 so technically that means we have a sample which is more extreme than we would expect if this null hypothesis is true but how extreme is this sample is the fact that we had seven below 13 grams per deciliter and enough information for us to reject this null hypothesis well that depends and that depends on a particular distribution so let's have a look this next distribution shows the outcome from 10 observations of the number of I've called them negative observations here but what I mean by negative is just less than 13 so if we just call the observations that are less than 13 we can call them negative and those that are greater than 13 let's just call them positive if indeed the median was 13 you'd expect five negative observations but of course you could get four or six or three or seven depending on the randomness of your sample but if you were to get an extreme number like zero or ten that might start casting doubt on this null hypothesis right if all 10 of your observations were less than 13 you would have quite a decent amount of doubt cast on that null hypothesis so where does this distribution come from well it's simply a binomial distribution with n being 10 and the probability of a single event being point 5 so this will give us a good picture of how extreme our sample really is and here it is if we had 7 observations which we did out of 10 that were negative the p-value associated with that is 0.172 that's just the sum of all the heights of these discrete outcomes so that's a measure of essentially how extreme our sample is or how likely that null hypothesis still is so in that case we can see that we're going to be unable to reject that null hypothesis here our samples not extreme enough certainly not at the 5% level of significance if we had 8 out of 10 observations that were less than 13 the p-value would be 0.055 so we're getting closer to being able to reject their null hypothesis of course if we had 9 or 10 we would be able to reject that null hypothesis because our p-value is very small in that case but as we only have 7 we can say there's not enough evidence at the 5% level of significance to suggest the median hemoglobin level for vegetarian women is less than 13 point Oh grams per deciliter and if I rewind back really quickly to the sample again just to recap we found that there were seven observations less than thirteen or seven negative observations which wasn't extreme enough we needed nine or ten negative observations observations less than thirteen for us to reject that null hypothesis and that really is the sign test so you'll notice that the only piece of information we were using there was whether each particular observation whether it's positive or negative in other words is it on one side or the other side of this median value so it's their sign that we were interested in what we're about to do is incorporate an extra piece of information in the wilcoxon signed-rank test just from the name you might be able to predict what piece of information that is but let's find out here's the exact same scenario again it's the same example with the same ten observations and the same question is the median hemoglobin level for female vegetarians less than 13 now my question to you is what if all the women above 13 grams per deciliter so the ones in blue well only marginally above that level whereas the women below 13.0 grams happen to be significantly below that level so if we have a look at our sample you'll notice that 13.1 is quite close to 13 as is 13.3 14.0 is still pretty close to 13 but if you look at all of the pink numbers here the ones that are less than 13 they appear to be a lot more significantly less than 13 have a look at this ten point five ten point nine ten point one that's almost three whole units less than this critical value of 13 so is there a way for us to incorporate some of this distance from that median level in our comparison yes there is obviously so we have the same null and alternate hypotheses here but what we're going to have to do is rank each of these differences that are created from each of these observations so here's a table I've created where the first column is indeed those observations the ten observations we have in our sample I've kept the color coding the same so each row that's pink is one from a woman with less than 13 grams per deciliter is her hemoglobin level and the blue ones are the positive ones the second column here is the difference between 13 and this particular number so negative differences are obviously if X is less than 13 positive is if X is more than 13 so what we're going to do is rank the absolute value of these differences so this third column is in fact those absolute values and the ranking starts at 1 for our lowest absolute difference and counts up to 10 for 10 observations to the highest absolute difference you'll notice there's actually a tired rank here for this 12.3 value or I should say the difference being 0.7 and the way tied ranks work is just that if you consider these to be ranks 3 & 4 you have to take the average of those two ranks because they're the same and after finding these ranks we then apply the sine back to them so this ranking of 3.5 happened to be a negative one whereas the ranking of 1 happened to be a positive one and already we were getting a bit of a picture that the positive ranks happen to be the smaller ones whereas the negative ranks the 10987 the 6 they're all the big ones they're all the larger differences from the number 13 now if you've got statistical software at your disposal you obviously don't have to create this table you could just press the button that says wilcoxon signed-rank test and it'll come out with a test statistic and a p-value I don't want to show you what's under the hood and it's actually not that difficult especially with a small sample that will have to calculate this test statistic now I am going to take the first of these two possible test statistics the first one is just the sum of all the positive ranks and the next one's the sum of all the negative ranks so you could look at this either way but for the purpose of what we're about to use I'm going to take this top one here the sum of all the positive ranks and we're going to compare it to a particular critical value that'll tell us whether we can reject this test or not so let's find out how to do that so I've repeated here that our test statistic is eight because that's the sum of all those positive ranks the 1 the 2 and the 5 and n is 10 so to run this test if n is small enough we can compare T our test statistic to the exact distribution table now this is something you can find online if you just type in Google wilcoxon signed-rank s-- critical values or something like that you'll get something that looks a bit like this so we're busy doing a one-tailed test so we're in this final column over here these final two columns and if we're conducting a 5% level of significance test then it's this particular column we're interested in and scrolling down you'll notice that there's N on this left-hand column so if else if our number of observations is 10 the numbers supplied to us here is 10 now this is the critical value equal to or below which we can reject the null hypothesis and keep in mind we're always comparing the smaller of the two possible test statistics so remember the some of the positive ranks was 8 and as some of the negative ranks was 47 this is why I chose the some of the positive ranks so we had a test statistic that was 8 right so 8 is below 10 meaning that we can reject this null hypothesis it means our sample is extreme enough for us to reject the null hypothesis at the 5% level at least at the 1% you couldn't do it because the critical value is 5 which is less than 8 now if n exists on this table I would suggest using the exact distribution table but if n is a little bit larger and this table doesn't get down to it you can approximate this using a normal distribution so remember at the beginning of the video I said we're not going to be dealing with normal distributions and means and all that when n gets large of course the normal distribution will be able to help us out if required I'm not going to get delve too much into these formula but you can just sub in all the values into this T being the test statistic and technically there could be either of the two test statistics here because we have this absolute value situation but let's just say we used the test statistic 8 here n is 10 so you can substitute away and find your Z statistic is 1 point 9 9 and in this case we'll reject this null hypothesis if Z is greater than 1 point 6 4 5 where did I get that from it's just a typical critical value for Z when there's 5 percent in the upper tail you could calculate that from your said tables if you so choose therefore either way here we're going to be rejecting our null hypothesis which if we recap it means that the 7 observations that we had that were less than 13 had ranks that were quite a deal higher than the 3 observations that were greater than 13 and the result of this test is quite interesting when read in the context of what happened in the previous test you'll notice for our sign test that we couldn't reject the null hypothesis but with the wilcoxon signed-rank test we could so why is it possible to get these contradicting test results for the same data set well it all comes down to what in formation we're feeding into it for the sign test we were just interested in whether the observations were less than 13 or greater than 13 but we incorporated some more information in the wilcoxon signed-rank test and that was critical we incorporated the ranks not just the signs of each of the observations so I don't know I think that was a very good example to try to distinguish these two tests and how they might be used in a non parametric context all right we ready to move on let's now look at the mann-whitney u test also called the wilcoxon rank-sum test and as i sign posted earlier we are going to have two distinct samples here so hemoglobin levels were sampled from ten female vegetarians and eight male vegetarians in this case is there evidence of a difference in median hemoglobin levels so here's our sample females it's the same sample as before for the females I've just ordered it differently I've put it from the minimum to the maximum and here is a sample of eight male hemoglobin levels so the question is is there enough evidence to suggest that the median female hemoglobin level is different to the median male hemoglobin level and in this case you'll see it's a two-tailed test just doing it to tail to make a point of difference to the last example now much like the wilcoxon signed-rank test in this case we're still going to be ranking all of the data but no longer are we interested in the differences from a particular value 13 we're just going to rank them from smallest to largest so the smallest observation is that female ten point one so that gets rank of one the next one goes up to ten point five and that's rank two and then we get that male observation ten point eight which is rank three and you can continue on for all of the observations we still have those tied ranks at twelve point three so that becomes a tied rank of ten point five and ten point five which should have been 10 and 11 respectively keep in mind that those tied ranks could happen across the categories here you could have a tired rank for a male value and a female value there's nothing stopping that happening but in this case it hasn't so again the question is going to involve the sum of the ranks for females versus the sum of the ranks for males and if you sum up all those numbers you get 85 for the femur house than 80 sorry 86 for the females and 85 for the males now you might look at those and think straight away that ah that's very similar so clearly there's not much difference between females and males but don't forget females have two extra observations so it's not as simple as just looking at those two numbers so let's find out how we're going to incorporate all of that information in brackets here I put t1 equals 85 and what tends to happen is that we choose the sample with the fewer observations which in this case is males and that rank-sum is going to be our test statistic so t1 happens to be TM which is 85 now to find the expected value of t1 and when I say expected value I mean assuming the null hypothesis is true so the expected value of t1 it's just found using this formula here where n1 is the number of observations in the smaller sample and n2 is the number of observations in the larger sample so it would be 8 and 10 respectively and subbing all that in we get 76 so all things being equal we would expect the some of those male ranks to be 76 but they're not the 85 so how are we going to assess whether that's far enough away from 76 whether that's extreme enough for us to reject our null hypothesis well again there's got to be a difference if n is small or large always use the exact table if there's if it's available to you so even though I've written if n is small there's no harm in using this table if you find an exact critical value for your particular combination of N 1 and n 2 but let's see how that works again you can find this on the internet if you just type in critical values for the wilcoxon rank-sum test or you could probably write mann-whitney u test as well you'll get something that looks like this the first two columns here represent the two sample sizes M being the smaller of the two sample sizes and n being the larger so we'll scroll down until we have our combination of 8 and 10 eight males ten females and W here provides us with the critical values for this hypothesis test so if our test statistic lies outside this interval so less than 53 or greater than 99 then we'll be able to reject the null hypothesis and for those advanced players at home you'll notice that the middle of this interval happens to be 76 which should be no surprise to you but we have a test statistic of 85 so clearly in this case we're not going to be rejecting just checking we're in the correct column this is for where we have a two-tailed test and alpha is 0.05 which is in fact what we have so that's good now don't worry so much about what D represents P is actually the p-value associated with this interval appreciate because this is a discrete distribution this p-value won't always be the exact value you would expect from these levels of if against it would be slightly less each time but that's a bit of an advanced topic I'm not going to jump into that at the moment the important bit is this interval here which we've now dealt with and just for completion let's just see what happens if N or I should say n1 and n2 here if they happen to be too large for this table how would you deal with it well again it's the normal distribution to the rescue we know that in large samples everything tends towards a normal distribution thanks to the central limit theorem so you can just use this formula subbing in t1 the expected value of t1 and all the n1 za-- tend to you can fill in there and we find that the z-score here would be 0.8 now technically I don't think n is large enough in our case to warrant a normal approximation but I've just done it there for completeness unsurprisingly it can chords with our result from the exact table because that z-score is quite low and there's no way we'll be rejecting a null hypothesis based on that Zed score so quite rightly we say do not reject H naught so there's not enough evidence here to suggest that the median for females differs from the median for males so that's it that's nonparametric methods testing for statistical inference thanks for watching as I said at the beginning all the other videos are up on Z statistics comm along with a whole bunch of other statistical resources there's a podcast which I've started which you might be interested in and a few other things so yeah finally don't forget to subscribe to this YouTube channel catch your own