Sign Idaho Banking IOU Computer

Industry sign banking idaho iou computer

all right i was having some screen tearing issues but i think we're good now so let me pull that a little bit closer all right welcome to this unofficial part two uh of this object detection series so uh what we're gonna try to understand in this video is how to evaluate a bounding box prediction so you know we have some target bounding box for an object and we have some predicted bounding box and we want to have a way of quantifying or measure how good is our predicted bounding box for that object and for that we're going to learn about a metric called intersection over union and then we're also going to implement that in pytorch so that'll be fun so without further ado let's get started let's roll that intro and then let's get started with intersection over [Music] union so the question is how do we measure how good a bounding box is so we have an image with in this case a car in it and we are given a target bounding box for that object and then we have some prediction bounding box for the same object and i guess here we can kind of visually inspect and see that well i mean it's not a perfect bounding box but it's not terrible either uh so it i guess this bounding box is sort of okay but we can't be you know visually inspecting every every bounding box prediction and also we want to have a way of quantifying and saying sort of more specific specifically how good this bounding box is so what we're going to do is we're first going to calculate intersection of those two bounding boxes which is this beautifully drawn yellow figure and then we're going to calculate the union which is the combined area of both of those bounding boxes which is this again just beautifully drawn pink region so the metric we're going to use is intersection over union and it's pretty self-explanatory in a way we we you know the iou is equal to the intersection which is the the yellow the p sorry the yellow area and then we divide that by the union which is the area of the pink region and so um i guess more formally the it's uh the area of the intersection divided by the area of the union so what we're going to get from this is a is a value between 0 and 1 indicating how good the bounding box prediction is so for example if we have a bounding box if you have a iu score of one then that means that the prediction box was you know absolutely perfect um and if we have a iu of zero then that means that they were not even there were no intersection between those two bounding boxes and sort of a as a rule of thumb is that if you if your iou is greater than 0.5 then it's considered decent and this is sort of the the mark where you go from saying that a bounding box is actually accurate if it has an iou of 0.5 if it's greater than 0.7 you say it's a pretty good bounding box and if it's greater than 0.9 then that bounding box is almost perfect because you're almost never going to get or you are never pretty much going to get a iou of one but still the question or the solution of how to find this is still not very clear you know how do we even get the intersection because remember that all you're given is just two lists of numbers one list for the first box and then for the second box and from these from these uh four values in the bounding box we can form uh the bounding boxes for for those two uh but you know we can visually see what the intersection is but it's not very clear um exactly how we do it from just the numbers uh so just one thing to keep in mind here when we're doing this all of these calculations that we're going to do or exam to show for this example is that the origin in computer vision for images is in the top left corner uh so you know we have that and the origin is at the top left and then as you go to the right of the image as you move horizontally your x value is going to increase and then as you go down vertically you're going to increase the y value so for our image here the origin is 0 0 at the top left and then at the bottom right depending sort of how large your image is maybe it's 300 by 550 in this case because it is slightly taller than it is wide or yeah i mean it's quite a lot taller than it is white but anyways so uh for our image then uh what those values as i mentioned in the previous video what these values correspond to are the corner points for for those bounding boxes so now what we are looking for is the is the corner points of the intersection between those two which in this case would be the yellow region and as we can see from this example is that that would be the the red box top left corner point and then the blues bottom right corner point but um but um how we can find these more generally and with a formula is that why x1 rather for the for the top left corner point of the yellow box is the maximum of the x1 values of the two boxes and in this case since the red corner point of the top left is further to the right than the blue one then we can see that the red one is going to be the um the maximum of those and also y1 we calculate by taking the maximum of the y1 values of the of the top left corner points so again we can see that this is also the red point because it is further down than the blue top left corner point so uh and again this is also what we see right the the top left of the yellow region is exactly the red point for the bottom right corner point of the yellow region we instead take the minimum of the x2 values so the minimum in this case is going to be the one that is um most to the left of the of the bottom right corner points and in this case that's going to be the blue point and then for the y2 value we're also going to take the minimum of the y2 values uh and again since the blue one is a higher up than the the red one is further down than the blue one and we want to take the minimum so again we're going to get the blue one for that which is exactly what we expect so let's look at a little bit of a more difficult example to again illustrate this uh this same concept so here we have two bounding boxes again we have a blue bounding box and a red bounding box and our goal here is to find intersection or rather our goal is to find the corner point of the yellow region so looking at the the top left corner point of the yellow region we first want to calculate the x1 value and again the x1 value is the maximum of the blue box the top left x value and the and the red one top left corner x value so the x one of the two boxes and in this case uh that's just going to be uh let's see that's going to be the red one right because it's further to the right than the blue one so that's going to be the red one and then for the y1 value we're again going to take the maximum but now the the blue one is further down than the red one so we're going to take the y1 value from the blue box so moving on to the to the bottom bottom right corner point of the yellow region that's going to first be the again now the minimum of those two boxes and the minimum of those two is in this case going to be the blue one so now we have x2 from the blue box and the y2 is going to be the minimum of those y2 values in this case that's going to be the red one because it's further up than the the blue bottom right corner so again we see that this formula holds in general so the formula for calculating this yellow region the corner point is that we take the maximum both for x1 and y1 and we also we then take the minimum of x2 and y2 all right so that's for finding the corner point of the yellow the intersection box we then need to calculate the areas and then we also need to calculate the union but i think i'm going to show that in the code because now we've done all of the tricky part so if you follow this far i think it's going to be a relatively easy following the code as well all right so what i've done is that we have first of all we have some test cases that i've done uh unit tests so that we can make sure that our implementation is actu actually correct i'm not going to go through that in this video the test cases i did but it's going to be in gita if you're just i guess curious to see how how that looks like but anyways we're going to go into this iou dot py file and our goal now is to to do this implementation so i guess what we need is an import torch then we're going to define intersection over union and we're going to get some boxes boxes predictions so i'm going to call the boxes preds and also going to get some boxes with their labels so boxes labels and so you can imagine that this is just one box with four values and one box with four values for the label but this can actually be a batch uh so i'm going to do it in a general implementation so that you can send in an arbitrarily number of examples and it's still going to work so what we want to do i guess first of all is just get uh box one x1 so that's going to be boxes uh preds of dot zero and one and then box one uh y1 is going to be boxes preds dot dot uh one and two and so if you're unsure about this dot dot that just means all the dimensions previously uh so uh you know i guess this is not really needed although you'll see later on it it's actually useful to do this uh when we're implementing the yolo algorithm but you know for now you can just say that it's uh n comma four uh where n is the number of bounding boxes all right so that's the dimensions of of a bound boxes preds so we can maybe write that uh boxes preds shape and then boxes labels shape is n comma four as well and then so we need box one x two is boxes threads uh 2 and 3 and i guess we don't have to do this we can just copy it like that and then we're going to need uh y let's see y2 and then we're going to need x uh box two x one box two uh y one and then let's see what we need we need to copy those and then we need x two y two all right so for let's see here box that's going to be three and four and uh yeah so damn this explanation was not that good so basically what i'm doing here why i'm doing this three to four is because i want to slice the tensor to maintain its shape so i want the output tensor shape of this to be n comma one all right if you would just index with with this like doing three that's going to remove the second dimension of the tensor just to be n but i want to keep it at as n comma 1. so that's why i'm doing this slicing in this way and then we're just going to do pretty much the exact same thing for box two like that uh three and four so you know here i'm not doing anything in particular really i'm just extracting the points and then we want to get x1 and this is the x1 for the intersection this is going to be the maximum of box 1x1 and box 2 x1 then x and let's copy that so we don't have to copy that we're going to do that for y1 we're going to do that for x2 and we're going to do that for y2 so for y1 we're going to take box 1 uh y1 and then box 2 y1 then for x2 we're going to take the minimum and we're going to do that also for y for y2 and that's going to be box 1 x2 box 2 uh x2 and then box 1 y 2 box 2 to y2 right so now we have the corner points of that intersection we're going to do intersection is the x2 minus x1 right that's the that's the width of the of the bounding box then we're gonna do uh times uh y2 minus y1 now there's one edge case that i didn't talk about and probably i should have is uh what if they are don't intersect at all um so that's a scenario but that we need to cover and if you i encourage you to just draw out two examples two bounding boxes where they're where they don't inter intersect uh but what we're gonna do is we're gonna do clamp of zero so we're going to do this subtraction and if they don't intersect then at least one of these are going to be 0. so again i encourage you to draw this out just to get some better intuition but this so this clamp right here uh clamp let's see dot clamp of zero is for um is for hk is for the case when they do not intersect because then if they do not intersect then the intersection should be zero all right so then we need to box one area is box one x x two so again we've calculated intersection of the of the area of the intersection but we need to now calculate the union so we're going to calculate box one area and box two area and that's just going to be box one x two box one x one then we're gonna do times uh box one y one minus box one y two all right uh and i guess maybe just to be certain because the area can't be negative that it shouldn't occur but maybe we can just do an absolute value just to be certain of that all right so we can copy that and we're going to now calculate box 2 area and it's going to be pretty much the exact same thing we're just going to change that to box 2 box 2 box 2. all right so now what we want to do is we want to return the intersection divided by the union and the union is going to be box one area plus box uh two area but one problem here is that we've added the intersection of those two areas uh two times right one's for the box one area and then one's for the box two area so this is this follows a more general formula if but uh so that's the box one area plus the box two area minus the intersection all right that's just a formula for union and then we want to add just one e minus six just for numerical uh stability all right so that should be intersection over union um so now let's try and run this iou test oh god damn oh yeah yeah so one thing yeah okay so we can do uh a box format as well since you know you can have a varying of these box formats i told you you can either have the corner points or you can have one corner point and you can have the height and the width but there are also scenarios where you have the midpoint so you give you get two values corresponding to the midpoint of the bounding box and then you get the height and the width and that's actually what you get for for the data set for the yolo which was based for the yolo algorithm so that's perhaps important to cover as well so i'm going to do box format as midpoint or yeah we can set that and that as default but so this right here this is if box format is equal to corners i'm just going to call it that that's what we what i mean when i say that we have the top left and the bottom right corners but if we would have you know if box format is equal to the midpoint then we need to translate this into the corners uh so in this scenario we don't have to do anything but if it's the midpoint we need to do some calculations so that we make sure that it's in at the midpoint uh right so that so that we make sure that we have the the top left corner and the bottom right corner for the bounding boxes so actually i'm just going to copy those in and i'll i'll explain them so they aren't really too difficult i think so um all right so basically if we want the x1 of the first box then this is the midpoint so if we divide the the width of that box by two uh then we'll get the top left x1 coordinate and then for the y1 again we're subtracting the the height by half because we're at the center of the object and then we want to get to the top left and the bottom right uh coordinates so we're just subtracting or adding the the height and the the width by half so basically if if the box format is midpoint which we're going to use for the old algorithm then we'll just re uh convert them back to the top left and the bottom right corner uh else if the box format is corners then we don't want to do anything right since that's then it's already done all right so let's go back to our test case and hopefully it works now failure is two all right two hours later oh my god that took so long to find jesus christ there should be boxes labels i'm so tired boxes labels labels labels i'm not even sure how we got some of those test cases incorrect to be honest that makes no sense oh yeah that's because most of my tests are for the midpoint but yeah that makes sense all right but it did it did fail so now we ran it and we got all of them correct all right that's pretty nice so yeah uh hopefully let's do some doc strings as well for this all right so now we got doc string and uh so yeah we take this input this box is spread boxes labels box format and then we output the intersection of union for all of those examples um you know one thing i just realized is that this implement tion is uh we don't even i think this is well we use torch here so it's dependent on torch but they do with very very minimal changes you could use this for numpy or i guess tensorflow as well so this uh this was uh intersectional reunion hopefully you were able to follow even though it's a it's a very pretty simple concept it can be pretty complicated sometimes alright so thank you for watching this video and i hope to see you in the next one [Music] you