PRUNE LOSS ADJUSTMENT Form

Instructions and help about PRUNE LOSS ADJUSTMENT

hi everyone in today's video I'm going to talk about how to compress and optimize your model let's say that you have trained or fine-tuned the model and now you're ready to deploy it for everybody to use but you find that the latency is too slow and you want to make your model faster well today I will show you four ways of making your model faster quantization pruning knowledge distillation and Engineering optimizations my name is B I am machine learning engineer and a PhD in natural language processing without further Ado let's get started with quantization neuron networks are large and take up a lot of space because they have millions or billions of parameters by default when you train your neural network usually the parameters are stored in fp32 which means that each parameter takes up 32 bits the idea of quantization is to reduce the Precision of the parameters into a format that takes up less space for example 16bit floating point or int 8 if we store all of the parameters in int