Knowledge Distillation Explained with Keras Example
In this video I will be explaining the concept of Knowledge Distillation. In machine learning, knowledge distillation is the process of transferring knowledge from a large model to a smaller one. While large models (such as very deep neural networks or ensembles of many models) have higher knowledge capacity than small models, this capacity might not be fully utilized. It can be computationally just as expensive to evaluate a model even if it utilizes little of its knowledge capacity. Knowledge distillation transfers knowledge from a large model to a smaller model without loss of validity. As smaller models are less expensive to evaluate, they can be deployed on less powerful hardware (such as a mobile device)
If you like such content please subscribe to the channel here:
kzfaq.info...
If you like to support me financially, It is totally optional and voluntary.
Buy me a coffee here: www.buymeacoffee.com/rithesh
Relevant links:
en.wikipedia.org/wiki/Knowled...
keras.io/examples/vision/know...
cs231n.stanford.edu/reports/20...
arxiv.org/pdf/1503.02531.pdf
blog.floydhub.com/knowledge-d...
stats.stackexchange.com/quest...
colab.research.google.com/dri...