Inside TensorFlow: Quantization aware training

Рет қаралды 14,675

Күн бұрын

In this episode of Inside TensorFlow, Software Engineer Pulkit Bhuwalka presents quantization aware training. Pulkit will take us through the fundamentals of quantization aware training, TensorFlow/Keras API used to achieve this, and how it is implemented during this tutorial.
Documentation → goo.gle/32MN60q
Github → goo.gle/30YihDB
Add the Inside TensorFlow playlist → goo.gle/Inside-TensorFlow
Subscribe to the TensorFlow channel → goo.gle/TensorFlow

Пікірлер: 38

@foolmarks 3 жыл бұрын

GitHub link doesn't work. Audio is terrible.

@autripat 3 жыл бұрын

Hey all, at 14:26, are we missing the quantize_annotate_layer wrapper over the Conv2d layer (inside Sequential), like this: quantize_annotate_layer(tf.keras.layers.Conv2d(32, 5, input_shape=(28,28,1))

@PremKumar-qi3cd 4 жыл бұрын

When I try to post-quantize(int8) the SimpleRNN model for a time series data, it is throwing an error saying only single graph is supported. So Does the RNN, LSTMs support for quantization and conversion to tflite models? And If yes, how can I address the error? Thanks in advance.:)

@athreyamurali1439 4 жыл бұрын

Hey can you re-upload with better audio, please?

@alias15vapour 3 жыл бұрын

Sorry about that. I recorded the audio locally so it's better, but forgot Airpods audio compression over bluetooth lost quality.

@athreyamurali1439 3 жыл бұрын

@@alias15vapour All good, it happens. The topic seems really interesting tho, so I'd really appreciate it if you could re-upload or re-record it sometime. Thanks!

@alias15vapour 3 жыл бұрын

@@athreyamurali1439 - Thanks. This takes a bunch of post-production work, so a bit unlikely tbh but me (or someone else on the team) will definitely do this and a better job for the next version.

@Hav0c1000 3 жыл бұрын

Hey Pulkit, Say I wanted to constrain quantization parameters to power of 2 values. Would that be supported?

@morekaccino 4 жыл бұрын

I can't hear anything

@andresfernandoaranda5498 3 жыл бұрын

same

@sunnyguha2 4 жыл бұрын

Get better microphone

@raisaalphonse4094 3 жыл бұрын

I'm using QAT for a functional model only, but I'm getting a value error saying, quantize_model '`to_quantize` can only either be a tf.keras Sequential or ' ValueError: `to_quantize` can only either be a tf.keras Sequential or Functional model. I'm not really sure why I'm getting this error. Could anyone please help me out in this?

@shubhammane6357 Жыл бұрын

I tried QAT, as result I got .h5 model with quantize wrapper layers, I want to remove it and get back my original model with modified weights, How can I dot that?

@sanjoetv5748 4 ай бұрын

i am having a problem when i convert my .h5 to tflite,, when i test the tflite on my mobile app the accuracy is so much lower than when i try to run the .h5 on jupyter.... my question is does quantization aware training can help me to lower the accuracy loss when converted it to tflite after the quantization aware training? please someone help!!!

@anishdeepak1826 2 жыл бұрын

I have trained ssd_mobilenet_v2 model using object detection api and saved the model as .pb file. How to apply the quantization to a my model. I dont have .h5 file.

@lisali6120 3 жыл бұрын

Thanks for sharing! Does it support mixed precision?

@bryanlozano8905 3 жыл бұрын

it should, he mentioned custom quantization for specific layers

@alias15vapour 3 жыл бұрын

QAT does emulation of model execution in certain precisions so model accuracy is preserved. If that's your goal, you can totally do it like Bryan mentioned. But it's unlike mixed precision for training.

@sreeragm8366 3 жыл бұрын

Is there any scenarios in which quantisation shouldn't be done? Like, Incase I want to convert it to other formats supporting optimization, such as TensorRT.

@alias15vapour 3 жыл бұрын

That depends on your needs. If you want to use TensorRT for optimization that works fine as well. Quantization is useful if performance is a concern for you.

@yoloswaggins2161 3 жыл бұрын

Can this be used for tensor cores on Nvidia GPUs or is it only for embedded devices?

@alias15vapour 3 жыл бұрын

By default it supports the TFLite Quantization spec. If you want to use it for Nvidia, you would have to write custom quantization configs specific to NVidia. But it absolutely can be done.

@yoloswaggins2161 3 жыл бұрын

@@alias15vapour Thanks for the answer, would that be writing CUDA kernels for this or could you wrap with something higher level like Tensorrt?

@alias15vapour 3 жыл бұрын

@@yoloswaggins2161 You wouldn't need to write any kernels. You would just need to arrange the TF graph in a way that it emulates the quantization on Nvidia chips. It would just reuse their existing kernels. Possible to use TensorRT but you would need to know deep internals of TensorRT to construct the graph correctly.

@yoloswaggins2161 3 жыл бұрын

@@alias15vapour I see, thank you.

@rupeshmohanasundaram6718 4 ай бұрын

for object detection, QAT Works, if so how?

@sairamvarma6208 3 жыл бұрын

The Github link in the description doesn't work

@alias15vapour 3 жыл бұрын

Sorry about that, there's a typo. Just use the link below.

@ramamunireddyyanamala973 Жыл бұрын

Very good Sir

@Lisa-hb3js 3 ай бұрын

I got this error whatever I do (the same if the network only contains Dense layers...) : ValueError: Unable to clone model. This generally happens if you used custom Keras layers or objects in your model. Please specify them via `quantize_scope` for your calls to `quantize_model` and `quantize_apply`. [Layer supplied to wrapper is not a supported layer type. Please ensure wrapped layer is a valid Keras layer.].

@opydas4548 15 күн бұрын

Have you found the solutions?