Text Classification Using BERT & Tensorflow | Deep Learning Tutorial 47 (Tensorflow, Keras & Python)

No video

Text Classification Using BERT & Tensorflow | Deep Learning Tutorial 47 (Tensorflow, Keras & Python)

Рет қаралды 162,592

Күн бұрын

Using BERT and Tensorflow 2.0, we will write simple code to classify emails as spam or not spam. BERT will be used to generate sentence encoding for all emails and after that we will use a simple neural network with one drop out layer and one output layer.
What is BERT? • What is BERT? | Deep L...
Code: github.com/cod...
Do you want to learn technology from me? Check codebasics.io/... for my affordable video courses.
Deep learning playlist: • Deep Learning With Ten...
Machine learning playlist: • Machine Learning Tutor...
🔖Hashtags🔖
#BERTModel #bertmodelnlppython #BERTtextclassification #BERTtutorial #tensorflowbert #tensorflowberttutorial
🌎 My Website For Video Courses: codebasics.io/...
Need help building software or data analytics and AI solutions? My company www.atliq.com/ can help. Click on the Contact button on that website.
🎥 Codebasics Hindi channel: / @codebasicshindi
#️⃣ Social Media #️⃣
🔗 Discord: / discord
📸 Dhaval's Personal Instagram: / dhavalsays
📸 Instagram: / codebasicshub
🔊 Facebook: / codebasicshub
📱 Twitter: / codebasicshub
📝 Linkedin (Personal): / dhavalsays
📝 Linkedin (Codebasics): / codebasics
❗❗ DISCLAIMER: All opinions expressed in this video are of my own and not that of my employers'.

Пікірлер: 147

@codebasics 2 жыл бұрын

Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced

@symaafsha6921 2 жыл бұрын

Sir can i use this bert layer code with LSTM/GRU for better accuracy ? @codebasics. It will be clear if you answer it?

@shreyshukla8168 6 ай бұрын

Import tensorflow_text = text Not working my Jupiter not book Mene install bhi kara to Colud not find a version that satisfied esa error aa raha he anybody please help

@codebasics 3 жыл бұрын

I had previously uploaded this video but code in that video had some issues handling imbalance in the classes. I've fixed those issues and recorded this new video. Thanks Abhishek and few others who pointed out the issues.

@mudassaraliansari8969 3 жыл бұрын

Hello, sir I was trying to seach "feature engineering codebasics" your playlist was not coming .After that I have to go to your youtube channel and from there I viewed that playlist.Please check if that problem is facing my me only .

@Knight-Walker 3 жыл бұрын

Thanks for making the video again. I appreciate the work you do for everyone.

@rayyanamir8560 2 жыл бұрын

@@mudassaraliansari8969 Same is happening with his machine learning playlist. Its not visible on his playlist but if i search it on search bar, It appears. Very strange.

@amoghdeshpande8068 2 жыл бұрын

I think these are the most underrated videos on deep learning. The concepts are explained so well. Please keep making more videos.

@mahedihassanshawon4821 Жыл бұрын

Underrated channel tbh..He needs more recognition. Thanks a lot for supporting us.

@fuehnix Жыл бұрын

Oh thank god, finally someone who explains well AND covers the topic in enough depth to be useful.

@ss-dy1tw 3 жыл бұрын

Thank u so much, need more videos related to NLP, need more advanced concepts/projects in NLP area

@codebasics 3 жыл бұрын

yes I will start working on NLP soon

@MS-BeatrizBosques Жыл бұрын

Thank you so much for your videos! You don't know how much you have helped me. I was really scared to dive into transformers but you have made it very easy to understand.

@maxhuttmann4760 2 жыл бұрын

You are awesome, this is first actual slow enough and easy BERT starting video i've seen and suited me very much!

@sanketsonu6701 2 жыл бұрын

Thanks. Very well explained. You explains the hidden insights which clears all doubts and make concepts simple and easy to understand.

@mahsasharifi2230 2 жыл бұрын

As soon as I watched this video I subscribed your channel. The videos and tutorials are super useful. Thanks for sharing these valuable knowledge with us for free :)

@kathira5691 Жыл бұрын

Thanks for sharing this nice well explained concept.

@shreyshukla8168 6 ай бұрын

Import tensorflow_text = text Not working my Jupiter not book Mene install bhi kara to Colud not find a version that satisfied esa error aa raha he anybody please help

@spider853 Жыл бұрын

I knew Jeff Bezos and banana has a lot in common 🤣 Great video btw

@natreven1122 Жыл бұрын

Thank you very much! You really helped me!

@Sandesh.Deshmukh 10 ай бұрын

Well Explained ..Thank you for this wonderful explanation 👏

@codebasics 10 ай бұрын

Glad you liked it!

@sudharsanb9391 3 жыл бұрын

sir i have seen your Complete Data analyst roadmap and Data Science roadmap video.I was very amazing.I request you to upload a video on Complete roadmap for learning DSA with resources.so that it can help students like me in placements.Thank You

@sciadventure Жыл бұрын

Perfectly Expalined! Thanks a lot.

@shreyshukla8168 6 ай бұрын

Import tensorflow_text = text Not working my Jupiter not book Mene install bhi kara to Colud not find a version that satisfied esa error aa raha he anybody please help

@kennethnavarro3496 2 жыл бұрын

Bro watching these tutorials make me want to blow my brain up with the amount of times you mention your previous videos.

@nastaran1010 6 ай бұрын

Hello. I remember you said that, NN is not suitable for working with text, as some disadvantages. why you used NN as a model here?.

@kathira5691 Жыл бұрын

I am getting No matching distribution found for tensorflow_text==2.12 error while installing tensorflow_text using pip.Could you please help on this.Thank you

@verma_raunit 8 ай бұрын

The code do not work now.

@muhammadalzabibib2650 4 ай бұрын

Do you have another code please i need it for university project

@nathaliliyanage5659 Ай бұрын

@@muhammadalzabibib2650 I need tooo

@Thayanhchamsoccoay 7 күн бұрын

@@muhammadalzabibib2650 stupid. I think u shouldnt go to university

@shakes115 2 жыл бұрын

Great Video! By the way, a small typo, the percentage of spam is 747/(4825+747),

@kemingwu2880 2 жыл бұрын

very helpful，thank you！

@dronacharyulu755 Жыл бұрын

can we use the hidden layers(only CLS) generated from the bert model as a features , to train the tfdistilled bert for binary classification task

@pythonking_stem1527 Жыл бұрын

Man, You are a legend!

@kisholoymukherjee Жыл бұрын

Thanks for the great explanation. Really heplful

@African-lady Жыл бұрын

Thank you so much for this video. This is very helpful for my master's project. Please the model you built in the video, is it a fine-tuning of the last layer of BERT or completely retraining all BERT layers?

@guitance23 2 жыл бұрын

Quick question: Once embeddings are created via Bert, can I not simply train a SVM or Logistic Regression / Naive Bayes on them? Do I have to really create a neural network? P.S. Great videos as always. I have learned much more in last 24hrs than few courses combined :)

@zououoz3588 2 жыл бұрын

I would like to know the answer as well :)

@chinmayamahapatra8783 2 жыл бұрын

Yes you can of course. But that would be context less classification and would kill the whole purpose of context in Bert. Normal classification results would not be that good.

@hiralpatel1838 2 ай бұрын

Hi, if I have multiple categories that I want to sort my data into (right now in this video there are 2: ham and spam) how might I adjust this model? The sigmoid activation function would not be usable correct?

@harsh2014 Жыл бұрын

Thanks for this nice tutorial !

@jatayubaxi4553 2 жыл бұрын

Great video. Very easy to follow.

@stanislavzamecnik3049 2 жыл бұрын

Thanks for the video. You are amazing!

@maxw5667 2 жыл бұрын

Very useful video, thanks a lot!

@luiscao7241 2 жыл бұрын

Thank you very much for the useful lesson. Can you tell me what the output format of multi-class text classification?

@anonanon9660 2 жыл бұрын

Simply add number of neurons as the number of classes in the last layer (in this video he used one, so change it), one-hot encode the classes, use a loss function that is used for multi-class classification.

@adamatom2826 3 жыл бұрын

Thanks for these great videos. Quick question - I have watched your whole maching learning+deep learning series so far, and I'm wondering: are there ways one can tell whether the models discussed in the ML videos (linear regression, logistic, random forest, etc) versus a neural network is better suited to a situation? Or can you only know by testing all of them out, like in a GridSearchCV? Could you perhaps discuss this in a video?

@codebasics 3 жыл бұрын

Neural networks are best for unstructured data (images, text, audio, video) and when the training dataset is huge. For structured data statistical models are preferred.

@adamatom2826 3 жыл бұрын

@@codebasics Thanks so much for replying and for selflessly producing these videos.

@sapnilpatel1645 3 жыл бұрын

Very useful video. Thank you so much.

@maheshreddy6871 3 жыл бұрын

You are teaching in Nice manner. Can we have NER task Architecture explanation for Bert & How it is Working and some code for implementation of NER

@junaidreshi2472 3 жыл бұрын

What about using sequence_output vectors as input to LSTM as it needs 3D input?

@arkeshashah4650 3 жыл бұрын

very nice Explanation, very helpful to me. Thanks. Can You make Video on ELMO Word embedding?

@ss-dy1tw 3 жыл бұрын

Mine is ur first like, God bless U and Ur family. Kudos to u brother.

@codebasics 3 жыл бұрын

I am happy this was helpful to you.

@anjalipatel9028 4 ай бұрын

When I am using bert,then it is not necessary to remove stop word from corpus?

@yusufsAnt31 2 жыл бұрын

great video, thanks sir!

@rajm5349 2 жыл бұрын

sir for getting embed values u explained grapes and banana r similar, then when we relate it to mails, it mean that the mails which r coming r not similar to his current situation and that result to spam

@sulaimankhan956 Жыл бұрын

very interested

@yonahcitron226 2 жыл бұрын

please do more in-depth stuff on NLP!!

@pearlrodrigues3036 Жыл бұрын

Thank you so much for your videos! But i have a doubt since balancing the data in multi label classification doesn't help cause words have similar meanings ,what can be done?

@sanity5277 9 ай бұрын

dude you are awesome

@prabhamelady1680 2 жыл бұрын

very nice video sir. Thanks

@rahil1575 3 жыл бұрын

Thanks for the video.can you please let us know,how can we proceed if the text input is high (more no of tokens)

@buddhikaweerasinghe4113 2 жыл бұрын

How do we save the model & use in a another application ? It throws an error when I load the trained model from saved path.

@roopagowda9271 Жыл бұрын

Hello...can you please confirm if removal of stopwords,numbers,stemming etc is required in this case ?

@LordPBA Жыл бұрын

wonderful!! but... what if we have 3 or more categories instead of just 2? Thanks a lot

@amityadav5797 2 жыл бұрын

everything work fine but when i m trying to fit the model it is giving me error as " ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float)." I m stuck so bad anyhelp would be appreiciatable, thank you in advance

@hadianasliwa 11 ай бұрын

when running this code return bert_encoder(preprocessor)['pooled_output'] I'm getting an error : TypeError: Exception encountered when calling layer 'keras_layer_7' (type KerasLayer). pruned(input_ids, input_mask, segment_ids) missing required arguments: input_ids, segment_ids. Call arguments received by layer 'keras_layer_7' (type KerasLayer): • inputs={'input_type_ids': 'tf.Tensor(shape=(2, 128), dtype=int32)', 'input_word_ids': 'tf.Tensor(shape=(2, 128), dtype=int32)', 'input_mask': 'tf.Tensor(shape=(2, 128), dtype=int32)'} • training=None

@aomo5293 Жыл бұрын

HI, thank you for good video. please have talked before about Elmo?

@Areshk 6 ай бұрын

I have almost 1200+ labels, It is a good idea to use this model?

@DiAd188 2 жыл бұрын

Hi Sir, Instead of your output I'm getting: Keys : ['input_mask', 'input_type_ids', 'input_word_ids'] Shape : (1, 128) Word Ids : Tensor("strided_slice_3:0", shape=(12,), dtype=int32) Input Mask : Tensor("strided_slice_4:0", shape=(12,), dtype=int32) Type Ids : Tensor("strided_slice_5:0", shape=(12,), dtype=int32) Do you know why I would get this output?

@rutvisheth2634 Жыл бұрын

if my task is classification with Bert and RNN-specially BiGRU then what output of BERT I need to use ? Pooled or sequential? If I am using Pooled then I am getting error of dimension as rnn requires 3d tensor. Please help with it.

@smartoo80 5 ай бұрын

@codebasics what if I have more then one category who to deal with that

@williamjiang1089 2 жыл бұрын

Hey, thanks for the video. I was wondering at the end for inferences, you get decimal values like 0.8 which leans towards "Spam". However, is there a way to specifically return "Spam" with like 80% confidence or something like that, instead of just the decimal values?

@aditya_01 2 жыл бұрын

yes right a custom function like def funct(score): if score>=0.5 return (score*100,"spam") else return (100-score*100,"Ham")

@himeshkoli8607 2 жыл бұрын

@@aditya_01 or else we can use np.where function to classify spam and ham

@rabailmushtaq147 2 жыл бұрын

I see this error when I try to run the Bert pre-processor and Bert encoder OSError: SavedModel file does not exist at: C:\Users\admin\AppData\Local\Temp\tfhub_modules\602d30248ff7929470db09f7385fc895e9ceb4c0\{saved_model.pbtxt|saved_model.pb} can you please tell me what's wrong ?

@bhuvneshsaini93 2 жыл бұрын

Hi, Last dense layer throws an error if classes more than 2, example spam, ham ,social. How to set output (Dense) (None, 1) 769 dropout[0][0] to output (Dense) (None, 3) 769 dropout[0][0]

@rohitnarwal3538 Жыл бұрын

I can't find the dataset can someone help me

@AI_4214 Жыл бұрын

Thanks for the video, I am getting an error saying failed to covert Numpy array to Tensor.

@shanumathew5449 Жыл бұрын

Can you do Spoiler Detection with BERT ? I have been trying for some time but I am not able to.

@chinnaanjaneyuludirisala5265 Жыл бұрын

Thank you for explaining Bert model. I am not why the model is taking 2 hours for each epoch, has anyone experienced the same ?

@bhupendrapatel7617 2 жыл бұрын

Using same code, I got error "Cast string to float is not supported [[node binary_crossentropy/Cast (defined at :1) ]] [Op:__inference_train_function_85259] Function call stack: train_function". I tried to solve error on my end but did not.

@subratasaha4446 Жыл бұрын

Sir If it is a multi class classification the where should I change the code

@umarghaffar9149 2 жыл бұрын

Hi sir, Thank you for this amazing video. I have followed your video and used bert model for text classification and the accuracy of my model is very low. Can you help me

@ayushbhalani7242 2 жыл бұрын

Can a BERT model be used for a task like scoring resumes according to job description ?

@maxlgemeinderat9202 2 жыл бұрын

Hi, great video! What would you to to fine tune this model? :)

@champruban3612 Жыл бұрын

helo sir, I want to know one thing for code mixing language like hindi in english letter does Muril BERT understand this hinglish which is english letter wriiten hindi pronounciation

@varunsayalauthor Жыл бұрын

Great video. I am facing an issue installing tensorflow_hub: cannot import name 'deserialize_keras_object' from partially initialized module 'keras.saving.legacy.serialization' . Any thoughts?

@user-kt2en3cv8n 6 ай бұрын

what i have to search in kaggle to get that dataset

@slatan7239 2 жыл бұрын

How can I download the datasets which is used in this video? Can you give me the link?

@pwchan9748 2 жыл бұрын

is it possible to add custom tokens or synonyms to the Bert model? for example, Ｊ=J, Ｑ=Q, A = Apple, something like these. if it's possible, how to do it on Tensorflow?

@kunwarankush6030 3 жыл бұрын

sir, please make one video on named entity recognization using bert

@christopherrahme780 2 жыл бұрын

Thanks a lot for the video. This is great for binary classification. Any idea on how to do the same when we have more than two classes? (e.g. science, technology, linguistics, other)

@ashwinshetgaonkar6329 2 жыл бұрын

loss fn, will achange and softmax activation fn

@guillaumeguitarian9642 2 жыл бұрын

Hi, thanks for the Vid. Is it possible to make that code run with and AMD GPU ?

@mithilanavishka4531 2 жыл бұрын

If the value is more than 0.5, how can it be a spam email? Why can't they be ham email?

@ajitkulkarni1702 10 ай бұрын

one question: we had 747 data points for each class, so how in confusion materix values are less as 187

@meceffeukada3767 3 жыл бұрын

What's the point of inputs classes (spam/ham) here ? why you didn't use them ?

@serhatkalkan2339 2 жыл бұрын

Is there much to adjsut for multiple classification?

@sadikaljarif9635 2 жыл бұрын

what is bert_preprocess???can i use this for distilbert for fake news detection???

@nekopus 2 жыл бұрын

anyway, can we convert this transfer learning to tflite format?

@tanmayparanjape3733 2 жыл бұрын

Can you please show how to plot the loss graph?

@saumyabansal3549 2 жыл бұрын

How can we apply BERT on multiclass classification problem?

@muhammadaminkhan9328 2 жыл бұрын

Thank you sir for your valuable lectures. Can you direct me to one of your content about XLNet Model?

@codewithgtm Жыл бұрын

Which algorithm we are using for the text classification here? Can anyone tell me please?

@mchafe 2 жыл бұрын

Please Sir, how can I use BERT Embedding as input to Embedding layer of an LSTM. Thanks

@reemaljaber 2 жыл бұрын

i copied your steps but the model is extremely slow how could i fix it .. it takes hours and hours;(?

@precious_orim Жыл бұрын

Thank you so much sir pls where can I find the code?

@hssp1534 2 жыл бұрын

Sir I have downloaded BERT encoder and preprocess as a rar file from the link. Please tell me how to use them in the coding? I dont want to load the files through http url but from the files i downloaded. Would be of great help, Thank you!

@sudarshanaramireddy373 2 жыл бұрын

Is there a code for NLP Model without labels (i mean Unsupervised ML) ?, i am struggling to find ;)

@arindammukherjee391 2 жыл бұрын

Can you please show a real time deployment of a model like this on AWS

@sulaimankhan956 Жыл бұрын

Sir, I need code as Classifier without sequential layers.

@AreebaBatool-nt2hz 3 ай бұрын

can you provide me the link of this data set

@mithilanavishka9884 2 жыл бұрын

Thank You very much for this greatd video, I successfully got theoritical knowledge throgh this tutorial and I could completed it, after that I applied this for real problem I need to solve when I train model, I got this error --> "Failed to convert a NumPy array to a Tensor (Unsupported object type int)." place where getting error --> model.fit(X_train, y_train, epochs=5) I tried different solution on the stackoverflow like below, but could not get a solution, 1.X_train = tf.convert_to_tensor(X_train) 2.X_train=X_train.flatten() If You have a idea on my error give me a hint

@ellezeus 6 ай бұрын

Grazie.

@ashwinideshmukh2198 2 жыл бұрын

sir what if i have multilabel dataset like 6 labels

@poojapatila5233 2 жыл бұрын

Sir if possible can you make a complete NLP playlist. Like how voice is converted to text, text preprocessing any how we know by learning from your tutorial. Then again how it will back converted to voice. ex: Alexa.

@codebasics 2 жыл бұрын

yes NLP playlist is in my plans