Hi Krish, I was waiting for this series from very long. Thank you so much for starting it. Hope you will cover all the necessary and important topics related to this playlist. Cheers

krish sir sach main jis taraha se aap samjhate ho na koi nhi samjha sakta itni saari videos dekhi hai maine bert or transformer ki aap bahot easy way main samjhate ho sach main ek baar main direct dimaag main ghus jaati hai aapki baat thanku sir hum jaise gareebo ka bhala karne ke lea.

Most awaited series sir , thanks

Very helpful and informative for beginners! Thanks a lot!

Please continue this series, it is very important.

Always great information from your channel sir🙏

Thank you Krish for making the video.You are doing great.God bless you.

we love it because your are great in work and perfect in every thing 😍

Great stuff,can you also show multilabel text classification with hugging face transformers

Sir, you made a mistake while doing the predictions. The model predictions were on your 0th index. But, you have taken it as 1. Thats' why in your confusion matrix there no FPs and FNs. But, rest all of the things are very good and insightful. Thank you sir🎩

awesome - so clear!

oh my god. Sir are you physics wallah? I used to watch your physics lectures during high school. Right now I'm studying computer science in Canada. Thank you for your help.

Thank you Krish!

Great video Sir

Could you please talk about how you decide the number of training epochs? And what are warmup steps?

Thank you i have a large corpus of manuals to fine tune which is not possible to prepare in Q&A format is there a method to use a sentence transformer to create a chatbot?

sir please make one video on any of your channel that as a data scientist what type of work you do in company and what type of work you were doing in past as a data scientist

Hi Sir, I come to know that to change the pre-trained embedding size, we have to fine-tune model. As per the pre-trained BERT models provide 768 size pre-trained embeddings, Can you please tell if there is another way for the same?

hii krish will you please make a video on fine-tuning the fine-tuned model ? Its quite unique concept and not much content is available about it

TFDistilBertForSequenceClassification is for sentiment analysis right?, which classifier should I use for question-answer classification?

How to train transformer model on any of sentiment analysis dataset like movies reviews etc..? Plz make any video on it.

Hi thanks for your video. Why did you train only with 2 epochs? is it so powerful? that the model is working perfectly.

Hi, I'am trying a different dataset with your tutorial, I am using a BERT Based transformer to fine tune with a protein dataset. However, when I get to the training part I get: "`Checkpoint` was expecting model to be a trackable object (an object derived from `Trackable`), got BertForMaskedLM". Do you know why could this be?

Hi Krish, how to fine-tune model for semantic search? I want to customize some text to be similar to some text. by default semantic search is giving the wrong result. so how do I correct it?

Sir, I have learned a lot from this tutorial. I was able to use the code as it is. My question is, how do I load the fine-tun model for use? I have file save as config.json & tb_model.h5. how do I load these two files and test them. Thank you.

Why we converting our features to list, is this because tokenizers will preprocess it as raw data and make it a numpy array or dataframe for transformers ? Can't we pass numpy array or df to a tokenizer ?

Hello sir, AS for TFTrainer is depriciate now, what to do for train the model now? I am using transformers version = 4.37.0

Hi sir, how can we convert the encodings to dataset objects using pytorch?

Hello Krish, This tutorial is very nice and helpful! Can you also make one video on Seq2Seq some transformer model? I am not getting any proper source (tensorflow) for it. One which getting not able to unserstand. Fine Tuning Pretrained Model On Custom Dataset Using 🤗 Transformer for Seq2Seq

my question is that in order do we need to manually create a dataset or is there any other way to generate a dataset to fine-tune on

thank you so much, you saved my life !!!, just wonder when I trying to fine-tune a model, fine tune a pretrain based model or fine tuning a pre-trained model that has already been fine-tuned by others? which is better ?

sir its a request please do all bert and transformer implementation using tf or keras only plsssssssssssss.

Sir, please explain some thing about deep generative models:

Just one question Krish..Have you done the preprocessing of text or transformer will take care of it ?? Instaed of sentiment analysis can we do document classification using transformers ?

Kindly explain the role of all the parameters that u are using in a project.

Hi Krish, can you please make a video on Wav2Vec2 model customization?

plz make a vdo to use vision transformer for image classification.

Maybe an example of how to train SBERT with your own domain?

Thanks krish🙏🙏🙏🙏

Hi Krish, It's really great learning from your videos. I have followed your approach and facing 1 issue. The target column of my dataset has 6 values and the pd.get_dummies gives me true, false values instead of 1 and 0. so when you applied y = list (pd.get_dummies(y, drop_first=True)['spam']), i can't apply that for my dataset. Can you please help me how do i do that and what all subsequent changes (if any) i have to make. It would be really helpful to me. Thanks!

Hi, Can we use ALBERT model on top of it to see the difference?

Nice video. How do we use BERT's masked language model on a custom dataset? For example, if I have a dataset with over 100,000 tweets, can the masked language model be used to predict a tweet for an input? Example: "You look [MASK] today."

Is this updating all the layers of the model, or only the last ones?

Thank you! It was great. I tried it and got an error: "Method `strategy` requires TF." Any clue how to solve it?

hello. I am getting this error "TypeError: '>' not supported between instances of 'NoneType' and 'int'" for the step trainer.train()

@Krish Naik, Can we use this tf_model.h5 for transfer learning? I hope yes. If not suggest.

Hello Sir, I was confused as to where we are using the positional encodings and the semantic encodings along with word2vec... could you please clarify this?

will it work for any text classification dataset?

Hi krish, can you post the video regarding key value information extraction custom model building from scratch using layoutlmv3

Hello! Can you fine-tuning T5?

After entering wandb key, I got this error : TypeError: '>' not supported between instances of 'NoneType' and 'int'

Hello @krishnaik06 can you please do a similar model for face recognition as well, for custom data set.

Hello Mr. Naik. Great video sir. I have followed your code step by step and faced a bizarre error. TypeError: '>' not supported between instances of 'NoneType' and 'int' Searched everywhere couldn't find the solution. Can you please help me with this. It would be a great help. Thank you.

how to use the train model for chatbot

how long does this fine-tuning process take please?

sir, when analyzed the data its imbalanced the spam category is less as cmp to ham so need to balance it first

Hi :) Why did you get 100% accuracy? The confusion matrix is showing that. There is only two epoch and it must be something wrong about it. I should admit that I have done the steps for *electra-base-discriminator* and get the same result as you for two different datasets. Can you please check why we get 100% accuracy?

I recently joined one company as DS intern. While watching your video, i understand it pretty well. But while working on anything in company, self confidence just goes down. Why this happens? It happens with everyone or with me only?

Can you share the exact documentation link?

I wish it was somewhat different from 🤗 documentation example

can I train this on 125 class dataset.?

train_encoding = tokenizer(X_train, truncation=True, padding=True), ValueError: Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers. this error is not remove can some1 please help me

If you could kindly make a video on translation model on custom made dataset like hindi to english language. and show the accuracy result.

so how if we more than two labels, can you help me ?

Here we are freezing the top layers of the transformer and just training the classification layer isnt ? When we are instroducing our custom data , that is changing weights for classification task only isnt

Hello, thanks for the video :) . I had one problem arising when training the model (around minute 11 in your video). I got this error when using your notebook on Colab and launching the instruction "trainer.train()": TypeError: '>' not supported between instances of 'NoneType' and 'int'. Does anyone know what I should do? Any help would be really appreciated

How can we deploy an app made on embeddings like these?

What if i have a bengali language dataset? Can we do the same in that scenario?

first time seeing this

Hi Krish , can u do some sentence similarity modeling exercise

i tried doing it but not able to import TFTrainer

Hello sir, I appreciate your work for knowledge sharing. I do have one doubt regarding the custom dataset which you have used, you are getting 100 accuracy on the test dataset. I think it is not possible I have seen in some papers they got 95 accuracy is the highest by using the BERT model. Please can you reply to this comment?

Need help with loading the saved model :(

Sir, how can we extend the same for multiclass classification or multilabel...please give me a solution.

Is it a CPU based on training.

I'm trying to do sentiment analysis and my dataset's label column have 3 values. 0 - neutral , 1 - happy, 2 - sad. (not true or false like yours) Is there anyway to work with that?

how to do that trained huggingface model on my own dataset? how i can start ? i don't know the structure of the dataset? help.. very help how I store voice and how to lik with its text how to orgnize that I an looking for any one help me in this planet Should I look for the answer in Mars?

Do you know why do I get error: "TypeError: '>' not supported between instances of 'NoneType' and 'int' " after trainer.train()?

Hi, Krish, when i run i got this error ImportError: cannot import name 'TFTrainer' from 'transformers' please help me to resolve it

Hi Krish ..Getting Error " TypeError: '>' not supported between instances of 'NoneType' and 'int' ' in trainer.train() please help

Anyone face this issue pls help me to resolve this issue TypeError: '>' not supported between instances of 'NoneType' and 'int'

is anyone facing issues installing TFTrainer ?

Please Fine Tune the "Question-Answering Model" with own datasets....

where is rest videos? i want complete course on hugging face. and i dont have card, can i pay through upi id please?

Hello everyone, can anyone please help me out as I am getting TypeError: '>' not supported between instances of 'NoneType' and 'int'

why you giving y_test of test data when you training the model. thats why its giving 100% accuracy this is not the correct way i think so

typeerror: '>' not supported between instances of 'nonetype' and 'int' trainer.train() is throwing this error... Have anyone faced this? If yes, please help me...

Hello, I got an error with the TFTrainingArguments: --------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) File ~/anaconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1076, in _LazyModule._get_module(self, module_name) 1075 try: -> 1076 return importlib.import_module("." + module_name, self.__name__) 1077 except Exception as e: File ~/anaconda3/lib/python3.10/importlib/__init__.py:126, in import_module(name, package) 125 level += 1 --> 126 return _bootstrap._gcd_import(name[level:], package, level) File :1050, in _gcd_import(name, package, level) File :1027, in _find_and_load(name, import_) File :1006, in _find_and_load_unlocked(name, import_) File :688, in _load_unlocked(spec) File :883, in exec_module(self, module) File :241, in _call_with_frames_removed(f, *args, **kwds) File ~/anaconda3/lib/python3.10/site-packages/transformers/models/distilbert/modeling_tf_distilbert.py:34 26 from ...modeling_tf_outputs import ( 27 TFBaseModelOutput, 28 TFMaskedLMOutput, (...) 32 TFTokenClassifierOutput, 33 ) ---> 34 from ...modeling_tf_utils import ( 35 TFMaskedLanguageModelingLoss, 36 TFModelInputType, 37 TFMultipleChoiceLoss, 38 TFPreTrainedModel, 39 TFQuestionAnsweringLoss, 40 TFSequenceClassificationLoss, 41 TFTokenClassificationLoss, 42 get_initializer, 43 keras_serializable, 44 unpack_inputs, 45 ) 46 from ...tf_utils import shape_list, stable_softmax File ~/anaconda3/lib/python3.10/site-packages/transformers/modeling_tf_utils.py:39 38 from huggingface_hub import Repository, list_repo_files ---> 39 from keras.saving.hdf5_format import save_attributes_to_hdf5_group 40 from transformers.utils.hub import convert_file_size_to_int, get_checkpoint_shard_files ModuleNotFoundError: No module named 'keras.saving.hdf5_format' The above exception was the direct cause of the following exception: RuntimeError Traceback (most recent call last) Cell In[21], line 1 ----> 1 from transformers import TFDistilBertForSequenceClassification, TFTrainer, TFTrainingArguments 3 training_args = TFTrainingArguments( 4 output_dir='./results', # output directory 5 num_train_epochs=2, # total number of training epochs (...) 11 logging_steps=10, 12 ) File :1075, in _handle_fromlist(module, fromlist, import_, recursive) File ~/anaconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1067, in _LazyModule.__getattr__(self, name) 1065 elif name in self._class_to_module.keys(): 1066 module = self._get_module(self._class_to_module[name]) -> 1067 value = getattr(module, name) 1068 else: 1069 raise AttributeError(f"module {self.__name__} has no attribute {name}") File ~/anaconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1066, in _LazyModule.__getattr__(self, name) 1064 value = self._get_module(name) 1065 elif name in self._class_to_module.keys(): -> 1066 module = self._get_module(self._class_to_module[name]) 1067 value = getattr(module, name) 1068 else: File ~/anaconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1078, in _LazyModule._get_module(self, module_name) 1076 return importlib.import_module("." + module_name, self.__name__) 1077 except Exception as e: -> 1078 raise RuntimeError( 1079 f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its" 1080 f" traceback): {e}" 1081 ) from e RuntimeError: Failed to import transformers.models.distilbert.modeling_tf_distilbert because of the following error (look up to see its traceback): No module named 'keras.saving.hdf5_format'

It throws, ValueError: Dimensions X and X are not compatible Where X is a random number

Sir can u share your cvs data file with us ?

TFtrainer is depracted now so refer to the official hugging face doc

100% accuracy possible ?

I am getting error as "FileNotFoundError: [Errno 2] No such file or directory: 'SMSSpamCollection'"

Let’s say I taught you four algorithm (A, B, C, D) for classification. I have collected one new dataset which contains 1000 observations with 11 features. We hired a team of experts to label the data class into either X or Y, they took 15 days to label 1000 entries. In next one month we‘ll also collect 1000 more observation. What strategy we need to follow to speed up our work. How can one answer this question?

2021 💀

I am getting below prediction output when I try the above on a custom multi class classification dataset. Does anyone have an idea, why this is happening, or how I can correct it? @krishnaik06 PredictionOutput(predictions=array([[nan, nan], [nan, nan], [nan, nan], ..., [nan, nan], [nan, nan], [nan, nan]], dtype=float32), label_ids=array([0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,

It asks me : "wandb: Paste an API key from your profile and hit enter, or press ctrl+c to quit: ··········" while running code - with training_args.strategy.scope(): Then I entered wandb key, I got this error : TypeError: '>' not supported between instances of 'NoneType' and 'int'

OperatorNotAllowedInGraphError Traceback (most recent call last) in 12 ) 13 ---> 14 trainer.train() 15 why i face error on trainer.train()