128 - Malarial cell classification using CNN and data augmentation

Рет қаралды 16,465

Күн бұрын

Code generated in the video can be downloaded from here: github.com/bnsreenu/python_fo...
Dataset link: www.kaggle.com/datasets/iarun...

Пікірлер: 52

@ocherki 3 жыл бұрын

"we have this sad looking cell, because it's affected with malaria" - ahah, he does not only explain things clearly but also has a good sense of humor :)

@roodypf 4 жыл бұрын

I like the idea that you can directly put a set of augmented images to the fit instead of augmenting and saving the images separately. Really informative, thanks!

@DigitalSreeni 4 жыл бұрын

I'm glad you found the video informative.

@ztabatabaei2612 3 жыл бұрын

You are the best, you solve a lot of problems for me! Thank you very much

@proterotype 2 жыл бұрын

Thanks for making this clear for me. Keep 'em comin

@DigitalSreeni 2 жыл бұрын

Glad to help!

@marydayanaa9130 4 жыл бұрын

Really Informative. The way you explain each and every line of code is amazing. Thanks a lot.. please add some videos on IDRID dataset for diabetic retinopathy detection.

@DigitalSreeni 4 жыл бұрын

Sorry, I'm not familiar with IDRID. But I hope the examples I show definitely apply to image analysis in other fields.

@muhammadroshan7315 4 жыл бұрын

How can we do augmentation for semantic segmentation tasks? like I have RGB image and I want to apply same transformation to the mask (each pixel consists of class labels like 0 for forest, 1 for water, etc. ) And also, I have 1000 images, how many images is it safe for me to generate with data augmentation?

@muhammadroshan7315 4 жыл бұрын

What if we donot have any background class in our segmentation mask? should we still apply reflection to the images and its mask? or can it cause any problems?

@MridulBanikcse 3 жыл бұрын

Thanks for the tutorial. Is it possible to use GridsearchCv() with image generator so that we can use different learning rate or optimizer in one go?

@neginpirannanekaran1236 2 жыл бұрын

Thanks for the simple and effective video! I have a question about reading other types of images like nii or dicom. Is there any way to define flow_from_directory for such formats? If not, is it wise and safe to convert all dicom images to JPEG formats and then use flow_from_directory?

@muhammadalam2498 4 жыл бұрын

Brilliant 10/10

@DigitalSreeni 4 жыл бұрын

Thanks.

@applejuice5785 3 жыл бұрын

since the images that get augemented are not getting used fr the model can I give the same folder to use for validation?

@ashilshah3376 11 ай бұрын

Thank you so muchh, it helped :)

@DigitalSreeni 10 ай бұрын

You're welcome!

@sivanit3150 2 жыл бұрын

I really appreciate your efforts thanks for these videos. I have one query that is when i use binary cross entropy my model predicts only one class. I looked up for class weights yet I'm facing same issue. Hope to get a solution for this issue.

@ZettyIlham 2 жыл бұрын

why does augmented images of both classes are mixed in the same augmented folder? Is it okay not to have them separated according to the corresponding folders as the original ones?

@aditiarora2128 2 жыл бұрын

sir can we apply imageaugmentation on single folder containg thousands of images without seperate training and testing folder?

@ronylpatil 2 жыл бұрын

Sir what about the labels if we have all tha images in one single folder???

@EkaterinaGolubeva-pr9ih Жыл бұрын

In the Multiple images case, when we apply the ImageDataGenerator, where do we indicate how many augmented images we want to generate ?

@DigitalSreeni Жыл бұрын

Isn't it just the batch size? The generator supplies augmented images for each batch.

@lakpatamang2866 3 жыл бұрын

how many images are generated per class with this augmentation? you did not explain this

@geniusviewtelugu1440 11 ай бұрын

I think there is no change in the count of the images

@daviddamifogodaramola3596 3 жыл бұрын

I am quite Confused as to how many images will be generated in the train_generator function, we didn't state it

@DigitalSreeni 3 жыл бұрын

If you have 1600 images in your dataset, if you set batch size as 16, the generator will generate 16 images in each iteration of the training. Iteration is usually referred to as steps per epoch which we normally set as the total number of samples // batch_size. So in our example, each epoch of training will see 1600/16 which is 100 iterations and in each iteration it sees 16 images. Remember that all images are generated based on the original images, your original images will not be directly fed to the training algorithm.

@pujabharti344 3 жыл бұрын

Hi Sreeni. I really feel all your videos are informative. I am facing one issue. When doing augmentation for both (images and Labels) we need to zip(train_generator, label_generator) and use Model.fit_generator(). It gives error: AttributeError: 'zip' object has no attribute 'shape'. Please suggest how I can get over this. Using UNET model.

@DigitalSreeni 3 жыл бұрын

Use an easier way to augment images and masks, I recommend Albumentations library. I will record a video soon but if you cannot wait a few weeks for the video just look up Albumentations documentation.

@shashireddy7371 3 жыл бұрын

Thanks Sreeni for sharng your knowledge . Could you please help me how to get label s of images after augmentation. Basically , here we are directly reading images inside ImageDataGenerator function. How we extract label for train and test image after augmentation so that we can use labels for model evaluations. Note : I have understood how we split train and test data in case non augmented data like you did in your video 71 Malarial classification

@marcusbranch2100 3 жыл бұрын

There is a way to apply different rotations simultaneously??

@DigitalSreeni 3 жыл бұрын

Not sure what you mean. If you want multiple rotations in one augmentation step it doesn't make sense to me. You are performing random rotation so why would it make any difference if you make 2 random rotations instead of one?

@ztabatabaei2612 3 жыл бұрын

For this tutorial, I couldn't find cell-image and cell-validation! please upload them

@ravimehta714 2 жыл бұрын

Sir, I am getting such type of error:; ValueError: No gradients provided for any variable: ['conv2d_21/kernel:0', 'conv2d_21/bias:0', 'conv2d_22/kernel:0',,

@zainabkhan5859 3 жыл бұрын

Sir please make a video on time series augmentation techniques

@hihi-ph8hj 4 ай бұрын

Thank you very much for this very useful tutorial i really get very much information from you. I do have questions about the original dataset for example i do have 2 cats pic then i generater 4 more,from my understanding u said we use only the generated one (the 4 pic) ,what about the original dataset ??? Do we need to train the model on it separated ?? Or can i use it for test for exampl or for validation ???. My last question is,here we augment each class with same number of images,can i augment each class with different batch size so i can create more balanced dataset ?? Is this okay and a good practice??? I hope u really answer my questions. And finally thank you very much,

@DigitalSreeni 4 ай бұрын

Thank you for your positive feedback and questions! The purpose of data augmentation is to introduce variability and help the model generalize better. But if you want you can include the original dataset along with the augmented images during training. Yes, you can certainly use different batch sizes for augmenting each class. This practice can help balance the dataset and prevent the model from being biased towards any specific class during training.

@gurinderjeetkaur7291 4 жыл бұрын

please create some videos on MRI BraTS dataset for brain tumor segmentation also.

@mayukhbanerjee8877 2 жыл бұрын

Hi how can I generate abnormal rbc

@anandsrivastava5951 Жыл бұрын

Hello Digital Sreeni Greetings and sorry about asking. Query out of box Just incase if you have Segnet (paper of vijay badrinayan...) and fully convolutional network for multiclass; can you please share/ explain on your channel That will be a great help!! Thanks in advance!

@beingraja9586 3 жыл бұрын

What is steps per epoch can you please explain and waiting for different types of gans and lstm encoder decoder

@DigitalSreeni 3 жыл бұрын

I explained this concept to another user's comment. So I am copying the response, hope it helps. If you have 1600 images in your dataset, if you set batch size as 16, the generator will generate 16 images in each iteration of the training. Iteration is usually referred to as steps per epoch which we normally set as the total number of samples // batch_size. So in our example, each epoch of training will see 1600/16 which is 100 iterations and in each iteration it sees 16 images. Remember that all images are generated based on the original images, your original images will not be directly fed to the training algorithm.

@dibyakantaacharya4104 3 жыл бұрын

sir .. you have used validation_images ...what is this??

@DigitalSreeni 3 жыл бұрын

validation data is used to test accuracy of the model during and after training. You can hold out part of your data for validation using train_test_split or you can import other data and use it for validation/testing.

@merveozdas1193 2 жыл бұрын

Can you explain how you constitute .hdf5 file?

@DigitalSreeni 2 жыл бұрын

You can use a tool like HDF View to explore the hdf5 file. The file contains model information, model weights, and optimizer weights. The optimizer weights can be ignored as they are purely used during training to figure out how to update the model weights. I will try to record a short video on this topic. Thanks for asking.

@dibyakantaacharya4104 3 жыл бұрын

KeyError: 'Failed to format this callback filepath: "saved_models/weights-improvement-{epoch:02d}-{val_acc:.2f}.hdf5". Reason: \'val_acc\'' sir i m getting this error ...please tell me the solution

@DigitalSreeni 3 жыл бұрын

Change val_acc to val_accuracy and see if it works.

@dibyakantaacharya4104 3 жыл бұрын

@@DigitalSreeni it's working sir , thank u

@DigitalSreeni 3 жыл бұрын

According to the 2.3.0 Release Notes: "Metrics and losses are now reported under the exact name specified by the user (e.g. if you pass metrics=['acc'], your metric will be reported under the string "acc", not "accuracy", and inversely metrics=['accuracy'] will be reported under the string "accuracy"."