73 - Image Segmentation using U-Net - Part1 (What is U-net?)

Рет қаралды 360,945

Күн бұрын

Many deep learning architectures have been proposed to solve various image processing challenges. SOme of the well known architectures include LeNet, ALexNet, VGG, and Inception. U-net is a relatively new architecture proposed by Ronneberger et al. for semantic image segmentation. This video explains the U-Net architecture; a good understanding is essential before coding.
Link to the original U-Net paper: arxiv.org/abs/1505.04597
The code from this video is available at: github.com/bnsreenu/python_fo...

Пікірлер: 193

@burakkahveci4123 4 жыл бұрын

Thank you for the video. I think the best video for basic levels / intermediate levels.

@iamadarshmohanty 2 жыл бұрын

the best explanation I found on the internet. Thank you

@shafagh_projects 10 ай бұрын

I am speechless. your tutorials are beyond the amazing. thank you so much for all you have done!

@DigitalSreeni 10 ай бұрын

Glad you like them!

@Rocky-xb3vc 3 жыл бұрын

This is the first video I'm watching on this channel, and I need to say huge THANK YOU. You helped me connect so many dots that were all over the place in understanding this. Amazing.

@DigitalSreeni 3 жыл бұрын

Thank you very much for your kind feedback. I hope you’ll watch other videos on my channel and find them useful too.

@Rocky-xb3vc 3 жыл бұрын

@@DigitalSreeni Of course, I've already watched the full course and the next thing is time series forecasting. Thanks for your reply and everything you do!

@bicyclingmartian1873 4 жыл бұрын

Web application that uses uNet model: backdrop.vercel.app/ Source: github.com/Prottoy2938/backdrop

@brunospfc8511 2 жыл бұрын

Thanks Professor, there's so much knowledge in you channel, i'll need months to go through as it seems it's right in the deep learning area i want to focus, as an Computer Engineering going throught Veterinary course, blood sample analysis may be my final project, thanks from Brazil

@DigitalSreeni 2 жыл бұрын

I am sure you'll benefit from my tutorials if your goal is to analyze images by writing code in python.

@zeeshankhanyousafzai5229 2 жыл бұрын

I can not express my wishes for you in the words. You are more than the best. Thank you so much.

@DigitalSreeni 2 жыл бұрын

You are most welcome

@andresbergsneider6644 3 жыл бұрын

Thanks for sharing! Very well presented and super informative. Saving this video

@HenrikSahlinPettersen 2 жыл бұрын

For a tutorial on how to do deep learning based segmentation without the need to write any code using only open-source free software, we have recently published an arXiv preprint of this pipeline with a tutorial video here: kzfaq.info/get/bejne/b8qEmbio07Kaqo0.html (especially suited for histopathological whole slide images).

@user-gy8km4km6y 8 ай бұрын

thank you, professor, helps a lot in my understanding of deep learning.

@azamatjonmalikov9553 2 жыл бұрын

Amazing content as usual, well done :)

@boy1190 3 жыл бұрын

I wish youtube give us an option of liking video after every minute, this idea came in my mind for the first time in this video, I really want to give this video a like on every small bit of concept. Because it is explained so well. Respect Sir.

@Vibertex 3 жыл бұрын

Great Video! Really helped me understand U-Nets for my own use!

@DigitalSreeni 3 жыл бұрын

Great to hear!

@sarahs.3395 4 жыл бұрын

Good explanation, thank you.

@tonihullzer1611 2 жыл бұрын

First of all thx for your work here on KZfaq, when I'm done with your series I will definitely support you. One question here: I thought that in the upward path you do add the upsampled features and the corresponding ones from the contracting path, but in your code you have concat?

@MrAmgadHasan Жыл бұрын

He's concatenating and then uses a convolution layer. This has a similar effect to adding since the convolution operation adds the results after multiplication

@tamerius1 3 жыл бұрын

Why does the feature space and thus depth increase as we go down? Is this a design choice or a consequence? It's confusing for me that each first convolutional operation increases the depth and the second one which seems identical, does not.

@hanfeng32 4 жыл бұрын

thank you, this video is the best

@mincasurong Ай бұрын

Thanks for your amazing presentation!

@varungoel185 3 жыл бұрын

Nice video, thanks! One question - this architecture is for semantic segmentation right? How would the final layer (or layers) differ for the instance segmentation, wherein the output would be bounding boxes or co-ordinates of the instances?

@DigitalSreeni 3 жыл бұрын

Instance segmentation requires different architecture, you cannot swap the final layer to convert them from one to another application. I only wish life were that easy!!!

@icomment4692 3 жыл бұрын

What implication do the cross-links have for backpropagation in the U-net architecture?

@kebabsharif9627 2 жыл бұрын

Can you make a video in which your code detect the orientation of page from a photography of the page , suppose to the page is up-side down or 90° let /right rotated.

@victorcahui732 3 жыл бұрын

Thank you for your explanation.

@BareqRaad 3 жыл бұрын

Great demonstration thank you so much

@lazotteliquide 6 ай бұрын

Incredible that someone as dedicated as you gave accss to such great knowledge. Thanks you, you help create better sciences

@VLM234 3 жыл бұрын

Great explanation....Please keep on posting such high-value videos..... If we have less data, then we should go for Transfer learning or Machine Learning approach??

@MrChudhi Жыл бұрын

Hi, Sreeni, Nice explanation and I managed to clear my doubts. Thanks. Do you have any videos on image segmentation with pertained models.

@Tomerkad 7 ай бұрын

thank you. can you please explain what does it mean to add C4 to U6 in the first Upsample step?

@ramanjaneyuluthanniru1428 4 жыл бұрын

Well explained....sreeni you have amazing teaching skills...your explanation pretty good. i watched more and more videos in youtube...you also one of the best person thanks for sharing information

@DigitalSreeni 4 жыл бұрын

Thank you so much 🙂

@carolinchensita Жыл бұрын

Thank you very much for this explanation. I have one question, could I use this same method on an RGB image? Or does it have to be grayscale? Thanks!

@rohanaggarwal8718 7 ай бұрын

This is a late reply but yes, you have to expand your thinking... You can't assume just because someone made a tutorial this is what i have to do. Ask yourself these questions instead of trying to get help, What is a grayscale image? (1 is white, 0 is black, in between is gray) Can I apply this concept to RGB? (Three color channels, each same principle) How does my code change, (Input shoudl be three, maybe I need to flatten differently), etc. Good luck learning!

@zeeshanahmed3997 4 жыл бұрын

hello! I want ask something, can I train my unet model with the input training images having only single channel? like (img_height, img_width, 1) or (img_height, img_width) ?

@DigitalSreeni 4 жыл бұрын

Yes. Please watch my other videos on U-net. Every network expects certain dimensions and you can reshape your arrays to fit those dimensions. For example if you have grey images with dimensions (x, y, 1) and if the network takes 3 channels then just copy the image 2 more times to convert to (x, y, 3).

@joshizic6917 Жыл бұрын

Hi sir i was wondering if you could help to train my model i am trying to create a dataset where only the element of interest is visible and the rest is blacked out with transparent background , will this be great or i should create a binary mask by coloring the element of interest in white and keeping the background white

@matthewchung74 4 жыл бұрын

Thank you for this very helpful video. In the unet diagram, there are 3 output features, but your implementation only has one. I'm confused as to why?

@julianwittmann7302 3 жыл бұрын

As im just starting to dig into this field im not quite sure but my suggestion would be that the output has to be a segmented image. Segmented images have value 1 for the segmented part and value 0 for the remaining non segmented part of the picture. Usually when using segmentation grey values are considered. And for grey values only one channel is needed.

@shanisssss5906 4 жыл бұрын

Fantastic video!

@vikaskarade5585 4 жыл бұрын

Amazing Lecture. You can also create one on UNET++ and attention UNET. I was looking for these topics and I wish you had one on these topics... :)

@DigitalSreeni 4 жыл бұрын

Great suggestion!

@pratheeeeeesh4839 4 жыл бұрын

classy explanation!

@-arabsoccer1553 4 жыл бұрын

Thanks for your video,but i have question regarding the U-net and i hope that you can answer me from my understanding that the u-net is ended by image of the same input size ?but how we can predict the class of each pixel. i understand classification problem that it the last convolution is following by flatting and fully-connected layer so the number of n-classes as outputs ,but i don't understand how we get the result in segmentation

@DigitalSreeni 4 жыл бұрын

The convolution pooling operations (down sampling) understands the 'what' information in the image but has no information on the 'where' aspect which is required for semantic segmentation (pixel level). In order to get the 'where' information Unet uses upsampling (decoder), converting low resolution to high resolution. Please read the original paper for more information: arxiv.org/abs/1505.04597

@saifeddinebarkia7186 2 жыл бұрын

Thanks for the video,so is it transposed convolution or up-sampling for the expansive path because they are 2 different things.

@DigitalSreeni 2 жыл бұрын

It can be either. Please watch the following video if interested in learning about the differences between the two. But, you can use either as the idea is to get back to the large resolution image from a smaller size. kzfaq.info/get/bejne/nLOnp8xn36zHiY0.html

@siddharthmagadum16 2 жыл бұрын

5:12 . which architecture would be good for cassava leaf disease detection dataset?

@ahmedhafez3758 3 жыл бұрын

I want to make a 3D medical image segmentation , can you tell me how to start, I want the input to be .obj file and the output to be either .dcm files ( for each segment ) or .obj files

@Irfankhan-jt9ug 3 жыл бұрын

Great work......which tool creates Image masks?

@adityagoel237 2 жыл бұрын

14:25 In upsampling (before adding C4), why the 8*8*256 got transformed to 16*16*128 ? Why not 16*16*256 ?

@DAYYAN294 Ай бұрын

Great job by you sir salute to u❤

@ioannisgkan8930 2 жыл бұрын

Great explanation SIR You made us simple

@DigitalSreeni 2 жыл бұрын

Glad to hear that

@tonix1993 2 жыл бұрын

Very helpful video thank you!

@DigitalSreeni 2 жыл бұрын

Glad it was helpful!

@anishjain3663 3 жыл бұрын

Sir i am doing image segmentation with coco like dataset sir already see yours tutorials but still not able to implement

@TheedonCritic 2 жыл бұрын

Awesome! I'm trying to use GAN for augmenting my images and masks which I will use as input to my semantic segmentation models, but I can't find any tutorials online. Most of them are for classification datasets, any advice, please?

@Julian-ri9od 2 жыл бұрын

Is there a reason why always two convolutions are applied after the max pooling step? Is it a convention to use always two?

@DigitalSreeni 2 жыл бұрын

No reason. It may appear that 2 convolutions are added after maxpool on some architectures but that is not the general case.

@josemiguelc.tasayco4028 3 жыл бұрын

Very well !!! more videos please

@maciejkolodziejczyk4136 3 жыл бұрын

Many thanks, well done !

@DigitalSreeni 3 жыл бұрын

Many thanks!

@NH-gl8do 3 жыл бұрын

Very excellent explanation

@DigitalSreeni 3 жыл бұрын

Glad it was helpful!

@mqfk3151985 3 жыл бұрын

As usual! Amazing tutorial. I just want to confirm, in the training phase, all images have to be of the same shape (width, height and depth), right? what if my training data varies in shape? Do I need to resize the images? Also, I Will be really thankful if you can give a tutorial on Mask RCNN. It's also a very good algorithm that can be used for semantic segmentation. Thanks a lot for your time.

@manishsharma2211 3 жыл бұрын

Yes. Always apply transformation on image ( like resizing and rotation etc)

@mqfk3151985 3 жыл бұрын

I see, thanks for the reply. Image rotation will be performed for data augmentation. but regarding the image resizing, I think it's a requirement by the algorithm.

@manishsharma2211 3 жыл бұрын

@@mqfk3151985 Yes , there is never that you might find images of all same size. Unless you go for normal competation So better resize :)

@DigitalSreeni 3 жыл бұрын

You will represent your data as numpy array so you need all images to be of same size. Yes, it is customary to resize images to a predefined shape in machine learning. I will consider making Mask-RCNN videos.

@chitti1120 3 жыл бұрын

can someone tell me and give examples of why the u-net architecture uses the 'copy and crop' for every block?

@doraadventurer9933 3 жыл бұрын

thank you for your sharing, however, do you have the training part?

@DigitalSreeni 3 жыл бұрын

Please keep watching videos on this playlist, I have training and segmentation part covered.

@CristhianSanchez 3 жыл бұрын

Great explanation!

@DigitalSreeni 3 жыл бұрын

Glad you think so!

@BiswajitJena_chandu 4 жыл бұрын

Sir, please do a video for segmentation of BRATS dataset

@bhavanigarrepally4164 2 жыл бұрын

Can you give the implementation for unsupervised semantic segmentation also

@poopenfarten4222 Жыл бұрын

what are the numbers above the layers, for eg in the first layer 16 is written above it, what does it signify could someone please explain

@chouchou2445 3 жыл бұрын

Thank you again Would you please tell me, is it possible to use data augmentation befor semantic segmentation an how to apply same function on both image and mask

@Shadow-pn2us 4 жыл бұрын

still confused with the concatnation operation how it works, such as adding 16x16x128 featuremap with upsampled 8x8x256, the dimension is different

@DigitalSreeni 4 жыл бұрын

You’ll be concatenating data with same dimensions, not different dimensions. Please have a second look at the graphic describing the architecture, the two layers fused together showing dimensions are being concatenating to form a dataset with combined dimension.

@alessioandreoli2145 4 жыл бұрын

Hi!which is the best segmentation technique I can use in python for cells image counting/object detection/size definition?

@DigitalSreeni 4 жыл бұрын

The best method is always traditional approaches of using histogram for thresholding and then some operators like open/close to clean up. If that is not possible then the next best option is to use traditional machine learning (extract features and then Random Forest or SVM). I covered that topic on my channel. FInally, if you have the luxury of 1000s of labeled images then use deep learning.

@alessioandreoli2145 4 жыл бұрын

@@DigitalSreeni , please let me one more question. My purpose is to avoid manual settings to use macros or python over big amount of images taken at cells on a big microscale range. Any suggestions there? Have you any reference for deep learning?

4 жыл бұрын

Thanks for video

@efremyohannes2334 3 жыл бұрын

Thank you sir, very nice video.

@DigitalSreeni 3 жыл бұрын

Most welcome

@temurochilov 2 жыл бұрын

Thank you very informative tutorial

@DigitalSreeni 2 жыл бұрын

Glad it was helpful!

@ApPillon Жыл бұрын

Thanks bro. Cheers!

@NeverTrustTheMarmot 2 жыл бұрын

Pick up line for data scientists: Why is U-Net architecture so beautiful? Cause it looks like U

@mohammadkarami8984 4 жыл бұрын

Thanks a lot for your video

@DigitalSreeni 4 жыл бұрын

You are most welcome

@lorizoli 2 жыл бұрын

Great video!

@DigitalSreeni 2 жыл бұрын

Glad you enjoyed it

@mohamedelbshier2818 Жыл бұрын

Thank you and Respect Sir

@DigitalSreeni Жыл бұрын

You are welcome.

@snehalwagh2283 2 жыл бұрын

Question: What happens if it is 128X128X1 ? will it still become 128X128X16 ?

@NS-te8jx 2 жыл бұрын

do you have slides for all these videos?

@Abhisingh-cl9xm 4 жыл бұрын

Best resource

@haythammagdi3956 8 ай бұрын

Hi every one. It is really amazing video on U-Net. But waht about U2-Net? is it better?

@4MyStudents 2 жыл бұрын

basically, ReLU is used to prevent overfilling to maintain non-linearity

@nickpgr10 3 жыл бұрын

@14.11.. can anyone please explain how size changes from 8*8*256 to 16*16*128 due to up sampling??. why number of channels get reduced in this step??

@zhenxingzhang6429 3 жыл бұрын

If you checkout the part2 of this video, you can see that it uses Conv2DTranspose (transpose convolutions) for upsampling instead of just simply UpSampling2D (repeat the value to match the desired dimosions), because the filter number is set to 128, so we end up with 8*8*256 -> 16*16*128. Check this for more details: www.jeremyjordan.me/semantic-segmentation/#upsampling

@ExV6120 4 жыл бұрын

I still don't get it, what exactly is the 16, 32, 64, 128, 256 that being called features in the next two layers each?

@DigitalSreeni 4 жыл бұрын

Think of it as applying 16 different digital filters and then 32 and the 64 and so on.... Therefore, if you take a single image of size 256x256 and apply 16 different filters on it you will end up with 16 responses from this single image --> 256x256x16 data points.

@andresbergsneider6644 3 жыл бұрын

@@DigitalSreeni What is the design principle behind this filters, any rules of thumb? Are they generated at random? Or are this manually configured? Thanks again for sharing this video!

@mstozdag 4 жыл бұрын

Hello, great content! Where is the code for U-Net? Can u post the link here pls?

@DigitalSreeni 4 жыл бұрын

github.com/bnsreenu/python_for_microscopists

@isaaciwediba3380 2 күн бұрын

You are doing a great work, I have learnt a lot from you. could you please treat segmentation using DeepLab? thank you.

@nourhanelsayedelaraby4271 2 жыл бұрын

first of all thank u for the great explanation and wanted to ask u about the slides if they are available

@DigitalSreeni 2 жыл бұрын

Sorry, I wasn't very planned with my presentation slides so unfortunately I cannot share them. Also, I often use images and content from Google searching that come with copyright. I cannot legally distribute them.

@RAZZKIRAN 2 жыл бұрын

input size for U-nET?

@talha_anwar 4 жыл бұрын

Thanks first of all. Can you provide the image you have used, the architecture image?

@DigitalSreeni 4 жыл бұрын

You can search for U-net on Google. I did the same and created my own, to make sure I do not infringe on copyright.

@sourabhsingh4895 3 жыл бұрын

@@DigitalSreeni sir you are great sir it would be a great help if you could upload a video on semantic segmentation using double-UNET model

@chouchou2445 3 жыл бұрын

thank you this is how you know what are y doing :)

@DigitalSreeni 3 жыл бұрын

You're welcome!

@ariouathanane Жыл бұрын

Hello, i have a rgb masks, it's possible to do the image segmentation? thanks in advance

@DigitalSreeni Жыл бұрын

Yes. I have done that here. kzfaq.info/get/bejne/oNyKoJuDqqyWe40.html

@kethusnehalatha6091 3 жыл бұрын

For better results what changes we have to do in the u net sir ???

@DigitalSreeni 3 жыл бұрын

Many things. For example you can try replacing generic encoder (down sampling) part with something sophisticated like efficientnet.

@pearlmarysamuel4809 3 жыл бұрын

How much memory does the original unet require?

@DigitalSreeni 3 жыл бұрын

Not a simple answer. Here is some good reading material on this topic. imatge-upc.github.io/telecombcn-2016-dlcv/slides/D2L1-memory.pdf

@shreearmygirl9878 2 жыл бұрын

Hello sir, plcan u provide the links of videos for creating our own dataset from scratch fro satellite images, pl sir.. its very important.I hope you will...

@DigitalSreeni 2 жыл бұрын

You just need to annotate your images using any of the image annotation tools out there. I use www.apeer.com as that is what our team does at work.

@mimo-wx9mc 4 жыл бұрын

why the first parameters don't work very well and how we can determine the best parameters

@DigitalSreeni 4 жыл бұрын

No sure what you mean by parameters. If you are asking about hyper parameters that go into defining your network then it is not an easy answer. People are still researching the effect of parameters for various applications.

@codebeings 3 жыл бұрын

13:54 Do check, second last layer in the decoder side have wrong connections !

@kunalsuri8316 3 жыл бұрын

How is it wrong?

@codebeings 3 жыл бұрын

@@kunalsuri8316 In the second last layer of decoder (corresponding to P1), its input to the last layer of decoder is incorrect. Just check the original paper, one can easily notice it.

@MadharapuKavyaP 2 жыл бұрын

Hello sir, can u please make a video on brain tumor segmentation using u net architecture integrated with correlation model and fusion mechanism.

@amaniayadi9591 3 жыл бұрын

so useful thnks :*

@nailashah6918 3 жыл бұрын

very good lecture just one thing I am unable to understand about feature space or dimension?plz reply with answer thanks

@DigitalSreeni 3 жыл бұрын

Not sure where your confusion is.... I am referring to the filtered results (after convolutional filtering) as feature space. This is where you will have multiple responses for every input image and these responses contain the information about features in the image.

@nailashah6918 3 жыл бұрын

I wanted to ask about feature space that was 64 in start then 128 in 2nd block of unet 64 means 64 output filtered results? is that true? or we can say 64 filters were applied, then 128 filters and so on ...?

@ahtishamulhaq1415 2 жыл бұрын

I can't Find Code Please Tell the name of folder

@deepMOOC 4 жыл бұрын

Thank you,but how can I get the code

@DigitalSreeni 4 жыл бұрын

You can get the code from my GitHub page. The link is provided under my channel description.

@leo46728 2 жыл бұрын

17:56 Does the model need to be trained after compiling?

@DigitalSreeni 2 жыл бұрын

Compiling just defines the model, you need to train the model on real data to update the weights and customize it for a specific job to be done, for example identify cats and dogs.

@leo46728 2 жыл бұрын

@@DigitalSreeni ok thanks

@govtjobs7063 Жыл бұрын

Sir is this unet architecture for multiclass segmentation or binary segmentation?? Kindly response

@DigitalSreeni Жыл бұрын

This is binary. I got many other videos on multiclass.

@govtjobs7063 Жыл бұрын

@@DigitalSreeni ok sir ...Thank you for your response...sir i have one more question ...when we are combining t2,flair and t1ce...do we call that combined image a single channel image or 3 channel image...please sir reply

@anishjain3663 3 жыл бұрын

sir how to use this 3d images and what is 3d image can you please can you make a video on that

@DigitalSreeni 3 жыл бұрын

I will try to do 3D image processing some day.

@aishstha6669 Жыл бұрын

@@DigitalSreeni do you va e vid on 3D ?