NEXT-GEN NEW IMG2IMG In Stable Diffusion! This Is TRULY INCREDIBLE!

Рет қаралды 184,611

Жыл бұрын

ControlNet is a brand new neural network structure that allows, via the use of different special models, to create image maps from any images and using these informations to transfer data from one image to another, making it a really powerful option in the image-to-image tab inside Stable Diffusion! This gives you even more power and more control over the final result than before. So in this video, I will show you how to install the extension and how to use the differents models to get the best results possible!
Did you manage to install the extension? Let me know in the comments!
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
SOCIAL MEDIA LINKS!
✨ Support my work on Patreon: / aitrepreneur
⚔️ Join the Discord server: bit.ly/aitdiscord
🧠 My Second Channel THE MAKER LAIR: bit.ly/themakerlair
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Runpod: bit.ly/runpodAi
Miro board: miro.com/app/board/uXjVPnB9L2...
Extension URL: github.com/Mikubill/sd-webui-...
The models: huggingface.co/lllyasviel/Con...
Special thanks to Royal Emperor:
- BSM
Thank you so much for your support on Patreon! You are truly a glory to behold! Your generosity is immense, and it means the world to me. Thank you for helping me keep the lights on and the content flowing. Thank you very much!
#stablediffusion #controlnet #stablediffusiontutorial
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
WATCH MY MOST POPULAR VIDEOS:
RECOMMENDED WATCHING - My "Stable Diffusion" Playlist:
►► bit.ly/stablediffusion
RECOMMENDED WATCHING - My "Tutorial" Playlist:
►► bit.ly/TuTPlaylist
Disclosure: Bear in mind that some of the links in this post are affiliate links and if you go through them to make a purchase I will earn a commission. Keep in mind that I link these companies and their products because of their quality and not because of the commission I receive from your purchases. The decision is yours, and whether or not you decide to buy something is completely up to you.

Пікірлер: 346

@Rafael64_ Жыл бұрын

Not even week goes by and there's image to image significant enhancement. What a time to be alive!

@NC17z Жыл бұрын

This extension is amazing! I'm having an absolute blast with it. It is solving so many of my problems with matching the look of realism photos with what I'm feeding it for an image and with my prompt. Thank you so much for what you do. You've been my first go to on KZfaq for weeks!

@peterbelanger4094 Жыл бұрын

I'm having fun with the sketch pre processor. Runs fine on my gpu (1060GTX 6GB), not even using the --lowvram option.

@Alex-dr6or Жыл бұрын

This is exactly what I need. Last night I was having fun with the blend option in Midjourney and wished SD has something similar. This video came at the perfect time

@Aitrepreneur Жыл бұрын

HELLO HUMANS! Thank you for watching & do NOT forget to LIKE and SUBSCRIBE For More Ai Updates. Thx

@MrArrmageddon Жыл бұрын

Do we know if these are safe? I don't even know how to scan pickle anymore. I have avoided them for months. Amazing video by the way. Thank you.

@joachim595 Жыл бұрын

“Type cimdy”

@peterbelanger4094 Жыл бұрын

👍👍👍👍👍👍👍 Great extension! paused the video, got it all downloaded and installed before I finished the video. Runs fine on my 1060GTX 6GB, even without the lowvram option. Actually 10x faster without it. only the xformers option is needed.

@MrArrmageddon Жыл бұрын

@@peterbelanger4094 If you can Peter? I have a RTX 4080 16GB. I've never used any Xformers. Should I look into them? And if so what purpose they server? lol If you can't explain that is fine.

@zwojack7285 Жыл бұрын

what SD version are you using? I only have..1.5 I think and the extension only appears in txt2img, not in img2img

@coda514 Жыл бұрын

Saw info about this on Reddit, I knew you would put out a how-to video so I waited to install. Glad I did, you did not disappoint. Sincerely, your loyal subject.

@StiffPvtParts Жыл бұрын

Yet again, you have manged to blow my mind! Thank you for showing this new amazing functionality! Feels like these tools are getting more insane every single day.

@Unnaymed Жыл бұрын

It's epic, power of stable diffusion is upgraded ! ❤️

@o.b.1904 Жыл бұрын

The pose one looks great, you can pose a character in a 3d program and use it as a base.

@mactheo2574 Жыл бұрын

What if you use your own body to pose?

@vielschreiberz Жыл бұрын

Perhaps it will be useful with some simplified solutions like Daz 3d Or with poses library

@Amelia_PC Жыл бұрын

Yup. I've been using Daz to help me with comic book character poses for years and it took only some seconds to put a character in different poses (if a person says it takes more than that, that's because they're newcomers or have not much experience with 3D programs).

@pladselsker8340 Жыл бұрын

and yeah, the 3D software thing is actually a good idea if you implement a good inverse cinematic thingy on it. It can probably save time and be faster than a google search for simple (or really specific and complex) poses. I've been learning how to do that last night in blender, I'm almost done with the model, it's actually not too hard to make. You don't even have to render anything btw, you just have to take a screenshot of it when the angle and everything seems okay, and then paste that in the webui. Works like a charm.

@sonydee33 Жыл бұрын

Exactly

@ysy69 Жыл бұрын

downloading the models now and will try. thank you so much. this seems very powerful. In fact, I've been spending more time on img2img lately. even at the current state, it is fantastic... can't imagine the possibilities with this new extension

@gloxmusic74 Жыл бұрын

Installed it straight away and can honestly say I'm super impressed 👍👍👍

@DjDiversant Жыл бұрын

Installed it just couple of hours before a vid. Thx for a tut!

@SnowSultan Жыл бұрын

If this works as well as it appears to, this is both game-changing and life-changing for artists like myself that work in 3D but want more illustrated or toony results. I've waited 24 years to be able to make true 2D art with 3D methods. Even if it's not perfect yet, this gives me hope.

@muerrilla Жыл бұрын

I'm playing around with the scribble model and i'm absolutely blown away!

@Smokeywillz Жыл бұрын

topaz studio 2 was coming close but THIS is next level

@PriestessOfDada Жыл бұрын

I had the same thought. Makes me want to train my cc4 characters

@SnowSultan Жыл бұрын

@@PriestessOfDada I've had decent luck using untextured DAZ figures as ControlNet pose references, but I do not know what the results would be if you train a checkpoint or LoRA on a complete 3D character. If you can still get 2D or anime results from it...well, I'll have a lot training to do. ;)

@mrhellinga9440 Жыл бұрын

this is pure gold

@dreamzdziner8484 Жыл бұрын

Wow. So exciting. Thank you dear Overlord 💪🙏🏽

@SteveWarner Жыл бұрын

Top notch training! Thanks for this comprehensive overview! Looking forward to testing this out!

@kylehessling2679 Жыл бұрын

I've been wishing for this since day one of using SD! This is going to be so useful for generating versions of my graphic design work!

@toddzircher6168 Жыл бұрын

Thank you for the wonderful walk through on this new extension. I have a lot of 2d pose sheets/sketches from various artists and I can totally see using them with a controlnet.

@ristopaasivirta9770 Жыл бұрын

My biggest complaint about SD has been the lack of control. In order to make comic books and alike you need to be able to precisely control the pose of the characters. Gonna see how well this holds up. Thank you for the video!

@thebonuslvl7181 Жыл бұрын

magic... so much more to come thank you for keeping us on the leading edge..

@upicks Жыл бұрын

Simply amazing, thanks for the video! This makes img2img even better than I could have imagined. .

@yeahbutcanyouredacted3417 Жыл бұрын

Amazing tool- solves a lot of RNG for us to get closer to the designs we are looking for ty again Aitrepreneur for helping get my home studio going

@haidargzYT Жыл бұрын

Cool 😮 A.I community keep surprising us every day

@xnooknooknook Жыл бұрын

I really like txt2img but img2img has been where I've spent most of my time. Scribble mode looks amazing! I need it in my life.

@harambae117 Жыл бұрын

This looks like a lot of fun and really good for professional use. Thanks for sharing dear AI overlord

@Varchesis Жыл бұрын

This is insanely great! Thanks for sharing this info.

@purposefully.verbose Жыл бұрын

i saw people talking about this concept on several other channels, and all were like "i hope this comes out for auto1111", and i'm all "it is!" - and linked this video. hopefully you get more subs.

@cinemantics231 Жыл бұрын

This just keeps getting better and better! Thanks for putting this together. Is there any way to merge two different images? Like take the pose from one image and implement it in the style or background of another?

@pastuh Жыл бұрын

its called inpaint. just use photoshop plugin for this. paint over (or put image) on different layer and click inpaint

@cybermad64 Жыл бұрын

Thanks a lot for sharing your miro boards, those are normaly test that I would do myself to understand how the system works, you are saving us a lot of tech investigation time! :)

@GolpokothokRaktim Жыл бұрын

I recently experimented with blue willow and I'm really amazed to use it. Blue Willow recently launched V2 with a brand new model upgrade. Now I got better-quality images with more aesthetic outputs

@JohnVanderbeck Жыл бұрын

I'm droolling at the thought of using the pose model!

@EntendiEsaReferencia Жыл бұрын

I've been waiting por the 1.5 depth model and now it's here, and with a few friends 🤗🤗

@user-bm9oy4gx2l Жыл бұрын

Thanks again for good contents! Pose thing looks interesting 👀

@A_Train Жыл бұрын

Thanks for being on the bleeding edge of this and imparting your knowledge for artists like me. My question is, what is the best way to implement Stable Diffusion in Blender. I used a version like 3 months ago and now that seems so outdated.

@digitalkm 9 ай бұрын

Awesome, thank you!

@XaYaZaZa Жыл бұрын

My favorite youtuber 🧡

@rickardbengtsson Жыл бұрын

Great breakdown

@jivemuffin Жыл бұрын

Nice, comprehensive video -- and thanks for the Miro board in particular! Makes me think there's great potential for AI workflows in there. :)

@DOntTouCHmYPaNDa Жыл бұрын

Awesome thanks for sharing! Btw, if you click the down arrow just to the right of the letters LFS it just downloads the models without opening another tab. Small but useful tip :)

@IlRincreTeam Жыл бұрын

This is VERY impressive

@xd-vf1kx Жыл бұрын

so cool! I love ya!

@Irfarious Жыл бұрын

I love the way you say "down below"

@sebastianclarke2441 Жыл бұрын

Why have I only heard about this for the first time today!? Wow!!

@azmodel Жыл бұрын

Absolutely Crazy. thanks!

@phatbuihong4014 Жыл бұрын

Thank you so much.

@dovrob Жыл бұрын

thanks so much mate

@gohan4585 Жыл бұрын

Thank you sensei bro 🙏

@girasan Жыл бұрын

thank you so much 🙂

@h8f8 Жыл бұрын

Never knew you could type cmd on the top directory... thank you so much

@desu38 Жыл бұрын

Goddamn, the webui just keeps getting more and more powerful. 😯

@Einscrest Жыл бұрын

Thanks for another great vid! I'm more interested in the open pose model because then you won't need to prompt a pose too much. Seems like in the video it can retain some details like the clothes color so it also needs a prompt for it to change. Very interesting. Edit: Some more interesting things - It can accept characters provided the model knows them (Azur Lane characters for example) - Like from the video it does eat up vram fast, my Linux almost crashed one time lol.

@brokencreationlordmegatrol3037 Жыл бұрын

Ooooooo! Very useful

@s.foudehi1419 11 ай бұрын

this is truly nextlvl stuff. im glad i found this video. has anyone already tried to create a depthmap with controlnet and then using that to create a 3model in blender? there's some good tutorials on here as well, you might wanna check that out :)

@bryan98pa Жыл бұрын

Wooow, i like this new tool!!

@CrazyEditsCrazy Жыл бұрын

this is awesome

@flonixcorn Жыл бұрын

Very Nice!

@StrongzGame Жыл бұрын

i need this video on a flash drive for reference for ever

@mikishomeonyoutube2116 Жыл бұрын

This is TRULY INCREDIBLE!

@Fingle Жыл бұрын

NO WAY THIS IS INSANE

@AmirZaimMohdZaini Жыл бұрын

This feature finally able to make new image with exact style from original input picture.

@BenPhelps Жыл бұрын

Brilliant. Any tips to install on M1 Mac?

@oldaccountfornow1111 Жыл бұрын

Big Thanks

@leafdriving Жыл бұрын

Dear AIOverlord: Thank you as always for being ahead of the pack, and showing me something useful and amazing. In every example, your two images (input above and input below) are the same ~ What happens if they are different? (Naturally, I couldn't possibly just try it lol)

@SuperEpic-vb8nq Жыл бұрын

This is absolutely amazing, my only complaint is that it doesn’t seem to work with batch img2img. If that gets to working, then this could easily solve the issue with stable diffusion videos where details tend to be “sticky” due to the seed not shifting with the video. This could help stabilize it. Edit, after an update, it works with batch img2img and it does exactly what I wanted. What a time to be alive!

@Ich.kack.mir.in.dieHos Жыл бұрын

yoo do you make videos in stable? because i do and I m just interested in batchmode /animation. consistens characters and places. can we connect on insta ?

@KingZero69 Жыл бұрын

amazing

@kuromiLayfe Жыл бұрын

cannot wait for this to be an extension to txt2img also : prompt for a specific character and then add a scribble or pre process image to the script to get the described character in the pose you want.

@BlackDragonBE Жыл бұрын

It already has this.

@BlackDragonBE Жыл бұрын

@@ClanBez Open the txt2img tab with the extension this video explains installed. At the bottom of the tab you can use the ControlNet just like in img2img, including the scribble model. By providing a prompt and a scribble, you can generate images with lots of control. I suggest lowering the Weight to 0.25-0.5 to start with as you ca get some weird results depending on your drawing skills otherwise. Good luck.

@zwojack7285 Жыл бұрын

for some reason it only shows up in txt2img for me lmoa

@kallamamran Жыл бұрын

Finally! If you can say that for something that didn't take 24h 😃

@pladselsker8340 Жыл бұрын

This is THE extension everyone needed. I don't know if people realise, but this is a MEGA GAME CHANGER. Up untill now we ONLY had tools for changing the details of the image. Inpainting, pix2pix, img2img, loras, etc. All of this is JUST for the details, and can even make the control of composition even harder to achieve (especially for TIs and loras, they do affect it negatively quite a lot in general). This new extension now handles the composition of the images. This thing is literally filling a hole in the AI image making process. One other such hole that I hope someone or a team will learn how to fill now is to be able to make loras of stuff that don't yet exist, like OCs.

@lefourbe5596 Жыл бұрын

someone like me ? totally agree, i strive to make my bad CG model into glorious anime characters in blender, using stable diffusion as render engine. i already have Dreambooth and Lora ready to use but lack consistency on character feature to make a nice animation. i lack time sadly and i'm still an amateur, definitly interested to have a discord gathering ppl togeter to find the best settings for everyone

@pladselsker8340 Жыл бұрын

@@lefourbe5596 I think touhou AI project is one if the best servers for that. A lot of people just throw in information everyday, and some of them share their whole workflow if you ask. It's so hard to keep up with them haha.

@Rickbison Жыл бұрын

I finished my last short with the old img2img. Downloading all the models and lets see how this goes.

@nathancanbereached Жыл бұрын

I'd love to see a video/cartoon where you transition from one scene to another by increasing midas weight slowly. Would be a very trippy/dream like way to transition to a totally different place.

@nic-ori Жыл бұрын

Thanks.

@angelicafoster670 Жыл бұрын

now we need a way to combine a pose + consistent character together.

@AnimatingDreams Жыл бұрын

That's amazing! Will it work on Colab as well?

@duplicatemate7843 11 ай бұрын

does it work on colab?

@Cneq Жыл бұрын

Man control net open pose + VR FBT with 11pt can seriously open up some possibilities.

@TheDkmariolink Жыл бұрын

The open pose is a game changer, it would be great for video input as well, for things such as dances or movement, is that doable? Seems like doing this in batch would take ages. Maybe this could be implemented within Deforum?

@bartygamer Жыл бұрын

waiting >D

@OsakaHarker Жыл бұрын

K you forgot to copy the checkpoints in the ControlNet/annotator/ckpts if you do then the hand pose works amazingly using preprocessor openpose_hand and the model openpose, thnk you for this amazing video, this changes a lot how we create images

@rmb6037 Жыл бұрын

where are those? I don't see them on the github page

@OsakaHarker Жыл бұрын

@@rmb6037 on the models link page go back once into ControlNet and then enter the annotator/ckpts folder

@ThisOrThat13 Жыл бұрын

@@rmb6037 look within "annotator" just above the models folder. Then ckpts into those files.

@ThisOrThat13 Жыл бұрын

That is where I'm lost at now. We would those files (pth & pt) go? Just into the model folder with everything else?

@OsakaHarker Жыл бұрын

@@ThisOrThat13 i noticed they were auto downloading but wasn't working for me and i put them all in here \extensions\sd-webui-controlnet\annotator\ckpts

@boythee4193 Жыл бұрын

i had to restart the whole thing to get the models part to show up, but it did work. too late to test it out now, though: )

@Smiithrz Жыл бұрын

Thank you so much for making this video. I was just talking about how this stuff will solve a lot of posing problems we have without it. Can’t wait to try it with my model photography as references 👏🏻

@ryanp515 26 күн бұрын

This is cool. I was wondering could this be used to make line art? That would be a time saver with poses, etc.

@luclenders Жыл бұрын

Great content! thanks for all the work. If i may give one suggestion. I often find myself coming back to a video to checkout the process again. But I noticed like in this video for example it does not mention Controlnet in the thumbnail or title. Its difficult to look for a specific video in all your videos :) Thanks again.

@gabe22222 Жыл бұрын

nice!

@mrrooter601 Жыл бұрын

This is great, the hand one seemed to work for me, at least on one base image. It refuses to work at all with waifu diffusion 1.4e2 though.

@itsalwaysme123 Жыл бұрын

There exists a safetensors version of the models that take up *signifigantly* less space! but other than that, golden.

@unknowngodsimp7311 Жыл бұрын

This is awesome. But I'm kinda confused about how the 3 "inputs" relate to each other. Perhaps this is me just not understanding img2img. Basically my question is how do the prompt, image, and 2nd image (for depth map) relate to each other for the resulting image? Doesn't this new extension mean that we could (also) use only an image or prompt with the image for depth map to generate an image? I would love an in-depth answer 🙏

@TomiTom1234 Жыл бұрын

Something you could have covered in the video, is the inpainting using this method. I noticed in their site that you can inpaint a part of an image. I wish you explained that.

@JohnVanderbeck Жыл бұрын

So a few thoughts playing with this. I specifically zeroed in on the pose model because one of the things I've been trying (and failing) to do for a long time now is mix my own photography with generation, but having any sort of control over posing was nearly impossible. Until now! First off, this works in txt2img as well, so you can supply it a pose reference image and then do your normal txt2img prompts and get a completely new generation in that pose. Mind fracking blown! That said, temper expectations at least for now as the pose estimation is not so accurate. This is rather surprising actually given 2d pose estimation has been a pretty well solved problem for longer than SD has been a popular thing, so not sure whats up there. Still it has already allowed me to start making fusions of my studio photography with SD generations and it is amazing!

@ysy69 Жыл бұрын

Are you saying this extension can also be used over at txt2img?

@JohnVanderbeck Жыл бұрын

@@ysy69 Yes! That's mostly where I am using it right now actually. I'm talking some of my models that i've shot in studio, bringing them into SD in txt2img, putting that photo in for controlnet pose, and then using the txt2img to generate a completely new person in roughly the same pose as the one I photographed.

@Seany06 Жыл бұрын

@@ysy69 txt2img is where it works, not img2img

@Seany06 Жыл бұрын

According to github it should be possible with the open pose model to control the skeleton but gradio isnt easy to work with. I'm sure in a few months we'll have a lot more control. These tools are insane so far!

@ysy69 Жыл бұрын

@@JohnVanderbeck when you say model, you're referring to SD custom models right? and not people as models? When you bring a photo to SD, that is img2img... in text2img, one doesn't use an image as reference, so I guess you meant img2img and then using the prompt to make the change into a new person, correct?

@asadanauyo5759 Жыл бұрын

WOW! :)

@YarivTawiliArt Жыл бұрын

I will definitely try it. I wonder if it works with multi characters

@artfoolmonkey2866 Жыл бұрын

Hi and thanks for your amazing guides. I followed thru the steps but for some reason i'm only having access to controlnet-m2m and can't find any other options. Is there any other requisites before install the controlnet extension or something to configure??

@CrazyEditsCrazy Жыл бұрын

can we use this for Batch ??

@cyberfalcon2872 Жыл бұрын

Your ControlNet playlist is out of order. This video should be the first on the list

@rickland1810 Жыл бұрын

Amazing videos! Thank you. Just a suggestion, maybe the part where you download models in your vids should be first, so they download while we do the other steps. I already know your videos so I get ahead of this. But again, thank you.

@jucabalmacabro Жыл бұрын

amazing. I´m buyng a new pc to be able to play with stable diffusion. IN LOVE.

@ThisOrThat13 Жыл бұрын

1tb isn't enough anymore with all the models I have installed just for Text2text.

@jucabalmacabro Жыл бұрын

@@ThisOrThat13 omg, my new pc have only 1tb, Im screwed

@ThisOrThat13 Жыл бұрын

@@jucabalmacabro most Mobo will hold two M.2s now. The 2nd one could be a 2tb for SD and games. Or have a portable HD to change models in and out.

@MrOopsidaisy Жыл бұрын

Are you able to create an updated installation video? I've been out of the loop for a few months with Stable Diffusion and feel lost with all the updates.......... :(

@shongchen Жыл бұрын

Hello, I have a question , how to find the control stable diffusion with human pose tab ? Thank you for your share.

@Warzak77 Жыл бұрын

people are going to bach colorate manga scans with this ! amazing discovery

@Amelia_PC Жыл бұрын

Please, consider talking about ControlNet with Anime Line Drawing for Stable Diffusion when it's available for the public! :D It's a revolution for comics/mangas and animation.

@ayanechan-yt Жыл бұрын

To use the seg model, venv/script/activate then pip install prettytable then restart webui.

@XavierCliment Жыл бұрын

Hello Robot, how to colorize an old photo in greyscale or black and white with controlnet? which options or parameters are better? Thanks

@thailandbestbest8172 Жыл бұрын

Using it with rtx2060 6 GB ram without low vram... Just extract the model and it will be around 1.444 gb instead of 5.7... :) :) :) the extract file in the GitHub repo. And--+ thank u very much u r doing a wonderful job!!!

@thays182 10 ай бұрын

Is there way to use img to img and control net to move an existing character, with clothing and style, into a new pose/new position, and have it still be that initial character? (WIthout a Lora, only going off of one image)?