Differential Diffusion - Inpainting on Steroids!
Рет қаралды 24,730
Күн бұрынDifferential Diffusion is very powerful and makes inpainting so much easier. Vlad Diffusion already has Differential Diffusion. And here is also how to use it in ComfyUI
#### Links from the Video ####
Get my Workflow here: / 101662667
github.com/exx8/differential-...
github.com/vladmandic/automatic
#### Join and Support me ####
Buy me a Coffee: www.buymeacoffee.com/oliviotu...
Join my Facebook Group: / theairevolution
Joint my Discord Group: / discord
AI Newsletter: oliviotutorials.podia.com/new...
Support me on Patreon: / sarikas
Thank you @I-Dophler for the Index
🎯 Key Takeaways for quick navigation:
00:00 🤖 Differential diffusion for better inpainting
- Explains differential diffusion and its benefits for image inpainting,
- Allows pixel-level control over inpainting with grayscale masks,
- Shows examples of impressive inpainting results.
01:00 💻 Setting up differential diffusion in Flud diffusion
- Differential diffusion is available as a script option in latest Flud update,
- Options to enable, set mask strength, invert mask and load models,
- Mentions having issues running it personally.
01:40 🛠️ Demonstrating differential diffusion workflow in Comfy UI
- Prefers demonstrating in Comfy UI for more powerful capabilities,
- Shares basic image generation pipeline as starting point,
- Introduces painting mask and preview bridge from Impact pack.
02:59 *🎨 Comparing differential diffusion to classic inpainting *
- Compares differential diffusion (green) to classic inpainting (red),
- Differential handles face structure better for natural inpainting,
- Classic struggles with proper integration like glasses, hair, eyebrows.
04:24 ⚙️ Installing differential diffusion node in Comfy UI
- Differential diffusion is a small node between model and sampler,
- Provides installation instructions via Comfy UI manager.
05:06 🧪 Further tests with mask size and denoising
- Tests with larger mask area but classic inpainting still struggles,
- Tries lower denoising and no mask blur but differential diffusion superior,
- Encourages commenting on experiences with this new method.
@OlivioSarikas 2 ай бұрын
#### Links from the Video #### Get my Workflow here: www.patreon.com/posts/101662667 github.com/exx8/differential-diffusion github.com/vladmandic/automatic
@LouisGedo 2 ай бұрын
👋
@mr.entezaee 2 ай бұрын
I wish it was free like before. We do not have access due to embargo...Unfortunately, for now, our only support is viewing and liking
@hcfgaming401 2 ай бұрын
I don't know about anyone else, but it almost pains me to see people taking their comfyUI frustration out in the comments of every single video. I can understand and respect not wanting change, humans are creatures of habit and doing things different or learning something that looks overwhelming from the outside can be scary. The problem with not wanting change is this is the state of AI right now. Its not a matter of Olivio making videos on Comfy because he secretly hates a1111, it's quite literally because a1111 doesn't get updates for this new tech half the time and all of the advancements for AI are happening on Comfy. If he only made videos for A1111 he would barely have any posts, and he already makes videos for A1111 whenever new stuff does come out. I.e. *A1111 is still getting coverage* AI is a rapidly expanding field. ComfyUI may be obsolete in 6 months, and then the new one that takes it's place may be obsolete in another 12. If this unsettles you, there is unfortunately nothing your teachers or content creators can do to fix this. You will just have to make do with the tools you have on an otherwise dying software that will no longer get updates.
@149315Nico 2 ай бұрын
I think it’s just envy. The coolest stuff can be done with a little bit of skill and work. We got the most tools, are the first one to enjoy new ones and can do that really “automatic”ally after building a workflow once. There’s a lot of Fomo going on, having to wait months for something they wanna use right now. Also , People despise having to acquire skills and put in the work. People not akin to programmer mindsets expect everything to work like a smartphone app, overly simplified and intuitive. The problem is when capability meets skill-requirement. People see crazy stuff, and want that too, and if they don’t get it on first try with 5 secs of labor, they get frustrated. What they don’t see is that unlimited opportunities necessitates a higher complexity. I’m mad at Blender users for I lack the skills in 3d design tho I’d love to have them. Yet I don’t hate on Blender tutorials for a living, for I realize blender isn’t that hard bc of gatekeepers, but bc it’s near infinite possibilities lead to a requirement for more possible options than paint has.
@hcfgaming401 2 ай бұрын
@@149315Nico Thank you for adding! This further explains it all without having my original comment being extremely long. It's truly a shame. And like the blender example you gave, can be found in almost every field from art and science all the way to physical labor and social skills. Deep down I know that my comment will achieve nothing to help the masses, but at least I got to blow off some steam!
@PretendBreadBoy 2 ай бұрын
I use comfy and it's powerful but I like auto 1111 much better not sure if it's change people are afraid of or the tedious complexity of comy. It also has a massive bloat problem, and it breaks every few months. I don't mind as I have the time and knowledge to fix it when it all of sudden stops working. For most people, it isn't worth the trouble. I agree that machine learning is a rapidly developing field and the next thing will replace comfy and auto 1111. Acting like it's the best thing in the world blindly is just as bad as complaining that it's the worst.
@TyreII 2 ай бұрын
@@PretendBreadBoy I don't use auto 1111 anymore but I don't like comfy either just stuck with it till it evolves! lol machine learning, I guess its true though its not actually ai.
@jgt76 2 ай бұрын
I tried doing this with vlad diffusion and it worked well, what an awesome feature!
@Rammahkhalid 2 ай бұрын
I always enjoy watching your videos, thank you for the efforts Keep them coming
@daviddiehn5176 2 ай бұрын
The intros wont stop getting better
@WackyConundrum 2 ай бұрын
Amazing!
@loubakalouba 2 ай бұрын
Thank you, this needs to go in krita asap
@yiluwididreaming6732 2 ай бұрын
Yup. Small but it works!!! 😆
@Chilangosta 2 ай бұрын
This really is a gamechanger, honestly. Great showcase Olivio!
@phenix5609 2 ай бұрын
Hi i started ai generation about a week ago and love it so far, started from a1111 and went into comfy after your video helped me a lot with the help of some other youtuber video aswell, but there one thing i’m not sure how to do, first is that a node or a setting that tell you from which pack the node in the workflow are coming from ? I don’t have that in mine. i made my own workflow from generating with sag and kohya aswell as auto cfg to a detailer to a first upscale using model that go into an another sampler and then as a last step an upscale with ultimate upscale, but everything work fine until it doesn’t lol when their mistakes on the body that i would have to inpaint, but i then wondered if their were a node like detailer but to correct the body. i was thinking of maybe like a mask of the whole body, except the head, that could go in a similar step that happening in the detailer, but i’m not really sure how i can achieve that, if you could tell me which node should i use or if my idea is correct and should work ? That would be wonderful thx
@Toertsch 2 ай бұрын
You can enable the pack names above the nodes in the settings for ComfyUI-Manager. It's called "Badge". The ImpactPack has several DetectorProviders that create masks (or segments) for its Detailer Samplers. The CLIPSegDetectorProvider accepts prompts like "person" and generates a mask. "BBOX Detector (SEGS)" then converts that to SEGS as input for the detailer. You could generate two masks, one for person, one for the head only, and subtract the head from the person hopefully resulting in a mask with only the body. Or use ControlNet OpenPose so the body is always in the same place and use a manually drawn mask.
@phenix5609 2 ай бұрын
@@Toertsch thx you for answering me i’ll try that and see if i can make it work :)
@TheMartiansLoveKuato 2 ай бұрын
"That's what the ladies tell me anyways" LOL
@John_Krone 2 ай бұрын
0:07 Judas Priest 🤘 ☠😉
@user-bx6kv1ik4p 2 ай бұрын
cant find differential diffusion node in the manager... :(
@andrewqUA 2 ай бұрын
Add Node - _for_testing - Differential Diffusion (it's built in ComfyUI)
@victor20886 2 ай бұрын
That’s amazing in comfy! I don’t know why people still use automatic
@nolanzor 2 ай бұрын
we need a compilation of all the intros
@kaso6164 Ай бұрын
is there an equivalent to "inpaint only masked" in comfy? like how would i be able to only rerender a small part of a big image while using comfy?
@TheColonelJJ Ай бұрын
If you get this working in A1111, PLEASE show us this again!
@eyalohim 2 ай бұрын
looks crazy when will it be up on automatic1111
@pfbeast 2 ай бұрын
How to use "instruct pix 2 pix" & "SDXS" in comfyui?
@Gilotopia 2 ай бұрын
I don't understand your patreon teers. What's the creative pack?
@williamweatherspoon4527 2 ай бұрын
Can we replace text yet?
@anoubhav 2 ай бұрын
Do you have a tutorial on how to create the end animated scene? What tech was used? Thanks for the videos.
@AB-wf8ek 2 ай бұрын
Try looking up Adobe Character Animator
@puoiripetere 2 ай бұрын
I think it's the same method used in Krita's impainting😊
@user-ls1vt7pj4u 2 ай бұрын
I'm a Comfyui newby and I have a quick question. All the videos I've seen only show 1 inpainting result. What node must I use to create multiple inpainting results. I've tried to add an empty late image node but I'm struggling with the connections. Is there a simple way to do this?
@zizhdizzabagus456 2 ай бұрын
thats cool, but i actually cant find out how to make forkflow go on after i apply mask in bridge. it doesnt wait until i apply mask, it just go on) also sometimes seems like this node gets copletely ignored. i tryed to replace it with a preview chooser, which pause process, but it gives out wrong mask i guess, so it doesnt work
@Macatho 2 ай бұрын
Ah so this is literally just basing the denoising strength on the depth map/other map "height" and correlating the two? Quite interesting take on it, I wonder how resources demanding it is. And how it deals with boundaries etc
@CL-rm6sb 10 күн бұрын
Yep. Arguably something that should've been there from the start. The mask editor needs some work as well as everything that comes out of it is full black mask.
@KINGLIFERISM 2 ай бұрын
I wonder if this can work with ipadapter.
@tripleheadedmonkey6613 2 ай бұрын
Great work. But don't sell yourself short :D I'm sure the ladies are perfectly satisfied :P
@OlivioSarikas 2 ай бұрын
@sinayagubi8805 2 ай бұрын
For which stable diffusion versions does it work?
@OlivioSarikas 2 ай бұрын
borh
@hleet 2 ай бұрын
Small but it works ! haha :D
@joebaranowski3008 2 ай бұрын
I had to disable FreeU to get it to work
@spearcy 2 ай бұрын
But what is the actual process for installing differential diffusion so that one can find that node? It's not a searchable term in the ComfyUI Manager.
@Toertsch 2 ай бұрын
no custom nodes needed! Update ComfyUI
@bigglyguy8429 Ай бұрын
Let me change the heading for you - Differential Diffusion - Inpainting on Comfy - bletch
@thomassynths 2 ай бұрын
KampfUI
@johns2861 2 ай бұрын
lol
@SwiftyGifts 2 ай бұрын
Works great at putting sunglasses and t-shirts on a character, but when I tried to put a sword in the character's hand it refused to do it. (Scratching my head.)
@bigglyguy8429 Ай бұрын
Actually said no or just failed at it?
@ChubbyOneM 2 ай бұрын
My comfy is not working 😢 please help, i am using windows 11, comfy not working 😢😢
@Daniel32396 14 күн бұрын
I don't have that "Manager" option. Please help.
@OlivioSarikas 14 күн бұрын
you need to install comfyui manager. please google it. the install is very simple
@AndreiTheBadger 2 ай бұрын
Hi. Do you know what check layers are in Photoshop ? Well there are some layer with filters applied to them that can check for value. saturation and color. Any idea on how can we achieve something like this in Afinity ? Also, how can we import multiple images in the same doc file.. like in photoshop. import images into stack. I know there is import stack files in Affinity but it merges them somehow...
@greypsyche5255 2 ай бұрын
So how do we install this thing, is the node exclusive to your patreon members?
@OtakuDYT 2 ай бұрын
Built into the latest ComfyUI by default
@greypsyche5255 2 ай бұрын
@@OtakuDYT yeah i figured that out 5 minutes later, but its odd he made it seem it wasnt and that we needed to install external plugins.
@OtakuDYT 2 ай бұрын
@@greypsyche5255 There are some external plugins in his workflow, probably why he is covering all bases.
@Dazzastrous 2 ай бұрын
InPANTING? hmm is that like Inpainting but when your thirsty? :)
@cXedis 2 ай бұрын
🙃
@I-Dophler 2 ай бұрын
🎯 Key Takeaways for quick navigation: 00:00 *🤖 Differential diffusion for better inpainting* - Explains differential diffusion and its benefits for image inpainting, - Allows pixel-level control over inpainting with grayscale masks, - Shows examples of impressive inpainting results. 01:00 *💻 Setting up differential diffusion in Flud diffusion* - Differential diffusion is available as a script option in latest Flud update, - Options to enable, set mask strength, invert mask and load models, - Mentions having issues running it personally. 01:40 *🛠️ Demonstrating differential diffusion workflow in Comfy UI* - Prefers demonstrating in Comfy UI for more powerful capabilities, - Shares basic image generation pipeline as starting point, - Introduces painting mask and preview bridge from Impact pack. 02:59 *🎨 Comparing differential diffusion to classic inpainting * - Compares differential diffusion (green) to classic inpainting (red), - Differential handles face structure better for natural inpainting, - Classic struggles with proper integration like glasses, hair, eyebrows. 04:24 *⚙️ Installing differential diffusion node in Comfy UI* - Differential diffusion is a small node between model and sampler, - Provides installation instructions via Comfy UI manager. 05:06 *🧪 Further tests with mask size and denoising* - Tests with larger mask area but classic inpainting still struggles, - Tries lower denoising and no mask blur but differential diffusion superior, - Encourages commenting on experiences with this new method. Made with HARPA AI
@PAEz... 2 ай бұрын
Thats impressive, I wish all tutorial makers used it. Really useful on the re-watch's. Wonder how well it would go on a really long stream like one of Purz 2+ hour streams.
@Daniel_WR_Hart 2 ай бұрын
Another AI for me to spend a whole day looking into
@I-Dophler 2 ай бұрын
@@Daniel_WR_Hart Fascinating stuff! Always excited to explore the bleeding edge of AI capabilities. Looking forward to digging into this differential diffusion concept further. Thanks for sharing, @Daniel_WR_Hart!
@I-Dophler 2 ай бұрын
@@PAEz... @PAEz... You make an excellent point! Kudos for recognizing the utility of this differential diffusion technique, especially for longer tutorial videos and streams. It's visionary thinking like yours that helps drive innovation forward. Keep up the insightful commentary!
@Daniel_WR_Hart 2 ай бұрын
@@I-Dophler I didn't share anything? If anything I should say that to you for making me aware of Harpa AI
@pavi013 Ай бұрын
😂 4:27
@FusionDeveloper 2 ай бұрын
Hahaha.
@denisedixie1625 2 ай бұрын
Please put comfyUI in the title, I've clicked off the last video and this one once I learned it's Comfy and not Auto1111 that will mess up your metrics...
@generichuman_ 2 ай бұрын
You should just learn Comfyui. Automatic1111 is checkers, join us and play some chess
@BottleMani 2 ай бұрын
Yea I too lost intrest in ai after comfy ui workflows.
@user-nl2kr1nk9s 2 ай бұрын
Take it as motivation to learn Comfy, the noodles look intimidating but using A1111 is sooooo clunky once you learn Comfy. Just the model loading speed alone is reason enough to switch even without the "one click" automation that you'll learn in literally two weeks of daily practise.
@Make_a_Splash 2 ай бұрын
No need to do that.
@MODDINGPRIME 2 ай бұрын
Your complaining is actually so annoying. Comfyui is a key part of sdxl workflow. Go somewhere else if you want simple workflows that don’t involve comfyui
@ruuuuudooooolph 2 ай бұрын
Fun fact: Forge webui has something that's called soft inpainting which functions similarly.
@149315Nico 2 ай бұрын
I think it’s not the same AFAIK it’s the inpainting method from fooocus which is also by lllyasviel There’s also a comfy implementation for that. I’d love to see these two methods compared
@bforartists 2 ай бұрын
A once more super complicated Comfy UI workflow behind a patreon wall? No thanks.
@Gregor_PL 2 ай бұрын
why don't you show A1111 anymore? I stopped watching you because of it
@therookiesplaybook 2 ай бұрын
We gotta stop trying to make ComfyUI a thing
@squirrelhallowino29 2 ай бұрын
It's so much better tho?
@therookiesplaybook 2 ай бұрын
@@squirrelhallowino29 How? I'm aware lf this style for creating visual effects but for creating a single image it looks like a lot of unecessary steps and a mess of spaghetti.
@blacksage81 2 ай бұрын
Stability AI disagrees with you, furthermore Comfy is already a thing.
@generichuman_ 2 ай бұрын
It's better in pretty much every way. Nodes aren't scary... they can't hurt you.
@therookiesplaybook 2 ай бұрын
@@generichuman_ How. How is it better.
@alcocat 2 ай бұрын
comfy 🤮
@lockos 2 ай бұрын
Comfy 🔥🔥🔥
@bigglyguy8429 Ай бұрын
@@lockos Comfy sux
@lockos Ай бұрын
@@bigglyguy8429 hoes mad xD wether you like it or not, Comfy is not going anywhere
Arch_AI_3D
Рет қаралды 942