L2: Cool Text 2 Image Trick in ComfyUI

L2: Cool Text 2 Image Trick in ComfyUI - Comfy Academy

Рет қаралды 43,010

4 ай бұрын

In this Part of Comfy Academy we build our very first Workflow with simple Text 2 Image. But then I will also show you some cool tricks that use Laten Image Input and also ControlNet to get stunning Results and Variations with the same Image Composition.
Find my Workflow here: openart.ai/workflows/oliviosa...
#### Join and Support me ####
Buy me a Coffee: www.buymeacoffee.com/oliviotu...
Join my Facebook Group: / theairevolution
Joint my Discord Group: / discord
AI Newsletter: oliviotutorials.podia.com/new...
Support me on Patreon: / sarikas

Пікірлер: 151

@OlivioSarikas 4 ай бұрын

Update: I limited the Workflow to only what is shown in the video, so it is more focused on the Lecture

@ceelentzpotter2110 4 ай бұрын

the ComfyUI can't find AV_ControlNetPreprocessor even after I fully install and update as the Manage tab said.

@CoolSongsweekly 4 ай бұрын

Same issue! Even on the openart server@@ceelentzpotter2110

@TheGalacticIndian 4 ай бұрын

@@ceelentzpotter2110 I have the same issue unfortunately🤷‍♂Solved it by and replacing AV_ControlNetPreprocessor with (right clicking on empty canvas space) Add Node/ControlNet Preprocessors/AIO Aux Preprocessor. Works like a charm👍

@LunarFerns 3 ай бұрын

@@TheGalacticIndian thank you so much!

@TheYules 2 ай бұрын

same here@@ceelentzpotter2110

@madhudson1 8 күн бұрын

everytime i watch a video with added functionality for comfyui, in this case it was the control net stages, i'm blown away with the potential of these workflows

@jesterjunk 4 ай бұрын

(( 4:24 )) Preview images DO save, but they save to the /temp directory instead of the /output directory. The saved preview images also still include the Workflow metadata. They remain there until ComfyUI is restarted, then that folder is deleted with all its contents.

@torquebiker9959 Күн бұрын

a lot of great tipps and hints, thank you!!

@MikevomMars 4 ай бұрын

4:00 - It'd be helpful if you explained WHY one should use a custom VAE instead of a checkpoint's built-in. Actually, NO ONE ever explained this so I am wondering about hat until today 🤔

@ethanhorizon Ай бұрын

This. Anyone?

@coldbeyond Ай бұрын

Custom VAE is by choise of course. Some models have baked VAE, so they don't require a custom VAE. But still you can use it. I'm new to Comfyui, but let's say you use Dreamshaper 6.31 VAE baked checkpoint in A1111; you can either use an additional VAE or you can set the VAE to Automatic.

@MikevomMars Ай бұрын

@@coldbeyond That should be clear to everyone - the question, however, is: are there any ADVANTAGES in using different VAE's and why not using the baked-in ones?

@joskun 20 күн бұрын

@@MikevomMars That's such a great question and my guest is for instance maybe you have a VAE trained specifically on Cyclops from X-men and also the style (realistic, cartoon, 3D etc.) so it would make obvious sense that the custom made specifically for Cyclops would actually be better give better results. Am wondering if that's when the Adetailers (specific in recognizing specific part of an image or removing them) would come in yet I think those maybe called embedding and maybe totally different thing

@madsnylarsen 4 ай бұрын

ohh man, i wish you had shown the entire build flow, the suddenly jumping in progression it's very confusing, especially when you are learning, while you point at a node and explaining what it does my focus is struggling with figure out how we got there, how i can get to that point, where each of the lines goes, what is an VAE, what is a controlnode etc. meaning there's very little attention on what you are trying to teach, if you instead had build it together with us,, isolated the function like explaining controlnet before using it in a larger setting, it would be way easier to follow along and learn... sorry if i am a bit harsh in my critique, i am just a little dissapointed since i looked forward to follow this series, i enjoy your work and have a ton of fun following how you explore the realm of AI :), now i'll watch the video again a few times.

@aivideos322 4 ай бұрын

If using aux preprocessors just use aio preprocessor and choose midas or the other depth maps. Olivio here is using default controlnet. They can both do the same thing.

@SaavedraCG Ай бұрын

I don't think Node Preprocessor

@JustMe-ty2rp 4 ай бұрын

Definitely some assumptions left to be made by the people jumping in at this point, but I like learning that way. Rather than download the workflow, I challenged myself to fill in the gaps and figure it all out on my own and learned quite a bit in the process. Took me a while to figure out how to get the preprocessors and controlnet models installed - especially given my setup is a little different as I am running ComfyUI via the ModelSurge Auto1111 extension, and I have my models in a non-standard area outside of my WSL2 filesystem; but here I am 3 days later and I got it all working swimmingly, with extras!!! Thank you for the wonderful tutorials @Olivio. You are appreciated. :)

@TheGalacticIndian 4 ай бұрын

Please share this apparently secret knowledge - how to get the preprocessors and install ControlNet models?

@fabiotgarcia2 4 ай бұрын

How can I add the Controlnet Preprocessor since its not appearing on the list of nodes?

@Shisgara77 4 ай бұрын

I had the same problem. But I think I solved it - in Manager in Custom Nodes I uninstalled and reinstalled the Art Ventura Custom Node used in this tutorial and all preprocessors appeared after refresh)))

@bonechieftain 4 ай бұрын

@@Shisgara77 this seemed to work for me as well. instead of double clicking and typing the search field for controlnet preprocessor, i right-clicked art venture,->loaders-.controlnet preprocessor

@vannoo67 4 ай бұрын

@@Shisgara77 I had a tough time finding this node. I downloaded Olivio's workflow, expecting to have the node marked as missing, but no, it was there. It's just tricky to find in the AddNode dialog when searching for ControlNet. It's actually called 'AV_ControlNetPreprocessor' (I guess AV is Art Venture) Update: I also needed to install "ComfyUI's ControlNet Auxiliary Preprocessors" before I could see more than 3 Preprocessors in the drop down list

@sethmunich4062 2 ай бұрын

What is Art Ventura?@@Shisgara77

@sethmunich4062 2 ай бұрын

Oh Art Venture!

@nikgrid 3 ай бұрын

I'm starting to see the benefit of comfy. Thanks Olivio! Excellent stuff

@amarmayekar7795 2 ай бұрын

Thank you very much Olivio Sarikas

@Alex-yi3yw 2 ай бұрын

Thank you very much for the lesson!

@chiptaylor1124 3 ай бұрын

Olivio, thank you so much for your tutorials!

@user-uh7vb4so4k 4 ай бұрын

Merci pour ce partage !!!! Vous êtes génial !!!!

@Pguzpet 2 ай бұрын

This video is pure gold, thank you very much Olivio Sarikas.

@gwcstudio Ай бұрын

Super useful, Olivio!

@tyegold2020 3 ай бұрын

Nice! Really getting into this now, love how calm and easy your voice is, makes it so easy to follow.

@evertonfagner 3 ай бұрын

fantastico. obrigado por compartilhar conosco.

@imagineArtsLaboratory Ай бұрын

Thanks a Billion Olivio! Helping Artists and Creatives become more efficient in technology

@kirillkhomenko_ 2 ай бұрын

I like the way you explain it. It's very simple and straightforward. Thanks

@pavi013 Ай бұрын

Very nice, i learned the basic use pretty quickly.

@majinvegeta5393 3 ай бұрын

Thanks for your lesson, it helps a lot, thanks !

@coldbeyond Ай бұрын

5:37 I wrote the same prompts and almost got the exact results like you, how cool is that 😄

@admireruk 4 ай бұрын

Nice vid, If you set the seed to the same fixed value on each sampler and set to progress by one each time. You are going to get more consistency between images.

@jccluaviz 4 ай бұрын

Thanks for your awesome video, my friend. Just playing around with nodes. Connected the latent output from the first Ksampler to the input latent from a second Ksampler and lowered a bit the denoise from this second Ksampler.Found that gives me some similar results to use contronet adapters. The question is ... using those contronet adapters gives us more consistency making the result look more similar at the original image ?. Thanks for all

@zizhdizzabagus456 2 ай бұрын

great course, thank u so much. i was alright with autmatic 1111 but comfy was not easy to understand

@cemilhaci2 2 ай бұрын

Thank you for an understandable and fluent education. How can I preview the recorded images in the interface? They appeared in the section below in some of the trainings I watched.

@LunarFerns 3 ай бұрын

Thank you for the video! It was tricky to understand how to install everything, but I did it

@sethmunich4062 2 ай бұрын

How did you install Controlnet preprocessor and the control_depth-fp16 model?

@LunarFerns 2 ай бұрын

@@sethmunich4062 to instail model you need to put the file here ...\ComfyUI\models\controlnet and for controlnet preprocessor you need to open manager and find custom node ComfyUI's ControlNet Auxiliary Preprocessors from Fannovel16 Let me know if this doesn't work

@Fiercesnake1994 4 ай бұрын

It would be really helpful if you would show how to set up comfyUI properly. Like install the models and comfyUI itself. :)

@Bullitain 4 ай бұрын

this is the 2nd lesson, he did all that in the 1st

@arieldrach809 4 ай бұрын

@OlivioSarikas I followed the first video and I'm trying to follow this video building the node but I don't have any checkpoints. What I'm missing

@helloRick618 3 ай бұрын

so cool

@kuromiLayfe 4 ай бұрын

definitely recommend getting the anything anywhere nodes to help keep workflows clean (you connect a node output to the anything anywhere node and it will automatically connect that output to all inputs of the same type (very useful with assigning models and VAE without having a bazillion lines in complex workflows). Also recommend coloring the text-encode nodes : Green for Positive and Red for Negative (will make it a bit clearer at a distance which is what)

@OlivioSarikas 4 ай бұрын

That's nice for your private workflows, but not good for when you want to share them or teach them, because then people can't see anymore where your cables are going. That's why i'm not going to use it in any of my workflows

@kuromiLayfe 4 ай бұрын

@@OlivioSarikas It adds a toggle to show/hide the cables (even animates them depending in the type of node it is transferring data from and to) like i said it is for the much more complex workflows that have like a massive logic group going on using multiple ksamplers and controlnets etc)

@phaylali 3 ай бұрын

this video explained perfectly the superiority of comfy ui , and made me give up on vladmandic automatic

@annavendemia9127 2 ай бұрын

Thank you for these lessons Olivio , they are very clear and quite easy to follow for a beginner. There is a problem with the ControlNet Processor in the workflow, both launched on the clud and run locally. It doesn't seem to be working anymore, so after the first generated image the othes can not follow. can it be reaplced with something else?

@dxnxz53 Ай бұрын

Thank you for the video! Where can i get the ControlNet Preprocessor Node ?

@tonyblack1981 3 ай бұрын

No no no! for this level of information I must both like, comment and subscribe

@shtogie5972 4 ай бұрын

I'm following along but running into a speedbump @9:00. I can't figure out which node to load in place of the "Controlnet Preprocessor" Node. That node does not exist in my environment and I instead have a package called Controlnot Auxillary Preprocessors, yet I still can't find any node similar to the Controlnet Preprocessor Node with the CNET_Name output.

@freakyninjaman3 4 ай бұрын

I ran into the same problem. With ControlNet Auxiliary Preprocessors installed, search for "MiDaS Depth Map" (or right click the workspace > Add Node > ControlNet Preprocessors > Normal and Depth Estimators > MiDaS Depth Map). This node will accomplish the same result as the tutorial (I just kept the default settings). I'm not sure what Olivio used to get that ControlNet Preprocessors node. Let me know if you figure it out :)

@sunl3583 4 ай бұрын

use MiDas Depth Map as substitute for the ControlNet Preprocessor, the rest is the same

@moky4mido 4 ай бұрын

@@sunl3583 and where the output goes?

@nowerameditationmusic3256 4 ай бұрын

@@sunl3583 Thank you, this worked for me. Note that you also need to google the control depth fp16 model safetensors and download it to get it to work.

@fabiotgarcia2 4 ай бұрын

MiDas is not connecting with the VAE Decode@@sunl3583

@pepschp5355 4 ай бұрын

There is another instance, where your workflow may be very important. Sometimes seemingly small prompts can change your image in a way you don't want to, because the model was trained that way and has some prejudicess. With your workflow you can circumvent the models prejudices and can approach your imagined outcome step by step.

@ZaindTV 4 ай бұрын

hey! i love youre content and channel! Thank you for work and keep going!

@canehoyer 3 ай бұрын

Where do I get the dreamshaper models? Can't find them in the manager. And what models would you suggest today?

@GogiRizhvadze 19 күн бұрын

I did it yeee!!!

@pavi013 Ай бұрын

Problem i been having is not only to get consistent face but also a body, i guess depth map can help with that 👍

@ogrekogrek 4 ай бұрын

Thank you Olivio for tutorial. What is difference between Batch_size and batch count.

@lefourbe5596 4 ай бұрын

Batch size : generate images in a all in one batch. Use more VRAM but is faster. Batch count : generate a number of image one after the other. Use the same amount of VRAM but is slower. You can combine them both.

@ogrekogrek 4 ай бұрын

@@lefourbe5596 thank you

@dimitrilucy 2 ай бұрын

I love your tutorials, thanks for that ! sooo Useful ! I have a issue and I don't know how to fix it : When I try to open your graph, I have a error : When loading the graph, the following node types were not found: AV_ControlNetPreprocessor This 3 packages are installed : ComfyUI's ControlNet Auxiliary Preprocessors ComfyUI_Comfyroll_CustomNodes ControlNet Auxiliar

@walidflux 4 ай бұрын

how can i add my own lora to the workflow

@kevinwang7340 Ай бұрын

I can not find contronet Preproccessor in my comfy UI, can you please explain how that part is done ?

@pastuh Ай бұрын

Someone mentioned, install extension by: comfyui-art-venture And choose: AV_ControlNetPreprocessor Also required: ComfyUI's ControlNet Auxiliary Preprocessors It will allow to choose preprocessor models

@LeVanLy-uk8lz 10 күн бұрын

I pressed control shift V but it just copied out but can't have these wire connections? Tell me why it doesn't work

@shshsh-zy5qq 2 ай бұрын

hey Olivio thank you for the amazing tutorials! I am following this tutorial and I keep getting Error occured when executing CheckpointLoaderSimple: Error while deserializing header : HeaderTooSmall

@workplaydie 2 ай бұрын

It says I don't have the AV_ControlNetPreprocessor node when I load the workflow? Edit: It might be really nice for the beginners for you to note that you are using custom nodes & that openart lists custom nodes and links the repositories. It took me a while to it figure out!

@Vestu 4 ай бұрын

UPDATE: It started working when I uninstalled some Aux...somethingsomething... preprocessor custom node I had installed some time earlier. Then restarted ComfyUI, then REINSTALLED it, restarted ComfyUI and now it works. I get an error saying "Error occurred when executing AV_ControlNetPreprocessor: 'resolution' is not in list" on the lower-most workflow.

@yingyingdeng1293 2 ай бұрын

don't know why the image is strange, eg: the car has more than four lights or wheels. is it the problem for the prompt text? appreciate for your kind reply ...

@ethanhorizon Ай бұрын

9:37 Apply ControlNet - the strength is tone down to 0.83, any special reason? What if I use 1 or more than 1?

@MrKairot 2 ай бұрын

I downloaded your workflow from OpenArt, but the images are coming out very blurry. I didn't change any settings.

@acezgaming2793 4 ай бұрын

how can i install checkpoint? I have a mac

@_carsonjones 3 ай бұрын

Following along with this and the Controlnet Preprocessor seems to be obsolete. Any suggestions for an updated workflow that achieves the same / similar output?

@matale66 2 ай бұрын

Same here. Trying to install comfyui-art-venture custom nodes but install is failing. Did you find a solution?

@iulianbabulea4997 2 ай бұрын

I do not have the manager option visible, how can I fix this?

@samwalker4442 2 ай бұрын

Might be a dumb question as just learning...Why would you use a different VAE and not the one within the model? Just for variation?

@samwalker4442 2 ай бұрын

p.s. great lecture...thank you

@RadTechDad 3 ай бұрын

My KSampler keeps showing the preview. How did you hide it?

@shubenoprod 3 ай бұрын

hi! I got an error whe i use the control net " deth-fp-16" 😢😢

@TheLoreLabs 24 күн бұрын

ooof control net preprocessor nodes missing, checked reddit, don't even have the folder, downloaded via pinokio... anyone have a fix

@nuejidub 4 ай бұрын

i can't find the controlnet preprocessor - instead i sent it to MiDas Depth Map directly. The preview image looks similar, but it won't work after that.

@OlivioSarikas 4 ай бұрын

you need the preprocessor, otherwise it is useing the color image and not a depth map. if you have the comfyui manager installed, you can install missing nodes with it very easily

@ist1jano 4 ай бұрын

@@OlivioSarikas The ControlNet Preprocessor got a red border indicating an issue, which is "depth_midas" is missing. How to fix it please? I run it in the cloud. Thanks and for the great tutorials too !

@nuejidub 4 ай бұрын

@@OlivioSarikas thanks! that worked for me!

@bobacks 4 ай бұрын

@@OlivioSarikas I just can't get this to work. It shows RED in the manager.

@ChristopherLensing 4 ай бұрын

I had the problem with the preprocessor but solved it via reinstalling art venture. But now, the only preprocessor I am able to select is "tile". I installed the models, also linked my A1111 controlnet-models and also copied them into "models/controlnet". Nothing helps. No preprocessors. Any ideas?

@lennoyl 4 ай бұрын

I had the same problem. I solved it ( thanks to the guys in the comments on Olivio's workflow download page). you have to install, in addition of the art venture node you installed, the 'ComfyUI's ControlNet Auxiliary Preprocessors' all the controlnet models will show after that.

@sethmunich4062 2 ай бұрын

Worked for me too. this needs more likes!@@lennoyl

@evertonfagner 3 ай бұрын

so frustrating. I'm stuck trying to get any ControlNet Preprocessor to work, and to no avail. one error after another, several installations and solutions, and nothing.

@Jaysunn 2 ай бұрын

Solid tutorial otherwise

@sickvr7680 4 ай бұрын

Gracias Olivioooooooo

@proximal13 11 күн бұрын

Error occurred when executing AV_ControlNetPreprocessor:

@4zreall 7 күн бұрын

A silly question, How do you download the Dreamshaper 8? A - Inside comfyUI >>> Install Custom Nodes B - Inside ComfyUI >>> Install Models C - By visiting the Civit A.I D - None of the above *Please reply honestly 🙏✨

@neema2117 10 күн бұрын

For people who have problem installing "ComfyUI's ControlNet Auxiliary Preprocessors" in comfyUI with this error: "install failed: ComfyUI's ControlNet Auxiliary Preprocessors" Solution: I disabled 'comfyui-art-venture' (if installed), then I installed 'ComfyUI's ControlNet Auxiliary Preprocessors' and re-enabled 'comfyui-art-venture'

@64kernel 4 ай бұрын

As a side comment, it's useful to use a different seed for every instance of the Ksampler or you'll get practically identical results.

@JavierPortillo1 2 ай бұрын

I don't know whats different about my workflow (i built it by hand, not downloaded) but the images get different backgrounds, for example, for the asian woman the trees are japanese cherry trees, and for the african woman, the park has dirt paths and african looking trees. Your backgrounds look very similar.

@JavierPortillo1 2 ай бұрын

I get better results if I reduce the denoising to 0.7 for the controlnet images. 👍

@sethmunich4062 2 ай бұрын

I have the same result.

@sethmunich4062 2 ай бұрын

I tried that and it made the backgrounds more similar but it made the woman's clothing less similar.@@JavierPortillo1

@sethmunich4062 2 ай бұрын

try changing the strength of the Apply ControlNet node to .83 like he has it in the video@@JavierPortillo1

@MyAI6659 4 ай бұрын

Hi Olivio, may I ask you to change your UI of choice to Invoke. It has everything, A1111 style UI, workflow UI, Cavas, almost everything. Is there a real reason why AI youtubers prefer comfyui to invokeai?

@Kal-el23 4 ай бұрын

This is obviously for pros lol. Where do you recommend learning more about SD in general?

@Pizzaguy2 2 ай бұрын

I give up with this tutorial, AV_ControlNetPreprocessor missing , how to install, I try ComfyUI's ControlNet Auxiliary Preprocessors and didn't work, why would make video tutorial and skip steps... :/

@MylesRobinette 2 ай бұрын

Restart the server and then refresh your browser like it says to do. Worked for me after installing that and refreshing both.

@Pizzaguy2 2 ай бұрын

@@MylesRobinette i think I finally fixed it, comfyui-art-venture was missing.

@GamingDaveUK 4 ай бұрын

One bit of feedback, perhaps focus on sdxl (especially when you get to stuff like control net) 1.5 should really be religated to the past.

@aivideos322 4 ай бұрын

tbh it should but its not advancing as fast as 1.5 . 1.5 is still getting more support for most things.

@GamingDaveUK 4 ай бұрын

@@aivideos322 I find 1.5 prompt crafting to be a nightmare compared to SDXL. Problem is that 1.5 gets mentioned in guides like this more than SDXL and then people learn 1.5 and are reluctant to go to SDXL so it becomes a self fulfilling loop. i would say that SDXL has already suparssed 1.5 in everyway though

@aivideos322 4 ай бұрын

@@GamingDaveUK It has for sure, I love xl and use it A LOT, but still resort to 1.5 for animations, and for the controlnet support(tile in particular) My best workflows use XL for bases and 1.5 for upscales/enhancing because of TILE controlnet. Somone needs to make this for xl... or it cant compete.

@sickvr7680 4 ай бұрын

Oops, invitation not valid when try to join Discord!

@acezgaming2793 4 ай бұрын

cannot click ckpt_name nothing is happening 1:07

@OlivioSarikas 4 ай бұрын

That means that you don't have any checkpoints in your model folder. you need to download them to your drive to be able to load them.

@alexanderhermann8299 4 ай бұрын

Are there any instructions where I can get them and is there a library where you can search for specific features for the models?@@OlivioSarikas

@alexanderhermann8299 4 ай бұрын

Are there any instructions where I can get them and is there a library where you can search for specific features for the models? I am new to the subject and find that this point is not clear. It would be nice to get another tutorial on this. Thank you@@OlivioSarikas

@bonechieftain 4 ай бұрын

i'm having the same issue @shtogie5972 is having loading the "Controlnet Preprocessor"

@shtogie5972 4 ай бұрын

Did you get this solved? I'd be happy to help.

@maxpavlov1 4 ай бұрын

I also wonder if you found a solution@@shtogie5972

@bonechieftain 4 ай бұрын

@@shtogie5972 i did not get this resolved, would appreciate any help.

@coryreeve1 4 ай бұрын

@@shtogie5972 Not sure about him, but I'm having the same issue. I even tried using Manager to "Install Missing Custom Nodes" and it fails that as well. (Still red on restart.) Did you find a viable workaround?

@groucho828 4 ай бұрын

@@coryreeve1 I had the same issue, went to install custom nodes through manager and installed Fannovel16's "ComfyUI's ControlNet Auxiliary Preprocessors" you'll then get the controlnet preprocessor node option with MiDas Depth Map