How to Add ControlNet & AnimateDiff Together in ComfyUI

Рет қаралды 3,681

Күн бұрын

#stablediffusion #aiart #generativeart #aitools #comfyui
ControlNet allows you to add conditions to your prompt to allow for an accurate representation of a subject when being passed into the diffusion model. Here's how to use ControlNet along with AnimateDiff in ComfyUI.
Time Stamps
Intro: 0:00
Reviewing AnimateDiff: 0:20
Setting Up ControlNet Nodes: 1:03
Passing to Canny Edge: 1:28
Passing to DWPose: 3:33
Reviewing the Output: 6:48
Outro: 8:33
txt2vid in ComfyUI video:
• Txt2Vid Made Easy with...
Workflow:
promptingpixels.com/comfyui-w...
Install ComfyUI Manager (You'll need this in order to install the custom nodes):
github.com/ltdrdata/ComfyUI-M...
ComfyUI-Advanced-ControlNet:
github.com/Kosinkadink/ComfyU...
ComfyUI's ControlNet Auxiliary Preprocessors:
github.com/Fannovel16/comfyui...
ControlNet Models:
huggingface.co/lllyasviel/Con...
Important links:
- Patreon: / promptingpixels
- 👾 Discord: / discord
- 🌐 Website: promptingpixels.com/

Пікірлер: 14

@ParadoxicLunatic 3 ай бұрын

Great video, clear, no annoying accent, concise example of how to and what it does. Cleared up some roadblocks I encountered. Gonna check out the rest of your content the coming days and weeks!

@PromptingPixels 3 ай бұрын

Made my day - thank you for the kind words!

@TheGladScientist 5 ай бұрын

Appreciate the conciseness here. Keep up the good work!

@PromptingPixels 5 ай бұрын

You bet!

@masoud.art.videos Ай бұрын

Very clear guide thanks. Any advice on how to optimize your GPU to run things like this. I can see you are using Mac. I have a strong windows machine with 64gb GPU but getting very very long processing time for anything more than 10 second output

@GES1985 Ай бұрын

How do you add face swap into this, to control the identity of the character?

@PromptingPixels Ай бұрын

Before the VHS Video Combine node at the end you can do a face swap to then modify the frames before they are stitched together.

@bratbart12 4 ай бұрын

Hi, thx so much for an informative tutorial. I have a problem though. When I use your "vid2vid Using AnimateDiff + ControlNet" workflow I can't get the final video. Everything works including the Ksampler but the last preview window and the video combine node show only black frames. Could you give me advice on this issue? Also do you know how can I include a LORA of a person to this workflow?

@PromptingPixels 4 ай бұрын

Black frames likely has to do with ffmpeg not being installed on your machine - can install this relatively easy via homebrew. As for LoRA just pass it from the model/checkpoint to the LoRA - then into your positive and negative prompts (comfyanonymous.github.io/ComfyUI_examples/lora/).

@534A53 3 ай бұрын

@@PromptingPixels I'm not sure I understand the answer about the LoRa. Do you chain the checkpoint with the LoRa and make it so the original connectors coming out of checkpoint on the right are now coming out of LoRa instead?

@wonderhighlights 5 ай бұрын

Your app is so excellent , everything in which i am interested is present. I am a LOGO Designer can i make it here? My english is not so good but i hope you understand☺

@PromptingPixels 5 ай бұрын

Yes, you should be able to use ControlNet to create a simple logo. Perhaps will get a video about this up at one point. To note, i have an older video where i covered logo design as well here: kzfaq.info/get/bejne/hqmnaLler9rPo58.html