Stable Diffusion ComfyUI And Diffutoon Create AI Videos

Stable Diffusion ComfyUI And Diffutoon Create AI Videos - Domo AI Alternative?

Рет қаралды 12,208

Ай бұрын

Stable Diffusion Animation - ComfyUI And Diffutoon Create Deflickering Videos - Domo AI Alternative?
We're diving into the world of Stable Diffusion's animation and exploring a fascinating new project called Diffutoon. This project takes video-to-video transformations to a whole new level by turning dance videos into anime or cartoon-style videos.
Related Video:
Stable Diffusion Video To Anime Style • How To Make Stable Dif...
Resources :
For Patreon Supporters: / 106877649
Diffutoon Page: ecnu-cilab.github.io/Diffutoo...
Github: github.com/modelscope/DiffSyn...
For ComfyUI: github.com/AInseven/ComfyUI-f...
Throughout this video, I'll be explaining how Diffutoon functions as a pipeline, linking up various diffusion models like ControlNet and AnimateDiff Motion model. We'll also discuss the integration of ComfyUI extensions, which allow for smooth video generation and rebranding of images.
I'll guide you through the process of setting up and running Diffutoon in Google Colab, highlighting the required diffusion models, line art, depth maps, and soft edges. We'll also explore the benefits of using ComfyUI's custom nodes like "smooth videos" and "My OpenPose" for creating stunning animations with low VRAM consumption.
Moreover, I'll showcase the DiffSynth pipeline, which transforms original videos into alternative styles using diffusion models. We'll delve into the concept of image-to-image techniques and how they can be utilized in the Diffutoon project.
To provide you with a comprehensive understanding, I'll break down the workflow steps and demonstrate the differences between using the smooth video JSON file and the smooth video with batch size JSON file. We'll also touch upon the significance of ControlNet and K-sampler in achieving desired results.
So, if you're interested in exploring the world of video-to-video transformations and creating captivating anime or cartoon-style videos, this video is a must-watch for you! Don't forget to hit that subscribe button and turn on the notification bell, so you never miss an update on all things AI-related.
If You Like tutorial like this, You Can Support Our Work In Patreon:
/ aifuturetech
Discord : / discord

Пікірлер: 60

@drucshlook Ай бұрын

Imma try it tomorrow. Cheers

@kalakala4803 Ай бұрын

Looks fun Diffutoon, will try it

@TheFutureThinker Ай бұрын

yup it is. :)

@peterfontein9227 Ай бұрын

Thank you! Inspiring.

@TheFutureThinker Ай бұрын

Welcome

@CodyMysteryStories Ай бұрын

Cool! Tiktok trending video style.

@TheFutureThinker Ай бұрын

Yes, and I know agencies are doing it in Tiktok.

@UnsolvedMystery51 Ай бұрын

cool ! will try it , thanks!

@TheFutureThinker Ай бұрын

Have fun!

@crazyleafdesignweb Ай бұрын

This is fun , 1 code can do anime video. I will try it.

@TheFutureThinker Ай бұрын

Yup 👍

@cfcmoon1 Ай бұрын

Fantastic work, like always

@TheFutureThinker Ай бұрын

✌️✌️

@SFzip Ай бұрын

This is incredible! What is the input-video resolution size limit and length? Is there any point in capturing my videos in 4K?

@TheEconoVision Ай бұрын

I don't think you can generate 4K, its SD1.5 he's using. You can upscale your video after process.

@SFzip Ай бұрын

@@TheEconoVision Ok, so down-res my 4K footage to 768x512 before running it in ComfyUI. Is there a limit on the length of the footage?

@TheEconoVision Ай бұрын

@@SFzip actually , you don't need to down-res your video file. In Comfyui, use load video, connect images to Image Resize node. So you can set the dimension. Way faster.

@peacetoall1858 Ай бұрын

Cool stuff. Would have been good for dance videos but sadly will get a copyright strike on KZfaq.

@TheFutureThinker Ай бұрын

Thats why the better way to change character and background, only use the movment.

@MrJgrez Ай бұрын

I get error at realistic line art for some reason? Do you know how to resolve this? Thanks

@TheFutureThinker Ай бұрын

Sometimes, it cause by resized image problem, or the AI model itself, or the Comfyui need update. You have to check what is the error message related to.

@ysy69 Ай бұрын

This is wonderful. I assume this is not SXDL ready yet, right?

@TheFutureThinker Ай бұрын

02:10 - they have list SDXL. 😉 Img2img in ComfyUI and Diffutoon script Good to go

@ysy69 Ай бұрын

@@TheFutureThinker really!!!! I will check it out!!! thank you!!!!

@digitalflick Ай бұрын

thanks! where was the comfyui node/workflow?

@TheFutureThinker Ай бұрын

In the example folder

@digitalflick Ай бұрын

@@TheFutureThinker ok its paywalled?

@TheFutureThinker Ай бұрын

@@digitalflick in the Github, for freebie geeks.

@user-eh7vz4de4q Ай бұрын

I watched the video and tried to follow along. I saw this error message on the video combine node side. Error occurred when executing VHS_VideoCombine: Cannot handle this data type: (1, 1, 512, 3), |u1 The only thing I changed the settings to is The SD1.5 version checkpoint I have and This is about the size of a sample video. The rest of the settings were the same, but an error occurred.

@user-eh7vz4de4q Ай бұрын

Try changing the size as it was in the original example file.

@user-eh7vz4de4q Ай бұрын

How can I fix this error?

@user-eh7vz4de4q Ай бұрын

Fortunately, I changed the example file and it worked.

@TheFutureThinker Ай бұрын

Great you can do trouble shooting 👍

@huichan5140 Ай бұрын

will get blurry result if the stylize image pattern (edges, contrast) is different from the original image due to the blending algorithm; Need extra steps to deblur

@TheFutureThinker Ай бұрын

Good test👍

@iamtoufick Ай бұрын

@@huichan5140 thanks for telling

@Suresh-br2zz Ай бұрын

tried diffutoon, it is not that good & very slow

@TheFutureThinker Ай бұрын

You mean the Python code itself, Comfy or the Colab Notebook?

@CyberPhonkMusic Ай бұрын

Is it possible to make it up to 30 seconds?

@TheFutureThinker Ай бұрын

depends how many VRam u have. usually, we can set a frame cap, then skip frame to continue generate frames.

@MaghrabyANO Ай бұрын

does Google Colab approve usage of stable diffusion notes?

@TheFutureThinker Ай бұрын

Why not. Its only limited vram for free account, and gradio library for webui public link

@MaghrabyANO Ай бұрын

@@TheFutureThinker what do u mean?

@Vashthareaper Ай бұрын

2 hours and smooth video node has not moved an inch, with vid 2 vid , 60 frames cap , lcm cp and lora, tried again and again

@TheFutureThinker Ай бұрын

Recent comfyui need update, I have experienced that 2 days ago

@greenTech88 Ай бұрын

GTX 1650 super work ?😢😢

@TheFutureThinker 29 күн бұрын

No 😊

@iamtoufick Ай бұрын

Can 8gb vram handle this😅

@vaporchickenwave6980 Ай бұрын

i have 4090 24gb doesnt handle properly

@TheFutureThinker Ай бұрын

That‘s a waste for the 4090 😅

@iamtoufick Ай бұрын

@@TheFutureThinker 😂😂

@vaporchickenwave6980 Ай бұрын

@@TheFutureThinker what you mean? I tried to use it, that plugin use all my vram and is very slow!!

@TheFutureThinker Ай бұрын

I am not sure... From my 4090 setup it works. evidence showing in the video. 😉

@sandeepm809 17 күн бұрын

this is worse than animdiff