Make your own ANIME with this new mind-blowing AI TOOL! (ComfyUI Tutorial + FREE Workflows!)

Рет қаралды 89,542

Күн бұрын

AI movies are finally controllable! I show you how to combine my consistent & poseable character workflow with a new tool called tooncrafter to create completely consistent AI animation.
If you like my work, please consider supporting me on Patreon: / mickmumpitz
Follow me on Twitter: / mickmumpitz
You can find the FREE WORKFLOWS & INSTALLATION GUIDE here: www.patreon.com/posts/new-vid...
You can find my PATREON EXCLUSIVE advanced guide, example files and poses here: [coming very soon!]
Check out my last video on this subject: • Create Consistent, Edi...
Tools used:
ComfyUI github.com/comfyanonymous/Com...
midjourney www.midjourney.com
tooncrafter (huggingface demo) huggingface.co/spaces/Doubiiu...
tooncrafter in ComfyUI
Chapters:
00:00 Intro & Overview
01:00 The Tool
02:00 midjourney
02:37 Tooncrafter on Huggingface
03:04 Using Photoshop
03:37 Create a model sheet in ComfyUI
04:19 Posing the Character
06:34 Animating the character
08:02 CleanUp & Upscale
09:05 Reference to Anime

Пікірлер: 290

@Animatorx83 8 күн бұрын

I've watched many ComfyUI tutorials and encountered numerous errors and methods that didn't work. However, yours was flawless. Everything from the file organization to the installation guide and the detailed, easy-to-follow tutorial was perfect. I wish you all the best in your endeavors. Thank you so much!

@21Zuu Ай бұрын

One of the best tutorials I've seen in a while. Thanks a lot for creating all of this and sharing literally everything with us!

@EmilyNilsen Ай бұрын

Smartest kid on the block! I mean it! Your workflows are always so innovative and clever. Thanks again for another great demo of what can be achieved to create animated films.

@mickmumpitz Ай бұрын

Thank you so much!

@HeyTikal 16 күн бұрын

@@mickmumpitz Hi please help me I lose the consistent character when I switch to the position of the perssonage yet I copy my positive and negative prompt use before

@JOLT_x3 4 күн бұрын

@mickmumpitz Dumbest kid on the block, I mean it... Your workflow is non-existant and uninspiring, Unappreciation for the first time for making such an unfulfilling demo of what weapons can be used to destroy the passion and creativity of humans.

@bleh2590 7 күн бұрын

Finally. I don't have to draw the wholesome myself. Wohoooo.

@tomorbataar5922 Ай бұрын

That's it, I'm moving over to Comfy

@ElHongoVerde Ай бұрын

I think I'm about to do the same. And I hate Comfy...

@JonBear Ай бұрын

Yeah, this video made me do it as well

@deastman2 Ай бұрын

This series of workflows is great! I have an anime-style music video that I want to create, and I thought it was probably going to be another year before the quality and temporal coherence of these tools reached an adequate level. This isn’t perfect, but it’s close enough that doing some touch-up in Photoshop becomes viable. Thanks again for figuring all of this out!

@Brostopher Ай бұрын

Super impressive! Really appreciate the information. Your workflows are some of the best I’ve used, if not the best.

@unheilbargut Ай бұрын

This is amazing. I am an artist and therefore wouldn‘t have an issue with those messed up frames or keyframes, the ai didn‘t do so well and just draw those myself. This just speeds up time so immensely. I always wanted to create an animated short film but the amount of work, I would have needed to perform, simply was way too much for my disabilities. With such tools and workflows I have hopes again. AI tools are such a godsend and help removing barriers. Either the barriers of not being able to draw or paint or whatever, or like in my case the barrier of partially paralysed hands, chronic pain and a severe fatigue. These are just amazing times.

@NoPhilospher Ай бұрын

What exactly are you trying to make? Did you try this workflow ?

@derrygorl Ай бұрын

True, this is the first AI tool that could actually be helpful for real artists. It isn't really that useful for people with no animation background unless they think the animations featured in this video are good enough.

@unheilbargut Ай бұрын

@@derrygorl Like all AI tools you should know what you are doing. Like a photo-camera doesn‘t make you a photographer-artist or the video App on your phone doesn‘t make you a movie producer. As an artist who faces some bodily challenges, I can tell you that those systems really helped starting with disco diffusion over two(?) years ago. „Real“ artists usually use everything to their disposal when creating artworks, at least those non gatekeeping ones, who take art serious and not put themselves on any sort of pedestal. I have seen just amazing animations using early stages of stable diffusion, but if you are looking for any one click solutions, you probably won‘t find it with this or other animation and video ai tools - to get them onto a level of midjourney it will take two or three more years, I assume.

@unheilbargut Ай бұрын

@@NoPhilospher in the long run I plan of making an hand drawn animation of my hamster (I usually make pictures of Hamsters) and yes, I tried this workflow with some keyframes I drew. But it didn‘t work satisfyingly good, because my hamster design is quite specific and so this wasn‘t that successful. I guess I would first need to train the ai on my own works and then try again. Or I first use the 3D right of my hamster and then use this workflow to change the aesthetics towards my 2D style. I suppose in the end it will just involve all the tools I got to my disposal. 😅

@reality4599 4 күн бұрын

Only problem is you dont own anything the AI generates unless you trained it yourself which means = People can legally profit off of your work since it isnt copyrighted.

@casseroletray Ай бұрын

amazing work. thanks for sharing!!

@drucshlook Ай бұрын

Once again, amazing workflow, working smoothly for low gpu. you're the best

@TecnoBato Ай бұрын

Damn! You are really pushing the whole nascent AI animation industry forward! Been following for a while and every video you come up with a more sofisticated approach! Congrats!

@jagz888 Ай бұрын

This will work best when you can draw the poses and fill in mismatch frames your self also you can use exaggeration anticipation, animators are getting upset but actually they are the best equipped to make content, their works will be so much better than the non artist prompters, I hope more of them experiment and push this,

@tr1pod623 Ай бұрын

what animator has been upset about this

@bake-io1cf Ай бұрын

cringe, this is for larpers

@tr1pod623 Ай бұрын

@@bake-io1cf 50/50.yes , but also can save time if you are an animator and want faster results (for fan animations, or for fun, just draw keyframes, color them in, and don't have to color each breakdown and inbetweens, (only for simple animations like headturns, smaller movements,etc.)

@bake-io1cf Ай бұрын

@@tr1pod623 if you were a professional this would only make the end result worse without actually improving efficiency at all

@jagz888 Ай бұрын

@@tr1pod623 I'm in 3d animation for around 16 years now my Linked-in is sprawling with frantically anxious concept artist animators motion designers worried about jobs, the market is not great for them right now, which is not all to do with Ai but there have been some layoffs because of it. games industry contracting, also the writers guild strikes halting productions, tax breaks for media production being stopped, but also studios with very limited budgets are modifying pipelines for ai lowering staff hires, I'm currently in a 33 man team of animators which I know did not hire any more this past year because it was not deemed necessary despite us having more projects than before & deploying AI. it's especially bad for 18- 23 age bracket of less experienced animators.

@mythicproductions9154 Ай бұрын

You know what, I am going to try this. Mess around with it a bit. Might not be perfect but the possibilities for creating proof of concepts with this intrigues me. It would give me something to do with all this writing that has built up over the years.

@DracoTheNinja Ай бұрын

this is amazing, thank you so much for puting this all together! definitely worth a sub

@mohali4338 Ай бұрын

love your content. Always something interesting ❤

@kenrock2 Ай бұрын

Most amazing work i ever seen.. great job on this one.. 💪

@PixelSubstream Ай бұрын

thank you for making these workflows free, you could have easily pay-walled all this information so i do appreciate it much!

@MythInkOfficial Ай бұрын

amazing thank you, ill be sure to look into this soon

@lijiayang-f1m Ай бұрын

Thank you very much for sharing. I would like to ask what kind of computer configuration is required for the whole process

@kleber1983 Ай бұрын

Got this error when executing the tooncrafter workflow, it happens when it passes through tooncrafter interpolation node: "Error occurred when executing ToonCrafterInterpolation: No operator found for `memory_efficient_attention_forward` with inputs (...)"

@smartduck904 Ай бұрын

Traditional forms of animation would be great for some scenes but it would definitely cut down on time for moments that are quite locked of or simple shots

@BianicEpicVideos 4 күн бұрын

And the staff needed as well

@playuptv4021 Ай бұрын

Incridible! Can i modify this to use a premade character without the "prompt" influence? it kinda mess up my input images

@user-rx4vy6xy2f Ай бұрын

thank you for all of the cool workflow ideas! if possible, do you maybe know how we can feed the workflow with an existing image of a character, for the creation of a character sheet? The PulID and InstantID models are not cooperating well with anime style characters as an image source

@Hotwire_ Ай бұрын

Your doing some Genius level SHIZZZ bruh thank you

@jeromeneareo4730 Ай бұрын

I was hoping you would release this one ! Just started watching but it looks promising 🤩🥰

@mayscabin6669 Ай бұрын

really coolll!!!! I love it!!!thanks for sharing!!!

@IntangirVoluntaryist Ай бұрын

How do you handle multiple characters at once with these methods?

Ай бұрын

Nice job!, thanks for sharing

@JonBear Ай бұрын

First off, thank you so much! You made it so simple to install and to play around with it. My first results show up as images without much animation between, character moving lips then jump to the next image. Any idea what I may be missing here?

@JonBear Ай бұрын

Reporting back after more testing - So I managed to get the animation to come out! Just chose images that look more similar to eachother. My first tests were too different key frames so it didn't even try to animate between them. I would like to experiment to try and 'force' it to interpolate even if the keyframes are very different (like getting one character to turn into something else)

@JonBear Ай бұрын

Didn't have much luck with the upscaling workflow probably because I am not using an sdxl model. Any idea how to adjust it for sd1.5?

@shireensingh2834 7 күн бұрын

hi , please answer whether is it possible to add movement to REALISTIC CHARACTERS generated by Midjourney. and also Comfyui is complex. any alternative?

@giusepperusso8624 Ай бұрын

Amazing ahaha if i want to creat multiple characters like a set of characters for a cartoon (let's say The Simpson style and each one has always the same style) do you think that is better to work with midjourney or whit this kind of system ?

@GiancarloBombardieri Ай бұрын

wow tht´s sooo cool!!! Thansk fot sharing!!

@Comic_Book_Creator Ай бұрын

amazing work

@jeissontoro5552 29 күн бұрын

Really good video! Do you know if theres any "video to anime" program that works well?

@theuptaker Ай бұрын

@mickmumpitz could I insert a IPAdaptor somewhere in this workflow to input a pre-existing look? If so...where

@GCH490 Ай бұрын

Excellent video, thank you for sharing. My art style is kind of like Invader Zim. I wonder would it be possible for me to create a model sheet in that kind of style, and then shoot my own video reference footage. I would like to think regardless of model style it should still work well. Hope to experiment this weekend, and I'll find out I guess

@Patheticbutharmless Ай бұрын

Can I suggest using something like FlowFrames to do the final clean up of interpolation? Maybe it is better to have less frames per second using your method and then interpolate the remaining frames(15 to 25 for example or 25 to 60) with software really dedicated to this. There is also paid software namely video ai 5 but it's expensive.

@user-nd7hk6vp6q Ай бұрын

Have you tried ipiv morphs workflow for something like this, I'm pretty sure you can get something similar and more smoother

@NOPEFallacy Ай бұрын

Thanks for making this video🙃

@davidpradaart Ай бұрын

Wow amazing!

@PD-THANH 15 күн бұрын

This is insane! I've always wanted to make my own anime, but my drawing skills are basically nonexistent. This workflow seems like a game-changer, even if it means spending way too much time tweaking prompts. What do you guys think, is this the future of animation or just another AI fad?

@patrickmros Ай бұрын

Does anyone know which custom node the set_vae, set_clip etc. are? The manger does not find those when trying to instal missing custom nodes.

@KarolisB123 17 күн бұрын

Can you mask your reference images to craft different chracters?

@Alex-rr7qc Ай бұрын

Hey man, that's amazing, would be great to see a short animated story via this tool. Any plans for this?

@smartduck904 Ай бұрын

This is really awesome I could help you with a combiner program which would allow you to combine all those photo outputs onto a single sheet automatically

@TheBradlackey Ай бұрын

Great Video, I love the mixed media approaches you are working with here, its fantastic. One thing that still stands out to me is all the steps in the sausage factory you still need to get results. I beg to question why wouldn't someone still work in a more traditional animation manner to create something without maybe some of the quirks? Regardless, the final effect you achieved is awesome, I just feel like if you're young watching this and diving into all this tech, maybe also learn toonboom or maya as well. It will only serve you well when mixing with comfy,etc..

@BanaKoolit Ай бұрын

Dude your intro is epic straight up

@user-xy5re3vm7f 21 күн бұрын

Is it possible to get Full HD or 2K output with 16 RAM?

@danielschweig3177 10 күн бұрын

That's great! Are you selling a course on how to set all this up and how to work with it? I want to learn it. :-D

@TheDavidTurk 6 күн бұрын

anybody else getting an error message during the pose step? I can't generate the character poses for some reason. The load ControlNet Model Nodes under the Character BG section are red and the process stops

@KEV.IN_ Ай бұрын

What if I only want to chage the pose of an existing character? cause when I upload a character, the preview image character changes A LOT.

@scetchmonkey007 Ай бұрын

Not convinced the amount of work to set this up is worth it. Whats really getting me is the difficulty on posing the character. When you make a pose and render the image you are not guaranteed to get what you wanted, complex pose gets messed up with that 3d poser as to depth maps for the hands... thats alot of work for each and every pose you may want. When you can take that 3d poser and make 3d animations as you would in maya and then be able to apply any character to that reliably, and be able to tweak timing of animations add in or remove frames, then I think you have something. Until then keep up the good work.

@EstebanRobert-bp8rc Ай бұрын

Yeah man you are really smart, thanks for share what you neuron cook with us to cook better!!

@dimitrishow_D Ай бұрын

can i input a character i designed myself instead of a prompt ?

@YaungMoney1 Ай бұрын

Is it possible to use existing anime styles as reference? For example, creating an image of myself in the style of Naruto

@Mohammedlll-jg1hw 26 күн бұрын

How do i save an image in JPG format in comfyUI? Please help me

@Amelia_PC Ай бұрын

While it's useless as a final product, for those unfamiliar with animation principles, particularly timing, it can be helpful to trace over the animation to fix consistency issues. For animators, it can speed up workflows, like figuring out the in-betweens needed to adjust the animation speed (for your sanity, keep it on twos, don't try to do 24 fps in ones with this process XD). I'll definitely not use Tooncrafter in its current state, but I'll keep an eye on it for sure. Thanks for this tutorial!

@aam7486 Ай бұрын

Can you explain more on the upscale thing? Make it simple

@MarcoKatITA Ай бұрын

lets say i want to animate a manga panel , i already have the sketches , how i use them as reference to create an anime colored scene , then , ithought to use tooncrafter to animate from a draw to another , but what if i need to manually insert new drawings in between , is there a way to make them ? , i wil set the character position for example , but im not good enaugh at drawing to make them the same as the original manga

@NoPhilospher 18 күн бұрын

Hey, are you still working on it ?

@MarcoKatITA 18 күн бұрын

@@NoPhilospher i want but i didnt find a solution for that

@felipevece Ай бұрын

i really can color an BW animation. how many correction in the color is needed?

@jetboyblue4478 Ай бұрын

Thanks for the info

@ritpop Ай бұрын

ai lip sync + ae face tracking would be perfect to the intro with the characters talking

@lenny_Videos Ай бұрын

Beautiful ❤

@NelsonStJames Ай бұрын

What the anti-AI crowd don’t understand is that AI is simply a tool, it is not replacing artists or creators. But really at the core of most AI hate is that it is another tool that democratizes an art form that was had an expensive entry point in addition to a steep learning curve.

@gbulmash Ай бұрын

I'm getting less of that from the artists than the fact that generative models were trained on their copyrighted works under an assumption of "fair use" that is not yet legally tested. There's some gatekeeping, but most of it is that they're not getting paid for the use of their work to train the AIs in the first place. There's too much black-box stuff going on. Bloggers are complaining that some models are live-scraping their sites, disguising their User Agents and ignoring robots.txt, and have proved it. OpenAI is not open about what was used in their training sets. It'll take another 3-5 years for any cases to progress to SCOTUS for a definitive ruling of what constitutes "fair use" for training for-profit AI models. Essentially, too many creators of gen AI models are "getting rich" off of potentially illegally training their models to copy the work of artists whose works are being used for training without compensation. Even Adobe, which claims to work with artists for Firefly, is being accused of appropriating art off Adobe Stock for training without honoring an opt-out. The biggest concern is whether you'll be able to trademark or copyright a character created with Gen AI. Last I read, the Library of Congress stripped an AI comic book of its copyright, then issued a new copyright just for the story and layout, but the art was not allowed to be copyrighted.

@musigxYT Ай бұрын

I seriously need to relearn how to use comfyUI soon :3

@serjaoberranteiro4914 Ай бұрын

explain how i can use it to color sketch animation

@TheSouthParkVidsFTW Күн бұрын

Stable diffusion plugin for krita is also very helpful it's come a long way.

@vinisius Ай бұрын

so nice. tks for sharing

@KeyBrute 17 күн бұрын

I manually download the zip for the xformers and replace them in the lib folder. In case anyone needs this for the toon crafter error.

@Modozenmusic Ай бұрын

How much vram i need for toon crafter workflow?

@elihusolano5993 Ай бұрын

the camera movement, if the camera movement can remain the same, or controlled, for example stop it from zooming in/out or panning (unless speceifiec) the animation would be better. Another thing, animation smear, I think the keyframes that one puts as an input if they are too far apart causes more noticible animaiton smearing. Smearing is nice if used correctly, specially in higly dynamic action scenes, but in calm slow scenes it looks off. I think maybe adding either more keyframe for slow scenes, maybe not more keyframes but psace them aout accordingly. its gonna be an iterative test until we figure out how much 'time' or 'frames' is best to put as reference

@kazukinakamoto7650 11 сағат бұрын

How long to wait for the queue?

@user-ld6xn2zq5o Ай бұрын

This is really cool!!!

@AllAboutYou2 Ай бұрын

Love you brother. You are awesome :)

@MrJonparkermain 24 күн бұрын

Do all these softwares work on Mac?? O_o

@AnimeBytz Ай бұрын

Add MuseTalk to your workflow and you have lipsync to finalize it. Always enjoying your tutorials!

@TheAmro003 29 күн бұрын

What's the day! I learned new things to day

@Naescent Ай бұрын

Animators have been using digital tools that speed up the workflow for years, so I don't see this as a bad thing for animators. For example a very popular tool in the animation industry would automatically fill in vectors for in between frames roughly and then the animators would fix the mistakes the software made and then move on. So I feel like anyone complaining about potential AI tools for animation probably aren't actually in the animation industry at all.

@SpectraPhantom-11439 Ай бұрын

Or they never tried it for fun. I tried animating a lot of times in total just for fun and it took so long just to make 1 second of animation which was about 15 frames and that was with simple clothing and stuff, plus i'm not good at backgrounds so i cant design backgrounds that well. And everyone that says "just learn it" doesnt know a thing cuz to animate u have to have high art skills which is NOT SOMEthing easy to just "get" like what everyone pretends it to be. So its easier to just use ai

@mamu_jaan_619 Ай бұрын

where is the link for model files? bro

@felipevece Ай бұрын

Show the process for coloring BW line animation

@s71402san 14 күн бұрын

It says cannot read properties of undefined (reading length)...

@BareillyFRS Ай бұрын

is tooncraft opensource??

@user-yl4nv9cl3i Ай бұрын

我想问一下在哪里可以下载或获取姿势参考。谢谢！

@yasminkastan 23 күн бұрын

Thanks

@dotayanim8 25 күн бұрын

This thing is going to f*cking up animators and artists😢 damn it

@animationbydreamsketch 23 күн бұрын

yes damn it, but hope it wont get as good as real animators and artists do😔

@inquizitive1 19 күн бұрын

When animators/artists take it on I think it’s gonna be the other way round. Those without drawing skills will be left in the dust

@dotayanim8 19 күн бұрын

@@animationbydreamsketch It already did, bruh

@animationbydreamsketch 19 күн бұрын

@@dotayanim8 to me, nope

@SiraEkabutPlus Ай бұрын

wow, really cool!

@Sunrymoon Ай бұрын

Cool, but i do art and animate because i like it.not for the results but for the process,and its why all these AI methods don't feet me but nice vid 🔥

@S.Korolev Ай бұрын

Just wow!

@inquizitive1 19 күн бұрын

I would love to see what real animators/artists can do with this

@Pimeda Ай бұрын

Sibal... It's really amazing

@DoctorMandible Ай бұрын

Ooooh and tooncrafter is apache license. Nice!

@TheGameStatistic 4 күн бұрын

the IPAdapter node is inconsistent with the one in the provided workflow. Posing is a no go.

@arabonymus Ай бұрын

Wow, you are smart an you know how to use alle the AI tools toghether

@Comic_Book_Creator Ай бұрын

can we run this on google collab? please can you check that?

@dakshroy1326 Ай бұрын

Yes

@josejaviervfx Ай бұрын

Amazing

@EddieAdolf Ай бұрын

What's the minimum RAM requirement?

@dunar1005 Ай бұрын

Would be cool to have this as colab workflow

@chariots8x230 Ай бұрын

Well that was complicated. I wish this could all be done in one application. Also, I’m still waiting to be able to use AI to create scenes with ‘multiple consistent characters’ interacting with each other.