ComfyUI: nVidia TensorRT (Workflow Tutorial)

Рет қаралды 2,556

Күн бұрын

nVidia TensorRT is officially implemented for Comfy UI and Supports SD 1.5, SD 2.1, SDXL, SDXL Turbo, SD3, SVD, and SVD XT. Using ComfyUI, you gain 14% to 32% faster image generation in Stable Diffusion. I explain how TensorRT works for Stable Diffusion in Comfy and provide a comprehensive workflow tutorial to generate TensorRT .engine files.
------------------------
JSON File (KZfaq Membership): / @controlaltai
ComfyUI TensorRT GitHub: github.com/comfyanonymous/Com...
ComfyUI (Official): www.comfy.org/
------------------------
TimeStamps:
0:00 Intro.
01:55 TensorRT in ComfyUI.
15:02 TensorRT Engine Workflow.
42:30 Build TensorRT Engine.
43:56 Workflow Integration.

Пікірлер: 30

@controlaltai 12 күн бұрын

TensorRT is more complicated than what I have covered. I have restricted the explain in relevancy to Comfy UI and Stable Diffusion. LoRA works, ControlNet support is not added yet. Hope the video is helpful.

@adisatrio3871 12 күн бұрын

could you make a video to compare between, Turbo, LCM, RT, FreeU, and which combination are compatible and the best in terms of speed and quality?

@controlaltai 12 күн бұрын

That won’t help. Training the tensor takes like only 3 to 5 minutes. Whatever figures I show is highly dependant on my system. I have shown some benchmarks in the video. As far as I know for turbo the gain is least. I suggest you just train the engine and throw it in your workflow to test. Tensor RT has zero effect on quality. If you optimize a turbo model, the quality is based on that model. TensorRT only impacts the performance.

@adisatrio3871 11 күн бұрын

@@controlaltai so is it possible to get way more better performance combining, turbo model + LCM + RT?

@controlaltai 11 күн бұрын

@@adisatrio3871 I have not tried with LCM. Just tried with turbo. I doubt it would make any difference. Since only with turbo gets me 14 to 15 percent faster on average, on my 4090. Maybe lower cards have more performance boost.

@fraz66 10 күн бұрын

I don't often leave comments but I had to for this video! I came in just wanting to learn about Tensor RT(which I did) but I also came out with a killer All-In-One workflow! Seriously, this is great, thank you!

@onebitterbit 11 күн бұрын

Happy to be a member... Thank you for all your work!!

@moviecartoonworld4459 10 күн бұрын

I am always grateful for the best lectures. I may have missed something, but the explanation about Static batch size is missing.

@controlaltai 10 күн бұрын

You can put static batch size to whatever you want. If you put a number you can use the engine only with that batch size. Typically 1, 2 or 4 as per your requirement

@moviecartoonworld4459 7 күн бұрын

@@controlaltai thank you!!!!!

@onebitterbit 10 күн бұрын

Error occurred when executing TensorRTLoader: 'NoneType' object has no attribute 'create_execution_context', only on SD3 workflow, when try to load the tensorrt... any suggestion? Thanks!!

@controlaltai 9 күн бұрын

Are you using the sd3 shared workflow or have you created one?

@onebitterbit 9 күн бұрын

@@controlaltai shared

@controlaltai 3 күн бұрын

Are you using the sd3 medium vae? Also have you trained the engine on your system or using the shared engine. The shared engine file will work only on a 4090 system.

@andrejlopuchov7972 12 күн бұрын

Does tensor rt work with animatediff already by any chance?

@controlaltai 12 күн бұрын

Not tested. Will test and let you know.

@andrejlopuchov7972 4 күн бұрын

@@controlaltaitested?

@controlaltai 3 күн бұрын

Sorry could not, give me a day or two will get back.

@controlaltai 3 күн бұрын

No its not practical for animate diff at this moment. ControlNet doesn't work, it would be highly restrictive. You can train the main checkpoint with your resolution. But the workflow would be extremely limiting seeing how ControlNet is very much used with animate diff.

@hleet 10 күн бұрын

"ComfyUI TensorRT engines are not yet compatible with ControlNets or LoRAs. Compatibility will be enabled in a future update." Not interesting for the moment