Real time Stable Diffusion in TouchDesigner

Рет қаралды 3,800

Күн бұрын

Video Tutorial on TouchDiffusion: Interactive Real-Time Generation using StreamDiffusion and TensorRT in TouchDesigner
TouchDiffusion: github.com/olegchomp/TouchDif...
If you'd like, you can treat me to a coffee: boosty.to/vjschool
Socials:
/ olegchomp
/ oleg__chomp
00:00 - Intro
00:09 - TouchDiffusion features
00:42 - Beginning of installation
00:50 - Python install
01:08 - Git install
01:18 - Cuda Toolkit install
01:38 - TouchDiffusion install
01:55 - where python
02:13 - webui install
03:10 - Fix for error in pop up window
03:27 - Model downloading & engine preparation
04:45 - SD Turbo engine
05:07 - TouchDiffusion settings
05:46 - Usage example
07:13 - Parameters explanation

Пікірлер: 42

@burakuville 14 күн бұрын

amazing job explaining the entire process...

@alessandrosaracino5463 19 күн бұрын

this is great! lean and clean, thank you!

@cayetanojozami4067 21 күн бұрын

Amazing

@marlus 18 күн бұрын

Thanks for this great solution! Amazing performance on 4070 12GB! SD Turbo and 1 sample step. I will test making other engine now. 3 sample steps were slow on 4070.

@detailsstudio 20 күн бұрын

Thanks bro

@Remvisuals 19 күн бұрын

instnaly fast and so nice to be in 512, congrats on this one, cant wait for more features!

@VJSCHOOL 18 күн бұрын

Custom resolutions already available, but keep in mind, that it will affect FPS.

@Remvisuals 18 күн бұрын

@@VJSCHOOL I tried building a model with different res and more steps, but it wouldn't finish... Super impressed with the speed, it's still super jittery and "stabley", but that's just animdif in general I geuss, excited to try out control net!!

@prictor 10 күн бұрын

hi, i followed every step and there are no erros, but in touchdesigner when i hit pulse on "load engine" nothing happens, can anybody help me?

@VJSCHOOL 10 күн бұрын

Create issue on GitHub or Discord and share full log.

@PeterBowlesify 6 күн бұрын

Cool! Can it work with controlnet?

@OrpheusMusicOffice Сағат бұрын

how about mac~ pls❤

@user-iz7zt6md4b 21 сағат бұрын

Hi! Does it also work with Mac?

@dleon9781 14 күн бұрын

hi ,i have followed your tutorial and every step went well .But in TouchDesigner,the log shows :cannot access local variable 'pipe' where it is not associated with a value.I don't know how did it happened

@VJSCHOOL 10 күн бұрын

Create issue on GitHub or Discord with full log

@user-ik6mr1jn7n 7 күн бұрын

Nevertheless I got finished engine preparation and selected the file which webui includes, it doesnt work.. I tried to press load engine and refresh engine several times, but it dosent work.. how to resolve this :(

@user-eo3zn9kj9h 21 күн бұрын

I don't understand how it's posssible to have multiple frames per second with 20 steps. Why don't we use TensorRT on normal UI's like Automatic or Comfy ? Sorry in advance for having such a vague question but i find this quite amazing haha.

@VJSCHOOL 21 күн бұрын

Real-time can be achieved with Turbo models and Acceleration Lora, on low steps like 1-4 with TensorRT. TouchDesigner allow to copy data from and to GPU, that decrease latency.

@user-tz5ub1yq6o 19 күн бұрын

Very interesting, but when I clicked load engine in touchdesigner, I got an error like "cannot access local variable 'pipe' where it is not associated with a value". What is the problem?

@VJSCHOOL 18 күн бұрын

That’s mean either model or engine can’t be found

@yanyu813 5 күн бұрын

Have you solved it?

@VJSCHOOL 4 күн бұрын

@@yanyu813 download portable version on Github and try with it.

@th3curator 2 күн бұрын

I am getting an error: name 'StreamDiffusionWrapper' is not defined. Any ideas?

@carloslavina9779 15 сағат бұрын

same over here :( did you solve it?

@placebo_yue 16 күн бұрын

does it have to be python 311? i have 310 and i don't wanna break all my other installations to try this...

@VJSCHOOL 16 күн бұрын

1) everything installed in venv, so it not affecting main Python env. 2) if you want to try with Python 3.10, then provide path to it in webui & use TouchDesigner version before 2023

@alexdoho 20 күн бұрын

What's the difference between this and streamdiffusion?

@VJSCHOOL 20 күн бұрын

Explained in first 3 min of video It’s near x2 faster

@TheDciel1 16 күн бұрын

Git is installed. Creating .venv directory... The filename, directory name, or volume label syntax is incorrect. Failed to create virtual environment. What is the problem?

@ogunkayikci7751 14 күн бұрын

chech where the python with "where python" command and copy paste in to webui.bat like he said