TouchDesigner Tutorial - Generate AI Images with Stable Diffusion using Image-to-Image Generation

Рет қаралды 42,697

Күн бұрын

Hey! In this tutorial, we'll go over how to use Stable Diffusion with a custom component I created to generate images in TouchDesigner. The project supports 2 forms of input using prompt generation and image to image so you can use any TOP in TouchDesigner as a starting point.
Project File: drive.google.com/file/d/1nJiO...
⚠️NOTE If you're on an M1 or M2 Mac⚠️ there's a bug in TD that causes the component to be stuck on Loading. The next release of TD will include a fix. In the meantime, there's a workaround on the form here: forum.derivative.ca/t/fixed-s...
Link to the API: computerender.com/
Link to my Patreon: / tblankensmith
Tutorial Part 2 on Audio Reactive Particle Systems with Stable Diffusion: • Generate AI Images wit...
Prompt Engineering with Lexica: lexica.art/
Opt out of Ai Training: haveibeentrained.com/
Huge thank you to Peter Whidden for his support on this and his work on computerender which makes this project possible
0:00 Overview and Examples
2:00 Setup The Computerender Component
2:36 Generating Single Images
4:04 Generate Multiple Images
6:18 Working with Iterations
8:05 Working with Guidance
9:23 Customize Resolution
10:19: Handling Errors
11:08 Auto Saving Images
12:18 Prompt Engineering with Lexica
13:09 Ethics of Copying Prompts
13:50 Generating More Image Examples
15:22 Using Any TOP as Input

Пікірлер: 114

@ParkinsonMax Жыл бұрын

Thank you for setting this up, it's really awesome. Fantastic work!

@EmptyDreamTV Жыл бұрын

Thanks I was watching old TDNeuron videos all yesterday and this is pretty much what I was actually looking for so Thank you so much 😊

@prismatic.visuals Жыл бұрын

Awesome work - and very well explained!

@vickiwu8279 Жыл бұрын

thank you so much for setting this up! It is super helpful and great work!

@brvoigt Жыл бұрын

At 11:30, to make it write to the correct folder after selecting a location using the File Browser, edit the script in generate_file_name to add the slash between folder and filename op('moviefileout1').par.file = parent().par.Imagefolderpath+ '/' + parent().par.Prompt...

@irenearias9411 11 күн бұрын

thank you sososososooooo much. you are a literal angel and saved my life and i swear i'll join your patreon when i'm not a broke college student 💗💗💗💗

@RusticRaver Жыл бұрын

amazing , tx for that, it's been 6 month I was waiting for AI to hit touchdesigner with good explanations! I'd also be interested by the python side of things.

@alexeyparkhomenko6237 Жыл бұрын

fantastic, thanks for sharing your work!

@ajk_ Жыл бұрын

This is incredible! thank you so much.

@antialias8588 Жыл бұрын

Amazing.. cannot wait until realtime video generation is possible

@tangojack_ Жыл бұрын

Is this not possible real time at the moment? I haven't tried it out yet. But it's so cool!

@WildRoseFreak Жыл бұрын

I would be curious to know this as well!

@scottiefox2525 Жыл бұрын

Hey friend! Great work!

@blankensmithing Жыл бұрын

Thanks Scottie!

@suleymanyilmaz9993 Жыл бұрын

Hey Torin, thanks for amazing tutorial. Is it possible to use other models inside the computerender module? And which version of SD we are using with compute render ? Again thank you thank you .

@vjkuru8058 Жыл бұрын

amazing!. Is there a way to run the stable diffusion model locally without the API? thanks, greetings from Mexico

@StuartFingerhut Жыл бұрын

Wow, incredible work!

@blankensmithing Жыл бұрын

Thanks Stuart!

@cristinasabato3919 4 ай бұрын

amazing work!!! thank u, soo cool :)))

@glecko9241 Жыл бұрын

dude is great! thank you so much

@gnomalab Жыл бұрын

EXCELENT! Bravo.

@andreadipa9969 Жыл бұрын

Hi man! Thanks for sharing this. I've a question: is it possible to connect the chop 'keyboardin1' into the 'button1'? How?

@catarinabarros01 Жыл бұрын

Amazing work!! Is there any way to generate audio through images using stable diffusion?

@rahsheedamcrae2381 Жыл бұрын

hey question! noticing sometimes when changing the prompt, the prompt will change the name of an already existing image in my directory but the output wont save.. not sure if that makes sense but have you encountered that and what's the resolve?

@jorgeguevaralarrotta Жыл бұрын

I created new folders

@thebusinessofsade4401 Жыл бұрын

This was an amazing video. I am trying to get serious about doing this type of stuff and I was wondering how you learned to code python specifically for touchdesigner. I am thinking of learning python but want to learn it in a way that is directly relevant to touchdesigner. Also how did you find computerrender are there other websites like it?

@blankensmithing Жыл бұрын

Hey Sade, thanks so much! Matthew Regan also has some great tutorials on using Python in TD: kzfaq.info/get/bejne/ndCRd6Ri0Juvd2Q.html . Depending on your experience with python I'd actually recommend just learning the programming language outside of TD first and then diving into using it in TouchDesigner. Udemy has some great courses, as well as MIT OpenCourseWare and many others. The main way I've learned how to integrate Python into TD projects is through their wiki page. My friend Peter Whidden developed computerender so I've picked up a lot of info from him. For finding this type of stuff there're many resources but can find a lot of new info on stable diffusion on reddit.com/r/stablediffusion. For new AI research, I really love Two Minute Papers: kzfaq.info/love/bfYPyITQ-7l4upoX8nvctg.

@thebusinessofsade4401 Жыл бұрын

@@blankensmithingReally appreciate the advice (: Love the work you are doing hope to continue following your youtube journey!

@dvdgoldberg Жыл бұрын

Hi Torin, any chance to add more parameters to the compute render component ? im missing negative prompts and weights :)

@blankensmithing Жыл бұрын

Hey It's definitly something that Peter who made the Computerender API is looking into. I think negative weights are the next on his list. I'm about to add a strength parameter into the component. New tutorial out soon :)

@blankensmithing Жыл бұрын

hey if you check out the file on my newest tutorial there are negative prompts 😄

@CemreekingulerMusic Жыл бұрын

Hey Torin, thanks for the tutorial! It seems that when I try to open the .toe file on my TD, it says that this type of file is not supported. Do you have any idea why?

@blankensmithing Жыл бұрын

What version of TD are you running?

Жыл бұрын

amazing!

@genekosoy4270 Жыл бұрын

This is great, thank you! I was playing around with a higher resolution, but it seems to cap out at 640x640. I don't see how to increase the output resolution past that, is there a way to go higher?

@genekosoy4270 Жыл бұрын

This happens when I do image->image and the resolution is higher than 640x640 on the input image.

@Coolguydudeness1234 Жыл бұрын

img2img automatically downscales to 640x640, as it’s faster and results tend to get distorted at resolutions much higher than this

@riverrem738 Жыл бұрын

hey! thanks for your sharing! and I wanna to ask a question: if it is possible to use Arduino(which could read the data from outside) to control the prompt? thanks for your answering:😃)

@blankensmithing Жыл бұрын

Hey totally. It just depends on how you want that sensor data to control the prompt. I'd recommend using Firmata with Arduino to easily read sensor data from the Arduino. There's a Firmata component in the pallet of TD and you'd need to upload the Firmata sketch to the Arduino

@riverrem738 Жыл бұрын

@@blankensmithing 😃thanks for answering,but when I download the project, the computerender in TD just keep loading and nothing happened, my computer is dell and is based on Windows, is it because of my computer or I just missed something? (also I did not know where to download the macosx and images🥲)

@CFraserBierwirth Жыл бұрын

Heya Torin, thanks for this wonderful tutorial! I had it working wonderfully for a while on my M1 Macbook pro with and now it just loops the loading text indefinitely... I tried downloading a fresh copy, running it on your 2022.29850 version, regenerating the key and refreshing the folder path with the proper ./images/ format... and still all i get is loading... Any idea what could have gone wrong? I've got $10 in credit which should go a long way so its not that...

@blankensmithing Жыл бұрын

Hey did you happen to update your TD version? There's a bug in TD for M1 Mac where the SSL python library is missing and it looks for it on your system. The TD team fixed it in the upcoming build, but someone also find a temporary fix: forum.derivative.ca/t/fixed-ssl-python-module-missing-from-apple-silicon-build/308299

@CFraserBierwirth Жыл бұрын

@@blankensmithing that did it! Thanks again!

@archbramzia Жыл бұрын

Thank you for share this great project with us ! And I have a problem at first, which is i clicked and nothing happened, I checked Textport and DATs, it shows Failed to establish a new connection. Then I figured out this problem, by using Astrill VPN stealth mode. After that everything works properly:) This suggestion specially for China users.

@riverrem738 Жыл бұрын

😂请问我用梯子去登录computerender 却没有弹出购买API服务的原因也是因为墙吗？

@archbramzia Жыл бұрын

@@riverrem738 api链接不用买要买的是余额你等免费次数用完了估计就提醒你买了

@user-sl9tw1hs5b Жыл бұрын

您好，请问第一步添加了API KEY然后一直loading是因为vpn的问题吗？我不知道如何调整到您所说的Astrill VPN stealth mode，希望您可以分享更多的细节，我将不胜感激

@Kenny-oq8gu Жыл бұрын

Awasome tutorial~ I'm a freshman in Touchdesigner~Would you like to share how to make the "computerender component" ?

@teavirid2487 Жыл бұрын

Hello! I have a question. Where can I find a tutorial to make a light disperse like in your opening title?

@blankensmithing Жыл бұрын

Hey! It's using a feedback network. Noto has a great tutorial on it: kzfaq.info/get/bejne/rZp0i9eHvMrQiZs.html

@blankensmithing Жыл бұрын

I think I added in some of the RGB delay component from the pallette as well

@teavirid2487 Жыл бұрын

@@blankensmithing thanks a lot for answer!

@flupke70 Жыл бұрын

any way to implment this on local gpu with local stable diffusion install ?

@blankensmithing Жыл бұрын

Yes, you definitely can. If you have experience in python you can bring in external libraries. There's a good tutorial here: kzfaq.info/get/bejne/lbtlmsaFqtnZoHk.html and the model is available on Hugging face's website: huggingface.co/spaces/stabilityai/stable-diffusion. You can use a Script TOP to run python code and output the content to a TOP. My goal for this is that no one would need to install any libraries and I'm a huge fan of computerender.com since they're running their service essentially at the same cost as renting the GPU machines so more people can experiment with this technology.

@flupke70 Жыл бұрын

@@blankensmithing sadly I have no python experience ;) I did install SD in a folder to run on a web ui and colab. I should have all the requirements but the scripts would not work as intended here I suppose

@kalugrafik Жыл бұрын

Hey Torin! Thank you so much for the great tutorial and for sharing this TD project file with us! I have a weird problem and maybe you know the answer. I was playing around with different keywords to generate some images and everything worked just fine. Suddenly it all stopped working. There was an error coming up that says "{"status":"error","message":"User has no credit","errorType":"INVALID_AUTH"}". I thought it might have something to do with my API key but it's still in place. I even opened your original file new to see if it has something to do with the changes I made but it still doesn't work. Can you help me please?

@blankensmithing Жыл бұрын

Hey yeah, you get $0.25 of credit on the api to start with. You can add more credit on the account page of computerender

@kalugrafik Жыл бұрын

@@blankensmithing oh okay, thanks for the fast reply. how long does like $1 last then?

@blankensmithing Жыл бұрын

@@kalugrafik 4x the amount of time it took you to use the starting funds. i’m a huge fan of using computerender because it’s been reliable and also the lowest cost api

@kalugrafik Жыл бұрын

@@blankensmithing I see, it's definitely worth the money. Thanks for answering!

@erwintuijl6342 Жыл бұрын

Awesome work, making all these custom components. I'm a total newbe with AI so forgive my stupid question but, when I've entered my API key and hit Pulse it just keeps staying stuck on Loading... Is there something stupidly obvious I'm missing?

@blankensmithing Жыл бұрын

Any chance you're on an M1 Mac?

@erwintuijl6342 Жыл бұрын

@@blankensmithing Yes I am, is that the problem?

@blankensmithing Жыл бұрын

@@erwintuijl6342 There's a bug I submitted to TD that's going to be fixed in the next release. There's a workaround someone found though. forum.derivative.ca/t/fixed-ssl-python-module-missing-from-apple-silicon-build/308299

@erwintuijl6342 Жыл бұрын

@@blankensmithing You rock, sir. Thanks for the help, I'm gonna check out that workaround.

@NGR Жыл бұрын

hi torin! thanks for sharing! my TD computerender op stays in "Loading...." scrolling, and no result happen,. im on a mac m1 laptop... any idea on what the problem may be? thnks!

@NGR Жыл бұрын

i also created the api key...

@blankensmithing Жыл бұрын

What version of TD are you on? You should update to the latest

@NGR Жыл бұрын

@@blankensmithing That was it, didnt knew there was a new version. thnx!

@damianandrianakis6599 Жыл бұрын

Hey Torin, First of all, thank you very much for the video! It is so nice of you to share your knowledge with us. I have followed your first steps and generated an API Key + copy/paste on TD and clicked on pulse but nothing happened. It is just loading and on my Computerender account it states that no invocations took place. Can you help me fix the problem?

@blankensmithing Жыл бұрын

What version of TD are you on? Make sure you're on the newest

@damianandrianakis6599 Жыл бұрын

@@blankensmithing downloaded new version, is working now! :)

@user-ll4xl1dt9h Жыл бұрын

Will there be no charge?

@phajel6629 Жыл бұрын

Have problems with the m1 processor, the computer render loads all the time. Does anyone have a similar problem and can help?

@detailsstudio Жыл бұрын

hey Torin, thanks for sharing with us :) i have a problem, when i open the file in TD 2022.24200 (windows version) and put my key on the computerender Comp and generate image, nothing happened. i just see the loading on Comp. can you help me to fix that?

@blankensmithing Жыл бұрын

Hey! Could you open up the Text Port & Dats Preview(drive.google.com/file/d/16Ha12t8MDptR55FkYRjwCt5ugbPQc3HG/view?usp=sharing) after you run generate and let me know what shows up in the logs there?

@detailsstudio Жыл бұрын

@@blankensmithing Hey! it's the result drive.google.com/file/d/10A9PWwgzGqTPDf-in_pAJQFU1GTTX57V/view?usp=share_link

@blankensmithing Жыл бұрын

@@detailsstudio Could you scroll down on the text port and also share the log?

@blankensmithing Жыл бұрын

@@detailsstudio I've tested on the latest version of TD using windows. Could you also try updating your TD version?

@malmal0v0 Жыл бұрын

@@blankensmithing Hi, I'm sharing this because I'm probably encountering the same error. The environment is Windows 11,Touchdesigner 2022.26590. (This is a newer version than the project file here) File "/ui/dialogs/dialog_error/list/list/script1_callbacks", line 24, in cook TypeError: unsupported operand type(s) for /: 'str' and 'str' error So, this is not a Project1 error maybe. And I set "int(availSize)/int(textSize)" to make "availSize/textSize" valid, but the image is still not generated. There are no other errors, and it seems the API call has never been made yet.(my StableDiffusion account is newly created today.) I will continue to investigate the cause. Thanks for sharing this great project! Attached image (drive.google.com/file/d/18ooX9MPNkIoEvzqXP6HAWrEvbQP8Fnyg/view?usp=share_link)

@alexheinitz9282 Жыл бұрын

Hey man! Thanks for sharing! I have a question..so I added my own API Key and tried to generate an image but it loads and nothing happens. My Internet connection is very good. I use a macbook pro (M1) Thank you! Bests! Alex

@blankensmithing Жыл бұрын

Hey! Yes, there was one other person on an M1 that ran into this. What version of TD are you using? Also just want to double-check your TD timeline is in play mode. As a temporary solution, running the latest Intel version of TD seemed to fix it which is strange because I assumed that version wouldn't run on an M1. I tried this on a friend's M1 on the latest silicon version of TD and it worked totally fine. I should have access to an M1 mac soon and can try testing on my end

@brentmarshall Жыл бұрын

@@blankensmithing I'm encountering the same issue on an M1. Dropped some funds into computerender but that didn't solve the issue. No errors reported, no invocations in computerender, and just the loading screen appears. TD version 2022.28040, currently in play mode. Digging into the code to see if I can spot something. Really cool use of the tech!

@brentmarshall Жыл бұрын

Traceback (most recent call last): File "/project1/computerender1/run_sd", line 12, in post File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run result = self.fn(*self.args, **self.kwargs) File "/project1/computerender1/run_sd", line 12, in File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/api.py", line 119, in post return request('post', url, data=data, json=json, **kwargs) File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/Applications/TouchDesigner.app/Contents/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/adapters.py", line 514, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='api.computerender.com', port=443): Max retries exceeded with url: /generate/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."))

@blankensmithing Жыл бұрын

@@brentmarshall Ok, I tracked it down!! It looks like the ssl python library isn't included in the apple silicon build of TouchDesigner. I'm not sure if it's deliberate. I submitted a bug request to TD, we'll see what they say: forum.derivative.ca/t/ssl-python-module-missing-from-apple-silicon-build/308299

@alexheinitz9282 Жыл бұрын

@@blankensmithing Hey man! Thanks for your help! I have no experience in python so I am just staying thankful!😅 I hope you enjoy a wonderful Christmas! 🎄

@edmosh Жыл бұрын

I have this error: (Parameter : Experimental) AtrributeError: ´td.App´object has no atrribute ´experimental´. Is it because of my version of TD? 2021.16419

@blankensmithing Жыл бұрын

Yeah you should update to the latest version of TD

@edmosh Жыл бұрын

@@blankensmithing Thanks!

@rjhj_official 5 ай бұрын

When I use the project, the computerender node has a big fat red X and says error on line 5 of script2_callbacks. How do I fix this?

@blankensmithing 5 ай бұрын

what version of TouchDesigner are you using?

@rjhj_official 5 ай бұрын

@@blankensmithing The most recent. I switched to your version and it works so ye

@huaweilai3031 Жыл бұрын

It can work according to the steps before, but after 1 month, there are words that the user has no credit on the computerrender, do you have a solution?

@blankensmithing Жыл бұрын

Hey, the computerender API costs money to run. You need to log back in and add credit. You started off with $0.25. at the moment it's the least expensive stable diffusion API I've found

@streamingdev1163 7 ай бұрын

How do I import this computer render into TouchDesigner. I am trying to load the .toe file into my application and nothing is showing up

@blankensmithing 7 ай бұрын

.toe files are stand alone projects. I have an example network inside it. You can copy the component over to your other project once you have it open

@RP-bu6el Жыл бұрын

Hi Torin! How could it be done in real time by connecting the Webcam?

@blankensmithing Жыл бұрын

hey you can use img to img with the webcam as input. it wont be fully real-time. also i’d recommend working off the project file from my more recent video on making an audio reactive animation. There's an extra parameter on the component called strength that will give you better results

@kamiusu2360 8 ай бұрын

Is it possible to connect TouchDesigner with local StableDiffusion build?

@blankensmithing 8 ай бұрын

yeah, you need to have the proper GPU to run it though. Lyell Hintz (dotsimulate) made a component and Bileam has tutorial here: kzfaq.info/get/bejne/at2gocJkr7TFeZc.html

@GraylandSmith Жыл бұрын

I think I love you

@hui9962 6 ай бұрын

Wow

@unveil7762 Жыл бұрын

hei hei , what about of an integration of invoke ai!! ?? anyway thanks same code can be used for other API with little changes!! BRAVO BRAVO !!!

@julies3491 6 ай бұрын

hi there just came across this, really nice!!:) I am trying to configure an API key in computer render, any chance anyone knows how to do that now? ( I can only use the request builder but not actually get an api key)...

@blankensmithing 6 ай бұрын

Go to the Account page. There's a form where you can enter an App Name. Once you add the name it'll generate an API key for you

@felixli3060 Жыл бұрын

woooooow

@lee_sung_studio Жыл бұрын

6:50 iterations 머신 러닝 모델 반복 8:25 guidance 프롬프트에 얼마나 밀접하냐

@user-sl9tw1hs5b Жыл бұрын

New discovery: Chinese friends can try to close your vpn after clicking pulse without response, I was able to use it normally after trying this.（That's a little bit wired cause you always should using vpn can get to the websites like chatgpt...

@yttzz5388 2 ай бұрын

or use phone, it works for me

@divinetouch5204 4 ай бұрын

🐯

@LouisGedo Жыл бұрын

👋

@user-kx7lv8ms7t 2 ай бұрын

Hey,Torin.This video is very very good, but after I modified the keywords in TD, it kept showing loading and nothing happened. My version number is 2022.32660. What should I do?