Tutorial 1/3 - New Body Tracking Plugin for TouchDesigner

Tutorial 1/3 - New Body Tracking Plugin for TouchDesigner - No Kinect Needed

Рет қаралды 17,901

Күн бұрын

Hey! In this tutorial, we'll go over my new body-tracking plugin for TouchDesigner. It runs off the webcam and works on both Mac and PC without any setup. In this video I'll show you an overview of the component and how to start using it in your TouchDesigner Projects. I've refined this plugin based on a number of motion tracking projects I've done with the Azure Kinect and built this to have the setup process simple and affordable. I'm really looking forward to seeing what you create with this!
The component supports BlazePose(x, y, z positions) and MoveNet(x, y positions).
Link to the Project File is on my Patreon: / tblankensmith
Link to OBS for Website: obsproject.com/
0:00 Intro and Overview
2:20 Project Requirements & Component Overview
4:34 Selecting a Webcam
5:15 Selecting an ML model
9:13 Example 1 Moving a Circle with Your Hands
12:36 Example 2 Mapping Hand Velocity to Circle Size
14:24 Example 3 Mapping Distance Between Hands to Circle Size
16:50 Example 4 Instancing all the Position Data for Rendering
18:53 Example 5 Passing your Own Video in for Body Tracking

Пікірлер: 50

@domscottart 11 ай бұрын

I was *just* looking through the laste MoveNet project I built using your WebSockets tutorial to see how I could better embed it and switch out the tools for others (MediaPipe JS instead of TFJS, and face landmark tracking rather than poses). This is amazing, thank you!

@Zenosart 3 ай бұрын

Amazing stuff. Not sure I will ever needed but very nice. Bravo!

@machines8069 11 ай бұрын

You bring so much to this community. Thank you for sharing this

@blankensmithing 11 ай бұрын

Thanks! that means so much :)

@alexeyparkhomenko6237 9 ай бұрын

this is such a nice tool, thank you soo much. looks like a gamechanger for me

@ivoryvignettes 5 ай бұрын

You're an icon! Thank you!

@1saac.mp4 2 ай бұрын

This is so awesome! I'm %100 gonna be hitting up your patreon for the project files, this is exactly the kind of stuff I've been trying to learn more about. I would love to implement this into a project. How many people is this able to track at once?

@augustababeta 7 ай бұрын

I cannot wait to start! Pretty experienced with motion tracking and real time technologies but never tried TD before so I'm suuuuuper excited! Are the download files / all the beginner download files available in basic or another tier?

@JonahHache 8 ай бұрын

so good!

@theXshoter1 11 ай бұрын

amazing!

@starinthefuture 8 ай бұрын

absolute goat

@augustineleudar 8 ай бұрын

WOWWWWWWWW thankyou !!!!!

@louisfievet9341 10 ай бұрын

OMG Mac users thank you !!

@Naevorc 6 ай бұрын

Hi Torin. Thanks a lot for this tutorial! Is it possible to create a hand reactive pixel distortion of any image with your plugin? similar to a watercolor/liquid movement Cheers

@marcusw.9684 9 ай бұрын

never used obs with this before setup would be helpful.

@darwinpeace 11 ай бұрын

Hi Torin, do you think it's somehow possible to directly feed video data (from touchdesigner) into the container without the hassle of going around OBS Studio? That would be a huge benefit! Besides that, what a great tool you've made! We've been searching for ages to find a directly integrated tracking solution in Touch Designer. Thank you so much for sharing this! Your tutorials are fantastic and it's really enjoyable to see what's possible in TD these days.😊

@blankensmithing 11 ай бұрын

hey thanks so much! Since the body tracking is running in the browser all you need is some way to send your video into a virtual camera. It looks like there might be an app for syphon that makes a virtual webacam, but it's built on top of OBS. There should also be a virtual webcam app for Spout depending on what OS you're running on.

@user-nl4nh3bd8y 8 ай бұрын

Hi I just downloaded the file. But TouchDesigner cannot use my FaceTime camera on my Mac, and the body tracking window is always white and showing nothing. Can you help me to fix this problem ?

@user-ip6km5po1n 5 ай бұрын

May I ask why the MediaPipe is not displayed and is black when I use the built-in camera in the Mac version?😭

@Matt1Up 11 ай бұрын

What about feeding in multiple synchronized videos of the same scene(person) from different camera angles?

@blankensmithing 11 ай бұрын

You could potentially stick together a video, pass in the video through OBS and enable MoveNet Multi pose

@freakyfred543 10 ай бұрын

Thanks for this video. I have a dream of creating a nighttime projection mapped busking show with body tracked projection. If you know of any links or hardware/software ideas that would help me make my dream become reality, I greatly appreciate it.

@valerakarpenko8079 7 ай бұрын

Hi, this is so cool! Is there any way to use video rather live camera?

@blankensmithing 7 ай бұрын

Yeah if you're on windows you can you Spout Cam. If you're on Mac you can use Syphon Out to OBS and OBS will make a virtual webcam

@feelzlab 3 ай бұрын

Hi Torin, I'm slowly getting to understand more about the plugin. I tried following this tutorial with the Media Pipe plugin, everything worked really great, just needed to adjust the lagging for easing but what I can't seem to find on that one is the lagging features in the recent version of the plugin. Does it contain the lagging features? if so do they have another name or where can I find it?

@blankensmithing 3 ай бұрын

hey, yeah you can use a Lag CHOP to smooth out the motion. Alternatively, you can use a Filter CHOP, but turn down the filter width

@feelzlab 3 ай бұрын

thank you!@@blankensmithing

@jaimealyss 9 ай бұрын

Hi, thanks for a great tutorial! I am using windows and am new to OBS and when I launch the software I don't see any source for Spout or for Syphon, do you now how to get these to show up? Thanks a lot ;)

@blankensmithing 9 ай бұрын

hey any chance you're on windows?

@blankensmithing 9 ай бұрын

If you are here's how to install the plugin: docs.offworld.live/#/obs-spout-plugin/README

@jaimealyss 9 ай бұрын

Hey! Thanks a lot. :) indeed I’m using windows. I will try it out. And just one question, if you’re using a Mac would you know if you need to install the plugin too?

@blankensmithing 9 ай бұрын

@@jaimealyss I think it comes pre-installed on Mac

@jaimealyss 9 ай бұрын

@@blankensmithing thanks so much! it's working great :)

@user-jt7td1ox2q 9 ай бұрын

Hi Torin! I was wondering if you have tried this on multiple people at once? How accurate is the body tracking?

@blankensmithing 9 ай бұрын

There's MoveNet Multi Pose, but it's honestly pretty jittery and the player poses swap around pretty often. The Blaze pose for a single person works really well though and is surprisingly accurate

@louisfievet9341 10 ай бұрын

Hello Torin! I tried your file on my Mac M1 but I can't get a video return in the main component. It seems that your cameras are still inside and don't take into account the one on my computer. I'm not sure what I'm saying, maybe it's something else haha.

@louisfievet9341 10 ай бұрын

Also, some options in the "Custom" tab are greyed out: Network adress, Websocket Port, Movenet Port and the MoveNet Camera Resolution is yellow. I don't know why, but if this helps you to find out what's going on.

@blankensmithing 10 ай бұрын

@@louisfievet9341 Hey! I turned off those options in this component to prevent people from tampering with them. My original camera options will show up on the component until the website has loaded correctly and pulled your camera options. You can hit the "Reset" button on the high-level body tracking component and the website will take a moment to re-load the website. Also, I can pass along the private discord to you through patreon if you need any help getting this started

@kallinikos100 4 ай бұрын

how can I add own scanned drawings to touchdesigner and be able to move them with MediaPipe'?

@blankensmithing 4 ай бұрын

you can bring images in through a MovieFileIn TOP. And you can use the translate TOP to move them around. Use the CHOP Data for any position and attack it to the translation

@Harshverma-hv7ck 3 ай бұрын

hey can you guide me i installed the TD but its only running in bg how do i fix this

@blankensmithing 3 ай бұрын

hey, I'm gonna need some more info on what you're trying to do and what's going wrong. What do you mean TD is running in the background?

@Harshverma-hv7ck 3 ай бұрын

@@blankensmithing thank you so much for the reply Im using a hp pavilion gaming laptop with ryzen 5 4600h and 1650ti with 16gb ram I downloaded and installed the full free version of touchdesigner but when im trying to launch ít nothing appears and when i open taskmanager and scroll down i can see touchdesigner running but nothing is showing.. i even downloaded thé earlier version but same problem.. help me please

@blankensmithing 3 ай бұрын

@@Harshverma-hv7ck Hey I think you might have better luck reaching out to Derivative's support team: derivative.ca/contact-us Have you tried updating your graphics drivers?

@Harshverma-hv7ck 3 ай бұрын

@@blankensmithing yes I tried updating too and no difference maybe its just à common problem ive seen online many laptop users are getting this problem btw thank you for your support