My Rokoko MOCAP Guide for Unreal Engine and MetaHumans

  Рет қаралды 80,473

Cinematography Database

Cinematography Database

Күн бұрын

This is my process of doing offline Motion Capture / Performance Capture using the Rokoko Smart Suit, Smart Gloves, and Rokoko Studio. Rendered, retargeted and MOCAP editing in Unreal Engine 4.26 and UE5.
*this video was originally produced for the Citizen Meta One KZfaq channel, but I've uploaded it here and will be taking it down from that channel

Пікірлер: 120
@mikegoesnike
@mikegoesnike 2 жыл бұрын
thanks for sharing all this info. I just got into mocap with unreal and metahumans a few days ago, and I almost feel like its a crime for me to able to access your knowledge that has accumulated through out years, condensed into a few videos. Your content on sharing your method, examples and entire workflow pipelines is really really valuable and beneficial for someone like me to get intuned with the current landscape. Thanks a whole lot.
@WhitewashStudio
@WhitewashStudio 2 жыл бұрын
Amazing that this can be done
@marcellrideg2965
@marcellrideg2965 Жыл бұрын
For anyone wondering what those batteries used with the gloves are, they are called Anker PowerCore 5000
@chosenonemedia8337
@chosenonemedia8337 2 жыл бұрын
Awesome video dude, appreciate it!
@hopperhelp1
@hopperhelp1 2 жыл бұрын
Nice. Great intro.
@GODLIMIT
@GODLIMIT 2 жыл бұрын
Great stuff, thanks!
@virtualfilmer
@virtualfilmer 2 жыл бұрын
This is great, Matt! The battery situation is bizarrely the most frustrating part about Rokoko hahaha. It’s because they can’t ship from… Denmark or whatever. 😂 I have my suit and a glove powered with one usb power brick, and my second glove on another usb power brick. And you’re right - if you put both gloves on the same usb power brick… they don’t work! Argh. I wish I could re-write their “help” stuff and also how Rokoko Studio works - connecting everything is a pain. And the 2g/5G lights - don’t get me started. WUARGHHH. 😂
@diehardzombiefan
@diehardzombiefan Жыл бұрын
Reading this after getting the anker battery….🤬
@joelarvidsson
@joelarvidsson 2 жыл бұрын
The metahuman looks more natural than before, and not just body (which should be better since you mocap) but the face expressions aswell.
@KwasnikPictures
@KwasnikPictures 2 жыл бұрын
Super cool!
@kyungjindaum
@kyungjindaum 2 жыл бұрын
Wow super cool! I was wondering if that is a custom metahuman body. Looks amazing!
@zentimtrades3470
@zentimtrades3470 2 жыл бұрын
Excellent thank you for the video - any tips you learned about calibrating the Rokoko suit - I think it could be one of my issues....
@azvasamusic7095
@azvasamusic7095 2 жыл бұрын
what the hell is a z8 loll amazing vid man! thanks for the inspiration
@jordanstarshine
@jordanstarshine 2 жыл бұрын
This is so cool 🤩🤩. Your channel is awesome! How would you go about making the motion look more natural? Changing the shutter angles, frame blending etc, motion blur? 🤔 it would probably disguise many of the small details that are too time consuming to program in the mouth movements and some of the harsh arm movements?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
With Rokoko, you would basically do a better job of cleaning up the MOCAP than I did. I was a bit heavy handed with this clean up because it was my first time really. I would probably filter a bit less on the first pass and try to use more of the raw performance. MOCAP clean up is it's own art.
@jordanstarshine
@jordanstarshine 2 жыл бұрын
@@CinematographyDatabase oh ok 🤩 I was thinking if you cut out every second frame , and applied motion blur it might also work🤔. It's almost slow motion but still so awesome 🤩 have you thought of doing music videos too? Virtual production is the entire future of the media industry🤩🤩
@renanmgs
@renanmgs Жыл бұрын
Hi Matt, your workflow to adjust the phonetic is amazing! I am in need of something similar as i will do some projects in my company. Can you help me or guide me to have something similar? Cleaning up phonetics is a pain with the keyframes, I am really in need of a solution. Cheers!
@jordanellapin5655
@jordanellapin5655 2 жыл бұрын
Hi! Cool videos thank you it’s very inspiring :) Do you know if the Rokoko gloves are compatible with the XSens suit? Xsens can detect gloves within its software MVN Animate and I don’t find any info about Rokoko gloves.
@sheraixy
@sheraixy Жыл бұрын
would love to see this character with the new Facial motion capture
@BumblingBuckeye
@BumblingBuckeye Жыл бұрын
Fantastic video. Which ASUS router did you go with?
@rushboardtechuk
@rushboardtechuk 2 жыл бұрын
I am developing a wearable of my own. I am curious as to what made you choose Rokoko vs. other mocap systems for tracking. It would be useful to know what ASUS router you used as well.
@dimeland2
@dimeland2 Жыл бұрын
How did it went? Did you manage to finish your wearable? I am also developing a full body system of my own. I know it is 1y later, but regarding the router, any dedicated router should suffice. Also the internet connection is not required (should actually be avoided) as only the wlan is necessary to connect to the sensors.
@agora3D
@agora3D 9 ай бұрын
@@dimeland2 Hi, I'm having some issues with my router is there somewhere we could talk I'd like to ask you some questions if you don't mind
@ukmonk
@ukmonk 2 жыл бұрын
its great but hopefully Rokoko will bring out a plug in to make this process easier! this is quite a chore just to get it working in UE. i mean with the retargeting etc. the blender rokoko plug in retargeting is awesome! they should bring out something similar for UE.
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
I’ve heard the C4D retargeting is pretty easy too with the Retarget tag. UE is more complicated in general compared to DCCs because it’s a game engine / real time. It isn’t simply baking data to curves/key frames. It took me a while to get comfortable with this workflow but most MOCAP to UE look exactly the same.
@ukmonk
@ukmonk 2 жыл бұрын
@@CinematographyDatabase thanks for replying. Your meta human combined with the Rokoko mo cap looks amazing though.
@ey2304
@ey2304 2 жыл бұрын
i want to do this so bad 😲
@marshal487
@marshal487 2 жыл бұрын
Thank You😊
@MeguidoCrimson
@MeguidoCrimson 2 жыл бұрын
Nice guide, thank you very much! Now all we need is for Rokoko to send their smart suit as promised and stop delaying again and again :(
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
yeah I've heard there is a lot of interest in the suits and gloves, so they are a bit back ordered
@virtualfilmer
@virtualfilmer 2 жыл бұрын
They are worth the wait, and it’s a small company so I think they are overwhelmed right now (with Covid etc). But the suit and gloves are awesome
@metaminds3812
@metaminds3812 2 жыл бұрын
I feel you. Waiting for so long already and it keeps getting delayed.
@MeguidoCrimson
@MeguidoCrimson 2 жыл бұрын
@@CinematographyDatabase They say they're sending like hundreds of suits in a per week basis. I think that's an over exaggeration otherwise it would mean the entire world is now getting rokoko suits xD
@MeguidoCrimson
@MeguidoCrimson 2 жыл бұрын
@@virtualfilmer I really need them to see how awesome they are ^^'. Right now only got the gloves, but can't do anything serious with just those.
@ChrisChris011
@ChrisChris011 Жыл бұрын
23:18 do you have the link you mentioned for the face?
@sheikhsadab
@sheikhsadab Жыл бұрын
Hello brother, can you kindly share a link from we can get the helmet (or more info)? And thanks for this great video!
@VSimCardV
@VSimCardV 2 жыл бұрын
Hi Matt, have you been able to try a Perception Neuron 3? How would you advise in getting a Rokoko Smartsuit pro II or PN3?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
I haven’t used the Perception Neuron system myself.
@4ergovyypodcast
@4ergovyypodcast 2 жыл бұрын
Hey, nice overview) can you tell, why did you choose Rokoko? I ask, because I plan to buy a suit, and right now on choosing stage between rokoko and perception neuron. Can you share your decision? Thanks
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
A year or two ago Rokoko sent me a suit to test out with Unreal Engine. Then I recently got back into MOCAP and they sent me gloves to test with MetaHumans. I see good stuff from the PN team but I’ve never personally used it. Shown in this video, I really like that the face, body, hand data is recorded all together into Rokoko Studio. That is such a simple process compared to recording each separately. Rokoko is also very active in the community and Sam does some great live streams to show how to set it up and shares helpful Blueprints for MetaHumans. Without his initial work I wouldn’t have been able to produce this video.
@4ergovyypodcast
@4ergovyypodcast 2 жыл бұрын
@@CinematographyDatabase thanks
@supremebeme
@supremebeme Жыл бұрын
wdu think of the new chest mount from rokoko for iphone?
@user-illllliliiiilliillllli
@user-illllliliiiilliillllli 2 жыл бұрын
Hi, I'm just studying Mata Human. I envy your equipment so much of your equipment.
@richardwulf1382
@richardwulf1382 2 жыл бұрын
what camera do you use? and what format?
@aislingphelan5114
@aislingphelan5114 Жыл бұрын
Anybody know where to find the livestream to the lip sync tutorial he mentions?
@rafl2627
@rafl2627 Жыл бұрын
Please can you tell me how to retarget the bones of the hands from rokoko to metahuman, I have only been able to make the body, and I have not been able to find a tutorial on this, thank you very much
@MiRaje8086
@MiRaje8086 2 жыл бұрын
genuinely thought it was real for the first 10 seconds
@prankfever8877
@prankfever8877 2 жыл бұрын
hello i have a question , if i record a basketball game and give the rokoko suite to one of the players , is it possible to replace this player with a real avatar ? can you use the motion capture system outside between real peoples ?
@metaminds3812
@metaminds3812 2 жыл бұрын
Hey Matt, after watching this video again I figured what the hell I want to watch the "very long" cleanup live-stream. Where can I find it? The link is not in the description.
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
Those streams are offline now. If you look up MOCAP clean up YT videos they are pretty much all the same process.
@loxtata5304
@loxtata5304 2 жыл бұрын
why i export nake body metahuman will got green spot on the skin around the neck and hand ?but u didnot?
@MrJonFinger
@MrJonFinger 2 жыл бұрын
🤘
@sukebanichigo7331
@sukebanichigo7331 Жыл бұрын
I hope someone can replay to this. Can this be used Visa Ipad face tracking like it can be used for an App capped animaze. I would like to know if I can make a MetaHuman rigged for V-tubing? I use Ipad 2021 with the A1 chip and Leap Motion tracking. These are the tools I have on hand. can these be used with a rigged MetaHiman?
@user-ef9zu3eh9o
@user-ef9zu3eh9o Жыл бұрын
Do I have to pay an annual program fee to use this program?
@gamehatchers
@gamehatchers 2 жыл бұрын
@CinematographyDatabase Was the Metahuman in T-pose to get it to work with Rokoko?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
Yeah the MH needs to be in a Tpose before the Rokoko LiveLink node. Rokoko should have a nice Control Rig for putting the MetaHuman into a proper Tpose. I made mine from scratch.
@FPChris
@FPChris Жыл бұрын
Looks great. The mouth still looks weird
@sharpsquint3098
@sharpsquint3098 Жыл бұрын
hello plz, Can i shoot with a Gopro camera?
@truegamer2819
@truegamer2819 6 ай бұрын
how much does the suit and helmet cost and where can i get one, whats the link friend
@HumbertoBessoObertoHuerta
@HumbertoBessoObertoHuerta 2 жыл бұрын
Hi! Can you use the face cap mount without the Rokoko suit?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
Yeah, but then it won’t capture head movement. If the camera is on a desk/stand it can also capture head rotation.
@user-nw3gk5gw5q
@user-nw3gk5gw5q 2 жыл бұрын
这个不错啊。不知道要多少钱。
@MathewHills1
@MathewHills1 8 ай бұрын
can you make an update on rokoko mo cap
@CinematographyDatabase
@CinematographyDatabase 8 ай бұрын
Yeah I’m kind of interested in their coil tech, I haven’t no idea how that works. But I don’t have that system atm.
@bahai2584
@bahai2584 5 ай бұрын
When wearing this hat, does it detect head movements, for example turning left and right?
@CinematographyDatabase
@CinematographyDatabase 5 ай бұрын
The iPhone can do the head rotation tracking but in this case the Rokoko head sensor does the head tracking.
@jzzneo
@jzzneo Жыл бұрын
Could you put the "lip sync tools" website address here?Thanks a lot!!!
@IamPhishizzle2
@IamPhishizzle2 2 жыл бұрын
Is there any fix for the accuracy of foot capture yet?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
Not that I've heard. You have to calibrate correctly, have a reliable way to keep the foot sensor on your foot, use the locomotion 2.0 filters, and then do manual foot contact clean up in Maya/UE/etc.
@cgicreator7189
@cgicreator7189 6 ай бұрын
hi brother i need Rokoko motion capture tools please locate by help me
@Assassins748
@Assassins748 2 жыл бұрын
friend where can I buy a helmet like yours for facial captures
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
I recommend Standard Deviation if you are using and iPhone.
@m1sterv1sual
@m1sterv1sual 2 жыл бұрын
Do you have your dedicated router connected to Internet, or it's just connected to power?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
I have a dedicated ASUS router that only allows the suit, gloves, iPhone, and Rokoko Studio workstation on it. It does also provide internet, but it's not needed.
@m1sterv1sual
@m1sterv1sual 2 жыл бұрын
@@CinematographyDatabase I have also ASUS and Rokoko, and it wasn’t allowing me to connect suit and gloves until I connected cable from ISP router to ASUS.. now its working as it should..
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
@@m1sterv1sual ahh thats good to know, I didn't try it without the ISP plugged in.
@chrisriehm4457
@chrisriehm4457 2 жыл бұрын
Where did you get the helmet?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
this one is from Technoprops, I recommend the Standard Deviation one for iPhones. Technoprops isn’t publicly available.
@virtualfilmer
@virtualfilmer 2 жыл бұрын
@@CinematographyDatabase $1350 inc counterweight - oooof that’s a lot of cash, considering we end up with AR kit data at the end of it 😂
@mb345
@mb345 2 жыл бұрын
Seems like AI and the work Nvidia is doing to automate a lot of the phoneme work will make this workflow seem like a thing of the past somewhat soon.
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
I think that real time MOCAP like this gets us around 80% towards realism and then GAN/DeepFake will provide tools to get the rest of the way there from the tests that I’ve seen.
@virtualfilmer
@virtualfilmer 2 жыл бұрын
Haha the HANDS CROSSING - yes. It’s a big issue with the Rokoko suit/gloves. I wish we could fix it inside Rokoko Studio. I used to think it was because my Actor (in Rokoko Studio) was set to be the wrong size. But that didn’t fix it. I then thought maybe It’s because I’m an unusual size (6ft5) so maybe the math is screwy with large bodies. But yeah. I can’t get the hands NOT to cross - it’s like the “mannequin” inside Rokoko is just… screwy. So we have to clean it up after. Booo! 👎 :)
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
Rokoko Studio doesn't have a ROM with prayer pose to try to estimate the correct bone lengths for hand position. It will definitely always require clean up later to hand positions where you want them.
@virtualfilmer
@virtualfilmer 2 жыл бұрын
@@CinematographyDatabase exactly. V annoying!
@ShilpaMehta
@ShilpaMehta Жыл бұрын
Hi! I am using Rokoko Smart Suit in UE5. Animation Successfully recorded in ROKOKO Studio. Live link is also done properly. But after recording animation with Take Recorder, the Metahuman Skeleton gets distorted in Animation Sequence. Can you please help me on that?
@CinematographyDatabase
@CinematographyDatabase Жыл бұрын
One question is how are you retargeting or are you just directly Bone Mapping? Either way, you need to make sure your animation that you are applying is using the correct source aka the correct skeleton type like Female Medium Normalweight for example or the animation will scale the MetaHuman into odd proportions.
@ShilpaMehta
@ShilpaMehta Жыл бұрын
@@CinematographyDatabase Yeah, I am using Character_BoneMap.uasset file provided by Rokoko. But that is for UnrealEngine 4.6 and I am working in UE5. So is there any other way for retargeting or Bone Mapping for UE5?
@ShilpaMehta
@ShilpaMehta Жыл бұрын
@@CinematographyDatabase Thanks a lot 👍
@wardogdauwdd3020
@wardogdauwdd3020 2 жыл бұрын
Can you do stuff like this with more then one person ?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
Yes. If you use an inertial suit like this though, interactions between the characters won’t really like up.
@wardogdauwdd3020
@wardogdauwdd3020 2 жыл бұрын
@@CinematographyDatabase so it's a yes and no with doing a project with two or three people and make a video like that ?
@prankfever8877
@prankfever8877 2 жыл бұрын
does the headset comes with the suite ? or do you have to buy it seperately ?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
The face systems are separate. Most consumers use an iPhone and a Standard Deviation HMC. High end is Faceware or Facegood.
@prankfever8877
@prankfever8877 2 жыл бұрын
@@CinematographyDatabase so you put the iphone on the helmet ? does the helmet also have sensors ? where can you buy this head set helmet
@mulagraphics
@mulagraphics 2 жыл бұрын
What's the name of the mocap helmet headset
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
This is a Technoprops, but I recommend the Standard Deviation for iPhones
@AndrewBlaze-t7k
@AndrewBlaze-t7k Жыл бұрын
I thought it was a real person OMG
@metaminds3812
@metaminds3812 2 жыл бұрын
Where can I buy these technoprop bad boys?
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
they aren't publicly available. I recommend the Standard Deviation HMC if you are using an iPhone.
@metaminds3812
@metaminds3812 2 жыл бұрын
@@CinematographyDatabase Awesome tip! Thank you they look exactly like what I was looking for.
@metaminds3812
@metaminds3812 2 жыл бұрын
@@CinematographyDatabase only the price is confusing. It looks like 100 but costs 1150. Is there anything that has a better value for money that you could also share with us?
@Creating_Space
@Creating_Space 2 жыл бұрын
can you link the 'very long livestream' about cleaning up mocap? Thanks
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
those streams are on my VTuber channel - kzfaq.info/get/bejne/rcdohqVhmLXDk4U.html
@not227
@not227 Жыл бұрын
Why is it private? 🫠
@correiaivan
@correiaivan Жыл бұрын
Now imagine this performance capture with ML machine learning deformer
@roryfarrell3587
@roryfarrell3587 2 жыл бұрын
why is your avatar moving so weird?
@gamehatchers
@gamehatchers 2 жыл бұрын
@CinematographyDatabase Have you ever used character creator 3 with your current workflow? Just wondering since they require T-pose but export with A-pose as there bind pose. Been difficult getting the character to link to Rokoko.
@CinematographyDatabase
@CinematographyDatabase 2 жыл бұрын
Character Creator you can choose what pose they export in and or just export the CC3 Tpose and use that in the AnimBP in UE. Retargeting is one of the harder parts of all of this.
@OmniVisionElite
@OmniVisionElite 2 жыл бұрын
this is sick but maybe put a shirt on dude
@stuff7274
@stuff7274 2 жыл бұрын
Hand movements are way over the top.
@Playworriors
@Playworriors Ай бұрын
Why I love envido elements?? i don't give a fu*k
Setting Up an Indie Vicon MOCAP Studio
14:43
Cinematography Database
Рет қаралды 51 М.
孩子多的烦恼?#火影忍者 #家庭 #佐助
00:31
火影忍者一家
Рет қаралды 51 МЛН
HOW DID HE WIN? 😱
00:33
Topper Guild
Рет қаралды 45 МЛН
Heartwarming: Stranger Saves Puppy from Hot Car #shorts
00:22
Fabiosa Best Lifehacks
Рет қаралды 22 МЛН
Happy 4th of July 😂
00:12
Pink Shirt Girl
Рет қаралды 61 МЛН
Indie Vicon System Cost and Solo Operator Guide
24:26
Cinematography Database
Рет қаралды 30 М.
New MetaHuman Animator is TOO REALISTIC in UNREAL ENGINE 5.2
12:59
ENFANT TERRIBLE
Рет қаралды 175 М.
Why motion capture is harder than it looks
8:36
Vox
Рет қаралды 567 М.
I wasted $4000 on a Rokoko Mocap Suit
15:29
Moon
Рет қаралды 64 М.
3 Hours vs. 3 Years of Blender
17:44
Isto Inc.
Рет қаралды 4,2 МЛН
How to take Metahumans to the NEXT LEVEL ! (Unreal Engine 5)
7:06
Cinecom.net
Рет қаралды 161 М.
Unreal Engine Metahuman Facial Motion Workflow with Faceware
6:52
Feeding_Wolves
Рет қаралды 96 М.
Create your own MetaHuman | Unreal Engine 5 tutorial
16:40
Bad Decisions Studio
Рет қаралды 743 М.
FREE Motion Capture for EVERYONE! (No suit needed)
6:07
Cinecom.net
Рет қаралды 110 М.
A Touching Story about Messi 's Kindness With Ronaldo vs Neymar !
0:38
Camera Woman Scared...#skibiditoilet #shorts
0:14
BunnyFunny
Рет қаралды 3,4 МЛН