Best AI Motion Capture 2021 - OpenPose vs DeepMotion

Рет қаралды 154,463

Күн бұрын

This video is sponsored by DeepMotion, check out DeepMotion now for their amazing and practical AI motion capturing service!
www.deepmotion.com/
and this is basically a video comparing FREE and PAID AI motion capture service.
I think we lack a lot of insights and comparisons between current SoTA(State of The Art) research and commercial methods. So I decided to take on this sponsorship and make a video about them. In this video, you will learn what you will straight up get the moment when you are greeted by the two of them. There are some bookkeeping stuff that I needed in the video, but I promise it'll still be pretty informative and I hope you find it helpful!
If someone is more proficient at WSL2 or OpenPose, please show us the way to export it to BVH. DM me anywhere and I'll link your tutorial of your know-how here if it works.
OpenPose
[GitHub] github.com/CMU-Perceptual-Com...
MocapNET (I kept on saying MotionNET lmao)
[GitHub] github.com/FORTH-ModelBasedTr...
This video is supported by the kind Patrons:
🙏Andrew Lescelius, Sascha Henrichs, Jake Disco, Demilson Quintao, Tony Jimenez, dicefist
Support me on Patreon if you hope to see more:
/ bycloud
or by becoming a member instead (same perks!):
/ @bycloudai
[Discord] / discord
[Twitter] / bycloudai
[Patreon] / bycloud
[Music 1] Zeeky - Froot Loops
[Music 2] Zeeky - Kool Aid
[Profile Art] / pygm7
#ad

Пікірлер: 116

@corruptedsmurf260 2 жыл бұрын

I don’t really care about the sponsored parts; I sub to this channel just to see what these AI people have thought of next. It’s like my news.

@gta12368 2 жыл бұрын

Even though it was sponsored, I think you showed their strenghts and weaknesses fairly. Great vid!

@chaosfire321 2 жыл бұрын

Definitely a ways to go before they render mocap suits obsolete. Still, amazing progress in such little time!

@daton3630 2 жыл бұрын

There are still instances where mocap suits will be need, so I don't think it will be going obsolete soon

@DeepMotionInc 2 жыл бұрын

Thanks for the feedback! We have been hard at work releasing quality updates and new features every few weeks. We don't expect to be able to replace tracking suits immediately and our current goal is to fill a gap for more cost-effective solutions that doesn't require a big investment.

@LloydSummers 2 жыл бұрын

They have ways to go before it is even really usable - I’ve found you’ll use up your credits in any tier just trying to get an OK base animation to work on

@shawnarms16 2 жыл бұрын

@@LloydSummers same here. I mean even with the suit I have…I have to do a few takes to get it right. 25 minutes tops a month an 60% unusable just want feasible yet. I love deepmotion and may use it if I can’t get my Rokoko suit to an actor.

@vikingzeppelin 2 жыл бұрын

can you add multiple camera sources to increase accuracy in DeppMotion3D? i missed it if you said so in the video

@drew230 2 жыл бұрын

Nope,

@sebastienletuan5647 2 жыл бұрын

Excellent comparison. I am assuming that once a 3-D animation file is generated, it is possible to modify it (I.e. make it do different things) ? Or are you stuck with the animation that was captured initially?

@hi_its_jerry 2 жыл бұрын

yeah you can always clean it up and modify keyframes

@veesoho93 2 жыл бұрын

amazing thank you !

@SuperlativeCG 2 жыл бұрын

The results speak for themselves.

@daydaykey Жыл бұрын

pleased to see master Ma

@LOCKSHADES 2 жыл бұрын

On the other hand, as long as the estimation of just one dimension then you should expect at worse 1/3 of the error and at best 2/3 of the dimensions( it does better in real life most times). The dinominator is the number of dimension there is to account for. At worse z depth isn't solved and just sparse data. At most xy and z are figured out but there's the handicap of just one perspective point(camera). No way to always cross check.

@LOCKSHADES 2 жыл бұрын

I think video fps and quality might be a better result. However, it's just for finer result. Some of the positioning will still be estimated at 0.7%-1.2%, currently even on high end imagery.

@holdthetruthhostage 2 жыл бұрын

Thanks for this

@OhTaco77 2 жыл бұрын

Can i export it to vmd format?

@sam_s3344 Жыл бұрын

Thank you for the amazing video! Do we have a tutorial which shows us Do we have a tutorial which shows us how to set up open pose? To someone with no coding experience.

@jaeslow6347 2 жыл бұрын

Creative processes are just going to made so much easier with this software like this, this is the time of indie games with small teams.

@RomiWadaKatsu 2 жыл бұрын

Given the jankiness of the results, that pricing is unreasonable

@MaxStudioCG2023 Жыл бұрын

what about the deepmotion copyrights? on the free version what are we alowed to do with those 30sec free anims? or what is the share we get from those?

@DemonOf4thGeneration 2 жыл бұрын

How to change the file to pmd for MMD.

@sadaqatalimeerani4807 Жыл бұрын

very helpful

@neknaksdn9539 2 жыл бұрын

deep motion: let's sponsor this guy so he can give us only a good review bycloud: na'ahhh

@mallhits 2 жыл бұрын

Dude. This info is fucking gold. I’m sold in deep motion. Thanks for the video

@tabby842 2 жыл бұрын

How do you like it so far?

@arifsultan2009 2 жыл бұрын

Is there any AI motion capture software which can capture my finger movements? Deepmotion doesn't capture fingers motion

@David_Sirovsky 10 ай бұрын

I think if you used this tool together with a cascadeur or mixamo, the result would be better

@NeroForte_ Жыл бұрын

can you compare deepmotion to ROKOKO?

@sweetambitions2381 Жыл бұрын

Is there something that could capture motions when the camera isn’t fixed? Say, a close up of walking with the camera moving 360 around it.

@cpsstudios Жыл бұрын

What version of OpenPose is this?

@warfaceindiablackburnfire330 2 жыл бұрын

Awesome 🔥

@mikealbert728 2 жыл бұрын

Can any of these free monocular solutions like openpose and vibe actually get root motion? As much as i like them, without root motion anything with root motion is inherently going to be better.

@ambocc 2 жыл бұрын

Anyone compare this to ipisoft?

@LloydSummers 2 жыл бұрын

Deep motion is not reliable, I found we needed to redo it a half dozen times but by then you used up your credits. In practice it is unmanageably expensive as you need to upgrade to deal with their unreliable framework. It also does some really bad jitter, and doesn’t do facial or hands. This has convinced me to just use OpenPose and contribute back to an open project and maybe tie in some smoothing specific ML. Of note: we also use rokoko- much smoother but still needs cleaning. I’ll see if I can’t post a video on how our mocap compares. Edit: Oh wow, looks like OpenPose even has a Unity plugin.

@tabby842 2 жыл бұрын

dunno if you're still using mocap, any changes on your opinion of deepmotion 7 months later?

@LloydSummers 2 жыл бұрын

@@tabby842 yes it's still quite unusable. I ended up cloning OpenPose (open source) and using that for a bit We actually have Rokoko suits, depth sensors, and VR equipment - so we have a lot more options than most folx. I think the pricing is unreasonable given the number one reason for needing to reporocess captures is poor coding on the Deep Motion team. There are deep motion alternatives now though that are cheaper and better at processing that you should check out. Just google for non marker motion capture solutions :).

@DB009 Жыл бұрын

@@LloydSummers ive been trying to get OpenPose to work for a while now on unity not working unity just keeps crashing.

@joprecious4570 2 жыл бұрын

What about brekel

@RR-gx4ec 2 жыл бұрын

'Ad' should be in the title - clearly visible from homepage and search results.

@razmoose213 2 жыл бұрын

This would be every small time vtubers dream.

@canalactivado 2 жыл бұрын

But don't the Vtubers already do it? D: I see some of them move their heads and some others move their fingers. *Don't they already do it with AI? Or with DeepMotion?* D:

@viktorchernyk 2 жыл бұрын

@@canalactivado probably Facerig

@waters.create Жыл бұрын

I think you've just saved me a few 1000 pounds here

@nicolaastanghe475 Жыл бұрын

anny multicam options ?

@grimsk 2 жыл бұрын

should know that it was the year for giant steps in motion capture..

@n1lknarf 2 жыл бұрын

The problem with Ai driven software for mocap is the lack of depth recognition, especially if recording with only 1 camera. Great for 2D animation but very bad for 3D as it won't recognize bones moved in the Y axis (front and back)

@GS-tk1hk 2 жыл бұрын

But that's the thing about AI, it can estimate depth from a single image by inferring it from, for example, the way objects shrink with distance or the parallax effect. If you close one of your eyes, you still have a pretty decent depth vision despite it being just one image.

@canalactivado 2 жыл бұрын

@@GS-tk1hk I don't quite understand it. Before, mocap suits were used and now AI is being used D: It will be possible to make animations with the AI without mocap suit?

@GS-tk1hk 2 жыл бұрын

@@canalactivado It already is, but it's not as good as mocap suits. I think a company called DeepMotion is working on it, among others. Basically the AI recognizes the human and how he/she moves in space, and translates that to virtual bones for animation

@bronationgamingultimate6760 2 жыл бұрын

im not sure If it does have support or not but you could probably use the xbox kinnect V2 with the depth sensing tech it has.

@laupoke Жыл бұрын

Yeah it can lol, wtf are you on about ? The whole purpose of AI is to make extermely good educated guesses. If it was trained on data that move on the Y axis (and it was) then it can estimate that. As a matter of fact, i'm pretty sure you would be able to tell if every point of every bone was in the same plane. This comment is dumb.

@JadenIrons Жыл бұрын

2:11 is that Chika dance?

@wensongliang1905 2 жыл бұрын

惊现闪电五连鞭

@user-ux5hp9nc8f 2 жыл бұрын

Very cool

@ATomCzech Жыл бұрын

Why there are just two extrems. Super expensive special suite and then attempts to track motion completely in any clothes. Why not use something in the middle with still extremely low price as some usual joga suit with some markers on it, which makes tracking much more accurate.

@sdfrtyhfds 2 жыл бұрын

I don't think you can compare 2d solution to a 3d one. I don't know why you are taking this approach when you already stated there are open source 3d pose estimation models. This comparison also doesn't make much sense given these advantages depend on your use cases

@bycloudAI 2 жыл бұрын

Seems like I wasn't clear enough in my video. They might have slightly better or worse models to estimate 3D positions but at the end of the day it relies on monocular 2D estimations first, then convert into 3D. The same goes for the commercial options too. Essentially, they both relies on 2D estimations, it's just that the visualization for DeepMotion is able to be presented in 3D. That's all. A comparison video is literally about testing each of them in different use cases. This video is just to demonstrate to people the difference and similarity so that everyone know what they will get out of them when they use these technologies.

@beefquiche Жыл бұрын

I've been working with mocap for 18 years. Large studios won't adopt these video based alternatives as they can afford vicon systems and a team of animators to beautify the input data. But this progress is exciting. Maybe by ~2030 the tech will be precise enough for AAA budget games.

@MrGTAmodsgerman Жыл бұрын

Looks like they lowered the amount of seconds you can upload for free accounts. This is now only 10 seconds!

@3dmax_Ani 2 жыл бұрын

@danielyeh1627 Жыл бұрын

*闪电五连鞭卧槽*

@McHornet7 2 жыл бұрын

Alot of bugs in the motions

@monster.3ds Жыл бұрын

Best mocap is x box 360 kinect

@dalemsilas8425 2 жыл бұрын

Their pricing rates are weird. How about they just straight up charge per minute. No " bundles". Like 1 min vid = $2, straight forward.

@liverookie2145 Жыл бұрын

Update: they give free 60 credits now and 30 second video lengths for free

@remainder_ 2 жыл бұрын

How many coding languages do you know

@Tsero0v0 Жыл бұрын

閃電五連鞭🤣

@user-cw3nb8rc9e 2 жыл бұрын

Great study. It looks like both solutions suck unfortunately.

@sam_s3344 Жыл бұрын

Anyone here who's tried the deep motion?

@fuzzyhenry2048 2 жыл бұрын

闪电五连鞭和蔡徐坤绷不住了🤣🤣🤣

@keisaboru1155 2 жыл бұрын

AHHHHH NVIDIA card t,t

@jiarunlu9050 Жыл бұрын

哈哈哈哈闪电五连辫遭不住了

@YongWu-ut8yr Жыл бұрын

马宝国

@themightyflog 2 жыл бұрын

Deep motion is to expensive and charges you for even bad captures.

@DeepMotionInc 2 жыл бұрын

We just added a new feature that automatically fails videos that do not meet a certain animation threshhold. We are also adding another new feature in the coming weeks that allows you to rerun a video that passed with new settings.

@shawnarms16 2 жыл бұрын

@@DeepMotionInc that is good to know. But with the price and quality I can buy a Rokoko. I think the pay options are limiting and should include more minutes. But that is just me. I’m doing a training course using deepmotion. It’s pretty cool so far.

@noedits5543 2 жыл бұрын

Don't worry. Within a year someone will come up with an easy free solution. 😀 Then you will scratch your head for paying folks for free stuff. Never pay anyone unless youre earning tons using that. Rather Use open source.

@DeepMotionInc 2 жыл бұрын

@@noedits5543 There are open source options out there right now and we welcome any and all comparisons to our service! We have a team that is continuously hard at work optimizing the animation quality and adding new features that provide added value, releasing a stream of frequent updates every few weeks.

@ThiagoBarbanti 2 жыл бұрын

no way, with a lot of drifts, pose issues and wrong rotations we will not see these solution on movies or games

@daton3630 2 жыл бұрын

yeah

@canalactivado 2 жыл бұрын

In the future it will be possible to make animations with the AI without mocap suit? D:

@nikhil22bhardwaj 2 жыл бұрын

Somewhere a blenderdev is watching it and...

@Rctdcttecededtef 2 жыл бұрын

…using it to create tasteful furry “videos”

@user-ci2bn4zx8m 2 жыл бұрын

不讲武德

@bhuwannepathya 2 жыл бұрын

@DrMattPhillips 2 жыл бұрын

I think a sponsored video should be more clearly disclosed, I watched the entire video and yet it wasn't clearly confirmed to be sponsored by you directly until reading the description and even then I had to click the 'show more' button which legally within my location wouldn't qualify to digital advertising regulations. You can also tag a video as an advert, of which this one very much felt like to me. I like your work, but I don't feel you were very ethically transparent at all with this being an advert, the only transparency came in the way of knowing the social cues of when somebody is trying to sell you something, but beyond that it'd be very easy to miss, and I feel you really dropped the ball there. Even a 'thank you for sponsoring this video' at the start and end would have felt more transparent when mentioning DeepMotion, as by sponsoring they are by definition being given preferential treatment, of which needs to be clear.

@crescentsmile3463 2 жыл бұрын

He says it verabally at the beginning of the video and there is also an overlay on the actual video saying it contains a paid promotion. Sooo... maybe you just missed those?

@DrMattPhillips 2 жыл бұрын

@@crescentsmile3463 The overlay wasn't there when I made this comment, he added that afterwards. Equally, the description was different too and wasn't as clear as it currently is. I find it odd that you didn't consider this as a possibility. As for the disclaimer in the video, I listened to the opening several times before making the comment, and I didn't find it clear enough. I never said it wasn't disclosed, just I felt it wasn't disclosed in a way that's transparent. However, I'd retract that now that he's made the two main and most important changes with the overlay and the description change. I feel these changes make it ethically transparent, and I'm glad he took such a perspective constructively rather than dismissing it or assuming bad faith like you did.

@crescentsmile3463 2 жыл бұрын

@@DrMattPhillips He literally says 'DeepMotion is today's sponsor' in the video... so...?

@DrMattPhillips 2 жыл бұрын

@@crescentsmile3463 There's something known as digital advertising standards, mentioning it 1 minute in isn't considered transparent nor contradicts what I stated. I have no idea why you are still pushing back against my fair constructive criticism when he clearly read it and made the correct changes I suggested, thus making my initial point resolved and making the content better for it. There's literally nothing negative here beyond your inability to respect a fair comment, made with good faith, not even directed at you specifically. It's bizarre.

@crescentsmile3463 2 жыл бұрын

@@DrMattPhillips "I watched the entire video and yet it wasn't clearly confirmed to be sponsored by you directly until reading the description" - actually mentioning it in the video does contradict what you said in your original comment. You're trying to throw shade at a dude and I'm just giving the facts.

@rafadananimasyon5274 2 жыл бұрын

Can you convert a 2D image to a 3D model for me for free?

@m.filmtrip 2 жыл бұрын

Ha ha

@noedits5543 2 жыл бұрын

I can convert a real human being into an animal for free.

@rusticagenerica 2 жыл бұрын

Bodysuits suck.

@nanaouzumaki6044 2 жыл бұрын

blender users are done for..

@BOSSposes 2 жыл бұрын

sure ... pose the face go ahead make 1 shot of a project with that horrible rig

@yimur5062 Жыл бұрын

all of these ai things is just bullshit..

@yimur5062 Жыл бұрын

chatgpt: wait a sec..let me explain: Motion capture (sometimes referred as mo-cap or mocap, for short) is the process of recording the movement of objects or people. It is used in military, entertainment, sports, medical applications bla bla bla..

@radekoncar2404 2 жыл бұрын

Label this as ad in the title. Worthless.

@buisnessmahn6642 2 жыл бұрын

I mean, did he not say it was sponsored in the video? When you watch a movie or a TV show they don't have their sponsors in the title or upfront. That just doesn't make sense. And worthless? You just weren't paying attention

@hamsolo474 2 жыл бұрын

I mean how dare he talk about a service with a free offering and explain he's sponsored by them and then spend half the video talking about a free alternative to them.