"computation time takes 10 minutes..." Oh okay "...per frame" Oh

The hand massage footage may be the most realistic looking cgi I’ve ever seen

This is freaking amazing, it shouldn't take too long for this algorithm to be optimized to run in real time smoothly

Me at 3am watching this random video and not understanding anything: I like your funny words magic man

I would be interested in knowing how the camera/lighting quality affects the algorithm outcome.

dang, i was really hoping i could finally have a virtual hand-washing simulator :(

3:28 imagine that happening during VR surgery

This really makes me appreciate the ultimate realism of the physical world.

I was staring in disbelief until the computation times were mentioned, but this is still a marvel to behold. I can't wait for the next advancement or two. 🙏

They made heaven for Italians to talk to each other

I really got to “Hand” it to them, this is incredible

Karoly, your optimism and passion is so refreshing in the current state of everything. Thanks so much for this channel and all the effort you put into it.

I tried to hold on to my papers but my fingers went 3:27.

Can't wait till the frame times are in the ms, so I can finally give somebody the finger and properly slap people in VR.

As someone who loves posing hands on blender, this was such a pleasure to watch! We now have accurate hand tracking (albeit very very early and time-consuming) - faces next. I won't be surprised if we'll experience Ready Player One in the near future.

This is so good. This is the first time I've found videos from a guy who's at the absolute cutting edge of his field, sharing everything as it happens. I feel like I've learned more about the state of the art in the last five minutes than I ever knew before.

I didn't realize the "uncanny valley" applied to hands. I'm blown away by how real this looks.

Imagine playing a game with this and seeing your hands dismorph like that. The potential for body horror is endless.

When Sign language is subtitled in Vr games, then the future has finally arrived.

The hand massage part was so realistic it convinced me to massage my hands

Now I can hold on to my papers in VR!

This would be amazing for sign language in VR

Unbelievably impressive result. Very excited to see where this work takes us in the next couple of years as more researchers turn to VR while it continues to rise in popularity.

I think for the purposes of VR, a lot of the things in this simulation could probably be simplified and it would still be a great advancement

3:22 Right - A level Italian Left - Fluent Italian

I like how they had to say "Fingers in incorrect positions" at 3:27 as if I didn't notice what is gonna scar me for life

i have never seen any tracking footage as realistic and accurate as this, including facial expression tracking. And all this with just a camera, no tracking points or lights or special sensors, really amazing

That's mindblowing. Really impressive work! Congratulations to Breannan Smith, Chenglei Wu, He Wen, Patrick Peluse, Yaser Sheikh, Jessica Hodgins, and Takaaki Shiratori for the incredible paper.

Isn't it way cooler to be alive while all the cool stuff is being created than living when we'd take it for granted

I was an early adopter of the leap motion as well as VR, seeing these improvements just BLOWS MY PAPERS AWAY. It may not be even close to real time, but it is incredibly impressive and we're well on the right track.

Baby Gamora: Is it done? Thanos : No Baby Gamora: What did it -- wait what ? Thanos : It's still Rendering. Edit: Shouldn't you guys be Holding on to your papers? ⬇️ People who didn't held their papers.

First time I’ve watched a 2 minute papers and literally gasped out loud. This is unbelievable.

This paper contains a lot of hand-waving.

Before I could type in "better hand tracking for vr UE4", this shows up at the top of my feed. Perfect timing!

Absolutely amazing! I can't wait until you can accurately render virtual me sitting at a terminal coding for 8 hours a day and unwinding by playing Zelda II on an NES emulator in my emulated OS on my emulated computer in my virtual reality

As a game designer I can't wait till VR games use this so I can mess around with it and see how it works

I missed that second hold on, and my papers went flying everywhere.

As a VFX artist seeing the hand massage footage blows my mind. That looks so freaking believable even without any hand textures and just plain purple color. Can't these papers be turned into softwares, even if they are paid?

One of the nice things abou the Valve "knuckle" controllers is that they allow you to let go of them and still have them close at hand. Then when you need to pick up something, you actually are gripping a physical object. So VR without a controller isn't as interactive, even if the hand simulation is better. And when it comes to finger simulation, those knuckles does something that I consider good enough. You can have semi-reliable 5 finger simulation. Once the controllers move to their next generation, they will be even better.

I’m absolutely blown away by not only the hand simulation, but also the perceptilabs ad. For once it’s something I think I’ll check out as a developer that’s used to learning things in a hands-on, experimental way and seeing the results. Thanks for not only sharing high quality papers, but also high quality ads :D

Amazing! And no gloves involved ... that's incredible. I'm curious to see the improvements 2 or 3 papers down the line : )

This could be useful for animators that are already used to spending hours to render each frame

"Doctors could be trained to perform surgery in a virtual environment" More like doctors could train a learning algrithm to perform surgery.

I've been watching the channel for a whole, and it has to be the coolest channel on this site. Especially with how much research and writing I've been doing for college, it can get really difficult to separate my love of learning from the stress of student academia, and it can also be really hard to find the time to keep learning for the sake of it, but Two Minute Papers never ceases to keep the fire burning. Thank you :)

Wow, I believe I witnessed the beginning of a revolution! Thank you Károly for presenting it, it looks great!

That 'hold into your papers' icon is wonderful!! 😃 🎉

The hand massage part is literally ASMR for CG AI researchers and everyone who's ever worked with bone systems in blender and such, it's so damn satisfying, i can almost touch it lol

I'm over a year late watching this video and I don't work in any of the fields related to your work, but I wanted to thank you for your videos and content. As a child of the 80s, watching movies like UHF and Flight of the Navigator, I am constantly amazed by the march of progress. "What a time to be alive", indeed! Keep it up, good sir!

Jaw dropping! Oh my the future VR/AR is going to be amazing.

So much progress was made in just 2 months?! I feel like I am living in a science fiction movie!

I absolutely love how you present these technologies and techniques, especially the way you understand that overstatements and click bait only take away from the fascinating subjects you cover. Thank you!

“You’ve won, but at what cost” is the perfect quote for this

We need hardware acceleration for this kind of software ASAP 😍 that's going to be the future of vr (as soon as we can have it in real time 😅)

THIS CHANGES EVERYTHING

this may not be close to being able to actually be processed in real time, but instead of a stepping stone, this is more like finally being able to see the other side of the river. The fact that it can be done at all is incredible and I can't wait to see it get used in something

That hand massage reconstruction is so satisfying to watch, it's weird, didn't expect that feeling!

I was sure that those chess patterns were some kind of body painting. Just incredible!

Wow, that's amazing! What a time to be alive!!

This is hands down one of the best videos for this specific topic. You've got to hand it to this guy for showing is this stuff. This stuff will certainly be handy for future tech.

haptic VR gloves with bladders and resistance in them are the future of VR hands right now. they will make you able to actually feel what your grabbing in the videogame world.

That is amazing! What a time to be alive!

amazing! thank you for this excellent video. at last some hope for really high-quality hand/finger recognition/interaction. hoping that this can be both linked with optical body mocap (which is normally hopeless with the fingers) and help body mocap do better with physics interaction (eg pick up up a wine glass or write with a pen). narrative filmmaking needs lots of really precise interaction and it is very tedious trying to do this with wonky and jumpy mocap. another thought: since this algorithm deals with two hands very accurately, what about dealing with two bodies? then we can could have accurate optical (markerless) mocap of the tango or people shaking hands or greeting with deux bisous (when covid rules allow this again...).

The thing about interacting with objects in VR is that if you're not wearing a suit that handles collisions, there's no force holding you back from crushing an object in your hand, smashing a wall or doing literally anything that involves using more space than they give you which results in phasing and breaking stuff. But the thing, about the suits, is, that, they could get hacked to suffocate you to death.

To avoid the complex computations, you can just simulate and save all the simulated hand motions. This allows every hand motion to be saves and avoid the computer hit at the cost of a fairly large filesize for the hand data.

0:56 Can you simulate holding on to your papers??

Everyone: impressed Me: now do fireball jutsu

I can see this being the next breakthrough in motion capturing in movies and games, I cant wait to see what the Entertainment Industry is going to do with it once it is a bit less resource-intensive to use!

This guy is so pure, he talks about VR technology being used for wholesome things like: training doctors, training pilots, reconstructing sign language. But I know the primates on this mud ball so much better, they're going to want it for their naughty VR games for rear running muscles and mammary glands.

If the next paper(s) manages to pull of Naruto hand signs for jutsus I'll be able to die happy. :)

Perfect VR hands, at the cost of your real hands.

As a 3D artist, the UV mapping of the hands almost gave me an aneurism! All the squares are supposed to not be barely deformed! hahahaha

My dad is a writer publisher and translator. About 10 years ago he wrote a short story about getting trapped in virtual reality end it was recently translated into English.

Perfect virtual hands, but at a cost: you lose your real hands

Cant wait to hold on to my papers in VR

The hand has always been a staple of CGI animation technology demos. If I recall, Ed Catmull and the team that would later go on to create Pixar debuted the concept of 3D, polygon-based rendering with a short animation of his hand, which had been covered with tracking dots and used as reference to show how complex polygon-based animation could be.

This would be fantastic for social vr. There’s a huge deaf community in VRChat, and this would make signing in-game so much clearer

I have the feeling we might not even need to model anything in a few years

2:06 I swear I held on to my papers but it blew me two meters in the air anyways.

I cant wait for further development of virtual hands! Ive always been excited about papers on stuff like this.

Szeretem látni hogy egy Magyar ilyen nagy sikert ért el...

3:25 i laughed so hard ahhahahahahahahaha

I want this for motioncapture

I love that you're whispering in my ear about these beautiful VR hands.

3:24 The guy in Far Cry when you have to heal, but he hasn't broken his fingers yet.

Roblox VR Games: "I'm four parallel universes ahead of you!"

Me: *truly interactive nsfw games*

For people interested in working with this kind of technology for example to teach or read sign language. I have done some similar work using the Leap motion controller. One big tip/mindset I can give everyone : the human body has limits, use those to abstract your detection. For example a simple rock-paper-scissors detection. Paper : All you need is the distance between the tip of the 4 fingers and the palm of the hand. For the purpose of "paper", the position of the thumb and space between fingers is irrelevant. You don't need to check every angle on every joint because there's no physical way they can be bent if this distance is at its max. Rock : Same as "paper", but the distance needs to be small instead. Scissors : While your first instinct might be to first check the distance between the tips and the palm, and in addition to that, check the distance between the tops of middle and index finger. You only need to check the distance between tips and palm for the ring and little finger alongisde the *sideways* distance between the middle and index finger. There can not be any notable sideways distance between these two fingers unless both of them are outstretched. (there can be a difference lengthwise, so be sure to keep that in mind). You can use these and similar rules to greatly simplify the detection of gestures people are making. Now of course there are hands that don't play by these rules, and thus will be able to fake some gestures by doing something an average hand is incapable of, like bending fingers backwards. Others may be unable to make the gestures at all. Depending on the application this may need to be accounted for. But this of course wil also be an issue with machine learning unless they feed the A.I. with enough samples of these irregularities as well. I would be very interested in seeing how this system handles such cases as it stands right now. To see how much it can already do and how horribly it would break.

My goodness, this is the most impressive thing I've seen in 2021 so far! Kudos to the researchers!

You can hold on to your papers, virtually, two more papers down the line.

Is this a re-upload?

Vr shooters, racing and space games in vr with this system would just be the most fun

dude I love your work in summarizing AI papers. it makes one of my favourite subjects so much more accessable for me!

2:06 what??? it looks so insanely realistic, imagine what crazy things could be done with convincing textures too, holy moly

This is crazy. Back in the 80s, games became rapidly more advanced and detailed, and I think we're living in another one of those periods with VR.

I was blown away, all of your videos leave me amazed!!! Its great to see so much new AMAZING stuff getting made so fast!

I never comment on these... but... This is just Beautiful! Can't wait for this to advance to realtime! What a time to be alive!

The hand massage is insane

I am excited for the effects this could have on deaf communication via VR!

That text input method of, "Scroll to select your letter" triggered a fight or flight reaction.

watching the hand massage footage was so satisfying it made me realise how tired I am and need a rest. Thank you hand massage footage.