USE YOUR HANDS IN VR | Hand Tracking Tested | MSFS

USE YOUR HANDS IN VR | Hand Tracking Tested | MSFS | OpenXR Toolkit with UltraLeap

Рет қаралды 44,294

Күн бұрын

OpenXR Toolkit features hand tracking, for cockpit interaction with just your hands. In this video we put it to the test in Microsoft Flight Simulator with the LeapMotion Controller from UltraLeap.
REFERENCED LINKS
Essential VR Series Playlist: • ESSENTIAL VR
OpenXR Toolkit: mbucchia.github.io/OpenXR-Too...
UltraLeap: www.ultraleap.com/
SIMHANGER KZfaq Channel Link: / simhangerflightsimulation
Join Simhanger on Discord: YOUR DISCORD INVITE: / discord
For more details visit Simhanger Website: www.simhanger.com
Connect with me on Facebook, Instagram and Twitter: @Simhanger1
✈ Would you like to support the Simhanger Channel? You can donate here: www.paypal.com/paypalme/simha...
Everything you wanted to know about VR - Playlist here: • ESSENTIAL VR
✈ You could also buy me a coffee here : www.buymeacoffee.com/SIMHANGER
Looking for help with MSFS, then check out my HOW TO GUIDES, link to playlist here: • MSFS HOW TO GUIDE
MSFS on Xbox playlist can be found here: • MSFS on XBOX
Thanks for watching
#xbox #microsoftflightsimulator #virtualreality #msfs #vr #cloudgaming

Пікірлер: 103

@mbucchia 2 жыл бұрын

Great video, and the first of a kind to cover the hand tracking with OpenXR Toolkit!!

@SimHangerFS 2 жыл бұрын

Thanks Matt, love your work 😀 and thanks for what your doing for the VR community.

@jeanfrossard 9 ай бұрын

Hi!!! I use the "Quest 2" for MSFS 2020 and ended up buying the Leap Motion so I can use my hands on the controls. Was it foolish to purchase the Leap Motion? hand-orange-covering-eyes I tried using the OpenXR Toolkit, but was unsuccessful. face-turquoise-covering-eyes In Quest 3, does leap Motion help with hand tracking or is it already native is sufficient? Hugs from Brazil!!!

@SimHangerFS 2 жыл бұрын

Let me know if your interested in this topic below, thank you 😊

@phunkaeg 2 жыл бұрын

amazing! I had seen some videos of people using hand tracking in msfs using Oculus Quest 2 passthrough. But without any instructions on what to do. This is exactly what I was looking for - Thank you!

@4secsharkmr352 2 жыл бұрын

Excellent video Mark👍 I got a handtracking addon for my 8KX and it was okish for starting or shutdown a plane. In flight I used my normal throttle and joystick. That was before OpenXR Toolkit. I had to modify some SteamVR files. Nowadays it seems to be lot easier and while in flight you are now able to disable the handtracking. What is great because seeing the whole time the clunky controllers was annoying😁 Thank you very much for this and take care.

@SimHangerFS 2 жыл бұрын

Agreed, concept is perfect, but implementation some way off. 👍

@darioinfini 2 жыл бұрын

Watching your video made me come to the same conclusion -- best use as pushing buttons and tweaking knobs, leaving throttle and yoke to real world gear.

@AuthentiKit 2 жыл бұрын

Well done Mark - great video. Somewhat leading/bleeding edge stuff but I am very hopeful that this will get slicker and become mainstream soon. It is an integral part of my vision for Authentikit- once hand tracking is smooth and accurate i feel that people are going to get more interested in replica controls that are in the right place. So well done on helping to bring this tech into the mainstream.

@pm9601 7 ай бұрын

Awesome video. As you said the tech is a bit finicky at the mo, but you can see where this is going and it will be awesome in a few years!!!

@rembratt 2 жыл бұрын

Thanks a lot for showing this tech. I was considering it but this clearly shows it needs a few more years of development.

@dabneyoffermein595 Жыл бұрын

no doubt about it. very, very rudimentary. if there's a way we can have our controls light up in the goggles it would be SO much easier. All they would have to do is have a template that aligns with say our Bravo throttle controller just so we can know where the buttons are , the throttles are, etc. Right now I make do but specific controls being lit up in our goggles would be so cool and when we touch or turn or push them, they would get brighter, flash, etc. Yes it would require some ingenuity, but we live in a high-tech world and OpenXR Toolkit could handle this type of thing since it's a API layer on top of OpenXR base product, which is in-turn, on top of the SIM itself. Lots of mapping and controllers involved in that, but the SIM has to include all the popular ones anyway, so there is already a cross-reference to all the dials, buttons, etc.

@tobytoxd Жыл бұрын

Wow, great invention! Thanks for sharing!

@garryadams67 2 жыл бұрын

Totally agree with you. Still lots of work to be done but shows good promise for the future. For now I will stick with my Yoke and Throttle quadrant 👍but I can see a future for this as you say may be one hand for knobs and switches. Great review Mark ad ever. Cheers Garry.

@SimHangerFS 2 жыл бұрын

Thanks Garry. 👍

@SimsSmith 2 жыл бұрын

Great video. Totally agree, hand tracking looks promising but still too painful. The technology needs more time to mature so tracking is more precise. I will still stick to physical hardware controls (HOTAS) in VR mode.

@VRFLYBOB 2 жыл бұрын

Without the tactile feedback, I reckon you feel a bit like a ghost pilot. Imagine if you could buy a box of magnetic switches and dials and a thin metal plate to attach to your desk, then print out a control template for any plane. Plus hand tracking would be pretty immersive!

@WarhammerQuestAdventurersGuild 2 жыл бұрын

EXACTLY! I have had this concept for years. If there existed a way to "sync" VR hands with your real world hardware the possibilities would absolutely ground breaking amazing! Perform the function just as you stated. A VERY inexpensive set up could be created (compared to what is required to invest currently) and packages could sold. You would NOT need any of the actual gauges. All you would, need to have a complete basic GA aircraft, would be Yoke, Pedals, Throttle, and how ever many knobs you wished to purchase to replicate the panel of your choice. When in VR, all you have to do is reach out with your VR hand and your real hand would actually FEEL the knob and turning the knob would perform it's proper function. Whatever company creates this first will WIN and dominate the market. They would certainly have my money!

@truthandunity623 Жыл бұрын

What about haptic, force feedback gloves, which would mean you don't need any physical things to hold a yoke etc.

@DinkyDoos 2 жыл бұрын

My new Reverb G2 came today, replacing my Quest 2 , what a difference.... well worth the money

@larue085 2 жыл бұрын

Slide the XRopen to 115% resolution and Open XR toolkit at 2300 x 2200 resolution and max out most settings in DCS but Select flat shadow and clouds you can increase in VR on the G2 as well. Leave the dcs at 1.0 pixels always adjusting the slider on openXR and the XR toolkit. Not a fan of motion reprojectiion on my rtx3080

@martinschmidt5541 Жыл бұрын

Very good Video...

@happymealchiz 2 жыл бұрын

This is really neat. Have you tried the game VTOL VR out? gives an experience similar to this but you still use VR controllers. It's a brilliantly fun and extremely detailed game nonetheless

@Aethid 2 жыл бұрын

That isn’t a proprietary connection on the leap sensor. That’s a standard MicroUSB 3.0 port. You don’t see it very often, because it is larger than the 2.0 microUSB, but not many devices needed the extra speed of USB 3.0 at the time, so most just stuck with 2.0. It was superseded by Type-C ports by the time 3.0 speeds were needed.

@ericveschi4904 2 жыл бұрын

Few months ago I was thinking to buy not our version, but the bar version because the field of view is better and there's 3D printing pieces who exists for HP Reverb G2 to avoid to past the censor directly on the headset set. If you don't use it, you just need to remove the 3D printing part. The pity in VR, is that we don't have a virtual keyboard when we need to make texts input on some cockpit tablets. Asobo seems to not take VR seriously. Because they didn't yet make corrections on VR windows menu, we need to stretch them every time, it's little bit annoying ^^

@SparkySven 2 жыл бұрын

nice show that it is possible but not usable just like the VR controllers that come with most VR headset. nothing can compete with Honeycomb Alpha & Bravo and good mouse in VR. But hand tracking should be further developed and keep testing and maybe one day it can be uses like will all want effortless. Great video Mark. I also flew over NZ mesh with the newest version of DC Designs F-15 v1.0.8 which made this jet Sparky fantastic and was joined with VR Pilot Jose in a F-16 and his wingman in an F-18. This NZ area is mind blowing pretty. No time to do hand tracking but like in top Gun, Don't think just do ;-)

@SimHangerFS 2 жыл бұрын

Hi SparkySven, I agree with you. Great idea, poor implementation.

@dj53144 Жыл бұрын

Great video. What is your recommendation for operating controls while using a a hotas (in VR)? The mouse is really clunky, Can one of the hp reverb hand controllers be used just to set things?

@oldgamerdufe2595 2 жыл бұрын

While hand tracking does not do it all, adding a yoke/throttle/rudder peddels help. I would put forth that voice commands really help. VoiceAttack is what I use.

@SimHangerFS 2 жыл бұрын

Agreed, VoiceAttack is excellent 👍

@adsilite Жыл бұрын

I wonder if the desktop or monitor setup would ease the operation since you don't have to look for your hands constantly with your headset and the pov would be a bit higher and further. In any case, thanks a lot for your videos, sir. Very, very helpful.👍👍

@ErikMiddeldorp 2 жыл бұрын

I wish I could use hand tracking for the switches etc and hotas/yoke/throttle and pedals for the flight controls. At the moment my msfs control setup is a mess; a mix of hotas and rudder pedals, Xbox controller for the drone camera, keyboard, mouse pad, and vr controllers. Plus using the vr controllers or mouse to interact with cockpit controls doesn't always work and is finicky. This video made me wonder if it was a mistake to sell my leap motion controller a couple years ago, but if quest 2 hand tracking works, maybe I'll try that.

@VRFlightSimGuy 2 жыл бұрын

Excellent demonstration Mark although I tend to agree with you it looks a tad cumbersome in places (although I've never tried it myself). I wonder if this technology would work with the Quest 2 hand tracking 🤔 Regardless this certainly shows promise in the future. Thanks again for the video and it's great to see VR content on your channel.

@SimHangerFS 2 жыл бұрын

Yes, compatible with the Quest and Quest2.

@patrikstrandquist1875 2 жыл бұрын

This looks very interesting, im a DCS player, i would love to see how it works there.

@therealcyril Жыл бұрын

It's already useable when it comes to MFD / UFC push buttons. Harder, much harder to interact with side consoles...

@NovaGeddon771 Жыл бұрын

It’s been 8 months since this review. Have you noticed any improvement with OpenXR Toolkit and hand tracking? Your videos are amazing! I really enjoy listening to your voice and what you have to say.

@SimHangerFS Жыл бұрын

Not yet

@DenisFerrari Жыл бұрын

Very interesting. But I can't manage te remove the pointer. I try to tap the index fingers, but nothing happens....

@rcc5947 10 ай бұрын

Great video. Have you tried hand tracking in X-Plane 12?

@SimHangerFS 10 ай бұрын

Not for many years.

@nmglastron 8 ай бұрын

Hi Mark, any plans on reviewing the Ultraleap version 2 which released recently?

@xMilesxHighxClubx 2 жыл бұрын

nice

@R3ALTR1LLK1LLA 8 ай бұрын

Can I use my valve index controllers for this?

@Verbal_DEU Жыл бұрын

How to turn off the long white line? Do you have a config for the openxr to share pls?

@daveowens7613 Жыл бұрын

Hi Mark, was wondering if you used the controller on the desktop or did you use the headset mount? Would be nice to know before I buy. Many thanks for a great series of videos that cover all aspects of flight simming.

@SimHangerFS Жыл бұрын

I used the headset mount

@piersrichards5212 6 ай бұрын

Does this work with the Quest 3's hand tracking without ultraleap?

@mr.phileo9699 Жыл бұрын

can I use the leap motion with the vr controllers?

@buikhan 2 жыл бұрын

Can I still use my honeycomb alpha and bravo with this on?

@adafonseca4990 2 жыл бұрын

Hi Great video unfortunately I can'find in Europe the motion controller with headset mounting kit any help????Tks

@SimHangerFS 2 жыл бұрын

I purchased mine directly from the states.

@aaronshahi2402 Жыл бұрын

Hi, and thanks for the great video. I have followed every single step, but "controller emulation" doesn't show up in my "input" tab! Have you or anyone here had the same experience? I would really appreciate your help!

@SimHangerFS Жыл бұрын

Ask the devs directly, they are very responsive.

@blacknass1943 2 жыл бұрын

its to tricky and you cant feel it ist very hard to play wif it

@nunzio_p1172 Жыл бұрын

Hi...first of all.. thanks for your videos..I can't configure leap motion in the best way with my quest 2 .Could you show your configuration?

@SimHangerFS Жыл бұрын

You need to use the OpenXR toolkit and enable hand-tracker within the app.

@nunzio_p1172 Жыл бұрын

@@SimHangerFS Thanks for your attention...yes I know ..but it's just the configuration of the hand movements that I can't do better ..maybe it's possible to find a best configured FS2020.cfg on the net for QUEST 2😊😇

@SpatialGuy77 2 жыл бұрын

Interesting topic, thank you. I didn’t know that! One point from the start of your video, you stated the Reverb H2 as the best headset for the price on the market. You didn’t say it was YOUR OPINION (not a fact), and it’s an OPINION I certainly do not share!

@SimHangerFS 2 жыл бұрын

Fair point, I meant in my opinion…what’s your choice?

@imacmill 2 жыл бұрын

I'm echoing SHFS's question: what's your choice for best VR headset for the price?

@SimHangerFS 2 жыл бұрын

@@imacmill hp reverb g2

@imacmill 2 жыл бұрын

@@SimHangerFS My question was directed towards the person that replied to you (SpatialGuy). 😊

@ooDemiurgus 8 ай бұрын

I'm not seeing hands and I don't know how to fix it. I only see my index controllers. How do I switch it to hands?

@SimHangerFS 8 ай бұрын

Make sure your leap motion software is updated. Also subject to what your headset firmware will allow.

@bjrnchrstn 10 ай бұрын

Can you make the hands visible but not functional in vr? I just want to use them for alignment with my own simpit.

@SimHangerFS 10 ай бұрын

I don’t know

@levelat350gaming Жыл бұрын

I have a question about the item here. I own the Leap Motion Control sensor and an Oculus Quest 2. Do I really need both a Pimax Hand Tracking module for my Leap Motion Sensor to work?

@SimHangerFS Жыл бұрын

No you don’t need the Pimax module.

@rogerferraro6661 2 жыл бұрын

Thanks for keeping a finger on the bleeding edge of Flight Sim Mark. I'm interested in VR and how to work the controls in as natural a manner as possible and hand tracking looks like it might be able to do it if the tracking capability was built into the headset. I have one of those LEAP controllers hiding in some bin somewhere from years ago, looks like it's time to dig it out.

@desert1791 Жыл бұрын

Hello Mark Merry Christmas To You And Your Family....There will be any VR headsets in 2023 Beat HP Reverb G2 V2 ? Thanks

@SimHangerFS Жыл бұрын

Varjo Aero has best visuals, but very expensive. Pico 4, but you need a 4090 to run it….so Reverb best bang for your buck now. Waiting to see what Valve come up with to replace their ageing headset later in 2023. See here kzfaq.info/get/bejne/fLKkjJx3r6nMlJs.html

@desert1791 Жыл бұрын

@@SimHangerFS THANKS

@AT-kn2dc Жыл бұрын

Hi great video , I installed all as shown, checked the API layers which are ok, but on openXR toolkit there is only FS2020 application but not "Unity Application (Ultraleap Control Panel.exe) ! What did I wrong ?

@SimHangerFS Жыл бұрын

Check if you have the latest version.

@fingerhorn4 Жыл бұрын

I've tried this in various forms and for me it has been a complete disaster. It just makes things in VR even more complicated than they already are. You cannot beat a decent mouse. I know devs are doing their best introducing ever more "sophisticated" methods of cockpit interaction but there is a reason why using the mouse prevails with the vast majority of 2d and VR users. The mouse works. It is almost totally reliable and avoids yet another layer of complication on top of all the other things that can go wrong. I see by your noble attempts that the skeleton hand is very much worse than grabbing a control with the mouse in either MSFS legacy or modern mode.

@SimHangerFS Жыл бұрын

Agreed, a work-in-technology-development

@Dg-zj6jo Жыл бұрын

hi mark dose it work in x plane dcs please

@SimHangerFS Жыл бұрын

DCS I think not, Xplane I am not sure I haven’t tried.

@Dg-zj6jo Жыл бұрын

ok cheers sir

@reg_inforeg_info3691 Жыл бұрын

I don't think that's proprietary connector - that's just a regular USB 3.0 Micro B

@gianninanino746 2 жыл бұрын

Thanks for your video. The progress that is being made with VR is truly impressive; we are certainly only at the beginning of the interaction with the cockpit and we will certainly have great news in the future. Watching your video I have a doubt concerning the lack of physical reactions by touching the yoke or the power levers. What do you think about this aspect?

@SimHangerFS 2 жыл бұрын

Agreed, a combination of virtual and physical interactions seems the most practical.

@chiefbrody3406 2 жыл бұрын

Minority report has finally arrived

@SimHangerFS 2 жыл бұрын

🤣

@123thedman1 Жыл бұрын

Great vid, but had you inserted video of your real hands it would of been better.

@juniorwizzard 2 жыл бұрын

Oh. 3.000 euros for the "ONLY" required hardware. Not ready for that! Lol

@oldmanballs 2 жыл бұрын

Imagine being a passenger and the pilot has skeleton hands!

@SimHangerFS 2 жыл бұрын

🤣 time to look for that parachute!

@imacmill 2 жыл бұрын

I have concerns that hand tracking will never become anything more than a gimmick.

@SimHangerFS 2 жыл бұрын

Progress over the past 5 to 6 years has been slow. 👍

@SimHangerFS 2 жыл бұрын

Yes, great concept, but practical implementation too limiting 👍

@dabneyoffermein595 Жыл бұрын

agree, read my above post under "Jonas"

@zipobibrok 4 ай бұрын

no more handtracking on openxr toolkit homepage!!!!

@265petsar 2 жыл бұрын

It's heading in the right direction but is far too unrealistic and doesn't look great visually. The skeleton hands dominate the cockpit visuals to much.

@SimHangerFS 2 жыл бұрын

Agreed 👍

@JackoJ9448 8 ай бұрын

So that nobody invests in a technology that is no longer supported: According to the developer, support has been removed from the OpenXR Toolkit because hand tracking never worked properly.

@hanswurst3811 2 күн бұрын

why is there this stupid virtual controller when you have a pointy finger to push those buttons ????

@deutschegeschichte4972 Жыл бұрын

Great video, however there was some straight lies in the beginning of the video when you said that was the best VR headset you could get because different games work better for different people with different headsets so its really just a preference thing rather than statistical.

@davidh7280 2 жыл бұрын

Too much fiddling and faffing. Just use a voice controlled first officer, Multi Crew Experience and a your there.

@jJustPlayingNZ 2 жыл бұрын

This is the future of VR control. Great stuff. Just a pity that, for me, the OpenXR Toolkit is unusable in VR. I have tried and tried but the fact that I have to use the keyboard in VR has proved impractical for me. I wear glasses but use inserts for VR and I have found it frustrating in the extreme to be swapping VR headset on and off and glasses on and off and really I need a non-VR interface to Open XR Toolkit so I can set everything up before entering VR but there isn't one. I understand this is just a personal problem for me but I know there are other people out there that agree with me.