What we need is real physical joysticks and throttle but then add some feedback gloves for touch control. Thats it. There is just so many different planes and cockpit layouts its impossible to simulate them.

I like that idea for the VR instrument Panel.

I’ve fallen down the VR rabbit hole. I am in a near trance running VR (HP Reverb 2) in the H145 as i slowly wander inside valleys and land at the bottom/top of waterfalls at Victoria Falls. Gobsmacked….but always craving MORE when it comes to immersion. Addiction.

Grateful that you are here, always having something interesting to discuss 🙏🏻

I know it's not everyone's cup of tea, but I find Voice Attack with HCS Aviator (MSFS) beta to be the answer - it is configurable and once you've got used to it, helps greatly with the immersion. I use Reverb G2 and the built-in mic on the headset works fine. Please keep up the good work! Regards, Alasdair

There's one thing I can't wait anymore are the gloves. Only that would contribute enormously to the immersion. A simple detection system that doesn't have to be a big panel in front of us would do the trick. Obviously there would not be a tactile sensation like on a real button but it would be very efficient, 100% adaptive and much cheaper.

Brilliant overview, thank you!!

At one point in time, mid 2000s if I remember correctly, there was a product, a tablet, onto which you could place physical keys in any configuration you wanted, and those keys would communicated with the base, wireless. This allowed to move those keys to any position on the base tablet and re-arrange them into any configuration you needed. Something similar would be super useful for VR. A set of 1x1 (foot) base plates, on a back of which you would have a RAM mount ball. Have these tablets connected via movable RAM style arms to the rig, in a position that resembles your cockpit. Then have a set of wireless switches, rotary encoders, etc that you can plop onto those panels via magnets or some other quick attach/detach connectors, maybe even Lego style attachment. It wouldn't be easily configurable across multiple aircraft that you may have in your sims, but if you are dedicated to one bird, this would be a really nice approach I would think. I'm doing something similar with just building multiple button boxes that have, close-enough, arrangement of buttons and switches that mimic the cockpit, or at least the most used functions. Each box is run by multiple arduinos daisy chained via i2c connections, so as to minimize number of USB chords. Not ideal, but you are completely right, VR does not have that tactile feel of the knobs and switches being turned and clicked and mouse is an immersion killer.

Was on about this years ago. Made a few comments on other channels as well. You are correct, It is what we NEED. I envision a system that would allow us to be in VR, "see" our hands, and then enter a set-up mode. You see a transparent hand in the VR touching a specific knob. Then you also see your VR hand. There is a real-world panel in front of you (possibly magnetic?) with your real-world knob. Then simply line the two up. Move the real-world knob to the location the VR hand and knob are and hit a button that tells the system the two are "synced". In essence locking it in place as far as visuals go. Now make the real-world panel with the ability to tilt & turn. It could come with simple panel "sheets" that could act as templates depending on what aircraft you are in. Would not work for everything but I could see a GA version and a commercial version. A potential immediate semi-solution: I remember seeing a glove that had a small button attached to your pointer finger. When pressed and held with your thumb, a line/pointer was portrayed, in the vr, coming out of your VR hand. This could be used instead of having to grab those awful controllers in the middle of a flight, pressing a button to get the line/pointer, pointing to the knob, moving to the window to select what knob you want the knobster to be, etc., etc. OR, better yet, have the hand tracking software recognize an obvious "pointing" gesture which would generate the VR "selection line" emanating from your fingertip. Move your finger around to highlight the knob you wish to interact with, then another gesture for the actual selection of that knob. Perhaps a short stabbing motion with your hand? Something to that affect..... Really can't understand WHY this hasn't been developed yet. All the tech appears to be there.... LOVE your channel!

what we need is Leap hand tracking where you can see your hands in VR and reach for cockpit knobs. wooden panels with just mock up wooden knobs, if you make the cockpit dash out of peg board you can move your fake knobs to locations where they would work for any cockpit setup. Let the Leap motion hand tracking work the controls.

Thanks Mark for your very professional Flightsim info, very nice to see

With a raspberry pico together with mini yoke and rotary with push button I created a "mouse" on a fixed position on my rig. Can easily steer mouse to any knob/button in VR and push/turn them with button/rotary. Works well and does not destroy immersion so much.

Wow, being able to see your hands and move around and touch what you can see in V R is very cool.

What MSFS needs is something that DCS already has: A pointer you use by looking at something, then use buttons on your yoke or HOTAS to select things and turn knobs. In DCS, you just turn off the "use mouse" feature, and then you have a green crosshair in the middle of your point of view. You can assign buttons or a hat switch to left and right click, and turn knobs up or down. It's ridiculously intuitive, and you don't have to take your hands off the controls for most things.

Super cool. This is definitely the future. VR is so much more exciting to me than just screens, and it feels so dumb to have multiple active screens when you can just just a VR hmd. I'm contemplating building a wall like that myself seeing that they repurposed the Honeycomb Alpha as the yoke.

Very interesting stuff Mark .. Its amazing what innovations and idea's people can come up with that's for sure ...certainly a case of "watch this space" Thanks for the video 😀

For DCS I'll follow up what a lot of people have mentioned, VoiceAttack. Very handy, especially when flying my Huey. Giving commands to my co pilot and gunners is really neat. And for the other proper multicrew aircraft (F15, 14, Apache) I'm sure it is very neat. Worth the few bucks for the pro version. I used to use the addon, VaicomPro as well but that was getting funky and I think breaking IC for MP

wow that looks good , very good potential

What is important for me is variability. While I like the idea of being able to reach out and feel a real knob (ha ha - I'm leaving it like that for laughs) - There would then be issue when changing from one aircraft to another - of course the knobs and levers would be in the wrong place. So, a universal solution is required. I like the look of that hand recognition software. If I ever use it I would hope that it is robust and I don't have to put up with it not working properly - in awkward places or with fiddly controls. I'd be curious to see how well that hand recognition software works when reaching behind to get at those controls behind the pilot.

Great vid. I really like what YAW aviation are doing, great for flying schools but like you say the expense rules out your everyday simmer unless they can somehow produce a stripped down version. As a VR simmer the one thing I wish for is the ability to see my real hands and cockpit environment. I think the XR or mixed reality setups like what Varjo, Viperwing and Brunner Elektonic are doing are the future because you get the best of both worlds.

You are RIGHT ON. To be taken seriously by real, experienced pilots, we need good haptic feedback in these VR sims.

Thanks for sharing, that was very interesting!

Many people do not know that there is a second and almost hidden separate VR settings page. It opens up grass, airports, vehicle traffic. etc. FOR VR USERS. It is found AT THE TOP where is says "PC". Toggle the right arrow to "VR". This is NOT the VR settings on the main page. It is a separate settings page for VR only. I found it from a MS blogger after months without it. I think it is under "Graphics". There are two different sets of graphics settings, PC and VR. When you set this, nigh time at airports will knock your socks off. And all this works in all modes including slew. It was actually done right. Meigs for example just comes alive with tons of moving vehicles and airplanes. And the night time views of the airport support vehicles is off the charts.

Winwing has an interesting idea for F/A-18 pilots. They have a VR version of their Phoenix Mip which gives you the three MFDs plus the UFC. While this is extremely limited to only the hornet, it is much cheaper than building your own cockpit.

There seem to be three basic options: 1) a complete cockpit replica with controls dials, switches etc. Ideally using the sort of passthrough tech that Varjo use on their trainers. 2) a highly configurable cockpit stand that you can attach multiple swiches, buttons etc onto. Using magnets, super strong velcro, or maybe some kind of mounting slots as seen with aluminium profile. You then move the buttons to the correct place for each cockpit. 3) really, really accurate hand tracking. People have made their own versions of 1) and they look amazing. But you'll only get that full immersion flying the one type of aircraft. (If someone wants to make me one for my top 5 favourite aircraft, I'll gladly use them😂) Option 2) sounds like too much faffing for most simmers. And probably too difficult to get things in the right position to avoid breaking immersion. It might be an easier option than 1) if you usually only fly 1 or 2 types and you also have something like authentikit for the controls. Option 3) would probably work for me if it is very accurate tracking (the limited option I've tried so far isn't accurate enough). And if you have realistic (ish) hands and arms visible as you interact. The blue ghost hands kill immersion for me. As to the lack of actual switches & buttons; the brain can do a good job of filling in the gaps when it has to. Bonus option) build a Startrek holodeck😃.

I fly sim exclusively in VR, and MSFS, I typically use the VR controller to manipulate knobs and touch screens while using peripherals for yoke/TPM. My biggest gripes are VR windows that don’t save the resizing and position, every time you open something you have to reposition and resize on every flight. The other biggest annoyance is knob sensitivity and the times when manipulating a knob sends the value off to some extreme end of the spectrum completely not what you wanted to do when activating a knob to adjust it. The press in button to activate a feature is extremely clumsy. These all should be solvable with simple software adjustments and allowing the user to set VR sensitivity of a knob, I never had these problems in Xplane, as far as VR knob UI goes it is far superior to MSFS. The Cessna hand tracking cockpit is interesting, but you’d need one for every cockpit you want to fly making that too cost prohibitive and likely you won’t switch them out easily. One of the biggest reasons I went VR is so that I could have a wide variety of cockpits without having to build each one, it’s far cheaper and more immersive.

I had the same frustrations…Can’t show anything yet but literary have a whole new solution to this coming along as we speak.. been a few years in the making and hoping to go into beta tests soon.

Hey Mark I thought of the VR dashboard last year and was working on some concepts. Just the blanked out cockpit with some encoder and buttons. Having the shape of the GNS 530 or similar with encoder and buttons in the right place is very easy to make. But I also discovered that muscle memory is a key factor when finding buttons and dials. I have the RealSimGear SR22 setup and I fly it mostly in VR. I can set a flight plan and everything needed by just using muscle memory and feel. Once you match the distance and position in VR to the actual physical controls, muscle memory takes over. Someone could make some money if they developed a working kit to match different aircraft. I don't know about the heavies but GA would be pretty simple. Cheers!

I actually tried this device at the event and I can say that it feels fantastic. During take off, crusing, and landing really feels like you are flying the Cessna. But at the price I saw, people like me will not be buying this devices any time soon.

I would like to see VR gloves which are tracked like hand controllers are now. In my setup, I placed hooks on my chair and fastened tie wraps to my controllers so I can use muscle memory to grab my controllers and put them back when they aren't needed. While this works, it is immersive disrupting. A physical panel with knobs in the correct places and headset camera hand-tracking is intriguing, but it ties the user to a particular aircraft.

The option I use is a DYI button/knob box. I have enough buttons, switches and knobs to operate any small GA plane. I'm not confined to any particular cockpit layout, I can fly different GA planes. The muscle memory allows me quickly reaching to the right knob or button. For the airliners (I fly Aerosoft CRJ) I add two Behringer X-Touch mini controllers with 8 additional knobs (great as autopilot controller) and Novation LaunchPadMini which works as FMS keyboard interface. For most frequently used commands I use Axis and Ohs voice control (like a commands to a copilot). For maps I bring WMR desktop to the cockpit (to my virtual lap) and my mouse is used only on the desktop. For VATSIM clearances I have Wacom tablet with pen on the same desktop.

I was thinking of using hand tracking with my Pimax crystal to 'interact' with the Virtual Cockpit, as you would with the mouse. Provided this will work.

I like the thinking here. The hard part IMHO is the development of the hand tracking software. It would be ideal for 3D printing designers to develop modular cockpits set out to match their chosen plane (I'm currently doing a F-5E Tiger) with all the buttons in the right place (similar to the YAW setup). Authentikit is a great place to start for this! However, without a configurable hand tracking option, integrated into your flight sim of choice, you will still be limited to your memory for accurate hand placement. Possibly someone has already done this? I'm going to look now :).

Perhaps a knob mounted on a mouse, move the knob over the control you want to adjust in the cockpit, one click to lock it there, adjust the knob then move to the next control. I still think hand tracking would be the ultimate though.

Good video.

force feedback is also a must!

The C-172 interactive cockpit is an incredible concept - now why can I already envision this being part of your own FS setup 😉

i use the top kensington trackball mounted above my bravo throttle quadrant. it works well, doesnt need a mouse pad, has a small footprint and looks good.

I use VR since 2016 and don’t mind using mouse at all. Bought PointCTRL recently to improve “immersion” but it does not work very well in MSFS, even with a custom HTCC firmware. Works great in DCS though, but this is the game it was designed for. A mouse is quicker and more precise to operate knobs and buttons in a cockpit, especially in airliners. For me the biggest immersion killer is MSFS itself, with all the graphical bugs like horizon line, black lines on water, misaligned ground mesh tiles and many others.

Thanks Mark, very interesting thoughts. I saw the Yaw 2 Cessna 172 rig recently, and liked the idea. That is without the ridiculous pricing, which may be doable for professional aviation schools (see ROI scheme) but not for us mortals at home. When you started your vids about VR, I thought about the idea that Yaw now presents; a plywood MIP and some rotary knobs (and Bob's your uncle). Building a cockpit myself, I thought interfacing the knobs to the sim wouldn't be the major problem. Alas, I hate blue hands (also immersion killers) and no clue how to get the knobs lined up in the correct spot (according to the image of the virtual cockpit on the screen). And there is also the "problem" of wanting to be able to use different aircraft. On the other hand, plywood is cheap, so making a few different type of MIP's wouldn't cost a lot. You could even swap the knobs to another MIP. VR sets that also let you see the actual cockpit (mixed reality with your own normal hands) are very expensive and you'll have to stick to 1 full sized cockpit with all the bells and whistles. Can't imagine having those of a 737, Cessna, and what else all together in my home (have only room for half a 737😉). So a development as shown by Yaw, would certainly be welcome.

Thanks

I have been waiting for Somium to release their headset as it has two 16Mp pass through cameras which have been designed with simmers in mind.

Great video and great advocate for the sim community

This concept reminds me of what I am currently building. I am following the youtuber "Heli Mech" and his tutorial on the Garmin G1000. Only I am not using a screen because I'm in VR. Granted, I was lucky to find someone with a 3d printer and I needed to learn the basics of Arduino and MobiFlight, but for a significantly lower price than RealSimGear I am able to have a fully functional G1000 that I can reach out to an interact with. Pair that with Reality Mixer from Steam, and VoiceAttack and I'm set....no reaching for the mouse ever. After I finish my G1000 I'm going to mess with Wacom's kneeboard for Open Kneeboard.

I would be happy with eye tracking to select the control, and something like a knobster to manipulate the control.

what we need is sort of a hybrid of vr and mixed reality, you have a physical cockpit and green screen. Any thing above the dash is green screen which renders the outside, anything below is like mixed reality so you see your own hands . like what the vario professional version can do but opened up to the public and all headsets. for example pimax come up with a clip on external camera that utilizes the usb port at the bottom of the head set that detects the green screen and renders the outside view, below the screen it displays all your displays and peripherals and you can see your hands

You can ramble all day 😂..I got myself a "Apache simulator" gonna put that on a motion platform. But yep still need a mouse for now 😢 next thing will be vr gloves.

I have shouted feckin loads on their Official Forum for VR KEYBOARD & fix the HOTAS HAT BINDS for Menu Navigation plus EXTERNAL VR ZOOM. They need an updated UI regardless of 2024 which is NOT what our money went towards!!!!

I agree that the mouse is an environment killer. It kills the experience and worse when on final. But that said what an experience VR has given us!!! I remember when I got my HP Reverb and set things up and sat my wife in the Piper Arrow in my next level racing sim pit... Her first question was, "Why do you bother flying real airplanes?" She had a very good point as time in a real plane is very expensive compared to time in a flight simulator. One big thing I noticed after flying in VR for 9 months was the lack of scenery objects in VR compared to using my monitor and TrackIR which I am using more now as I create content. Thanks for sharing your thoughts and the YAW VR link. I think the folks there may be on to something and I hope they expand on it. 👍

The biggest immersion killer by far are the wibbly-wobbly reprojection artifacts and distortions. Which is why I never ever use reprojection 😅

I know that Varjo are / were working on a knee based device which combined eye-tracking with the device. So in essence you look at a knob or switch, then use the device strapped to your knee to either adjust the knob or flick the switch - I have no idea how that development is going but the benefit is you could use it for any airplane.

The problem for me with the 172 and knob positions is that’s great for… a 172. With so many aircraft to fly I think very accurate hand-tracking is essential, but also the sim (whatever sim) must be able to do something useful with it - and that seems to be lacking as well. I don’t think you can have the latter until the former is nailed down first though.

Thank you Mark as always good information I am at home now waiting for my Tobii Track eye 5 thanks to your excellent review got it on tobii summer sale 15% off 😁

Are you talking to me?! I like where this discussion is going. We need more representation and conversation so developers start creating solutions. The mouse is absolutely an immersion breaker!

Great information👍...SimWorld is merging reality...I would buy it but the price is out of my range😅cheers🍻

I use HSC Voice pack for fs2020 and VoiceAttack. A great enhancement for VR.

I hope one day to enjoy VR...Can I come over to your house Mark...Faxe on me!😂

What I'd like for DCS is eye tracking to replace the mouse cursor, rather for eye tracking to move the mouse cursor to the knob or button that I'm looking at. Then it'd be only a matter of binding 2 buttons on my HOTAS to act as left/right mouse clicks.

Ive been flying in vr for many years the mouse is definitely an immersion breaker .i think the hand as controllers in the vr cockpit would be a great start.perhaps a generic ad on camera to the front of the vr headset would work.

To be honest, My peripherals are set to where i'd actually find them in VR, so i'd just e happy with the YAW 3dof chair, its on my list and at circa £3k i wouldn't really call it silly money when that's about the cost of a good sim setup and VR anyway.

Do. They a setup for bigger planes like 747 and a320 etc?

Yeah immersion in VR is a thing. I’m getting my next step coming in tomorrow or Thursday though. DOF Reality P3. Can’t wait. But your right, the switches and dials is a thing. I’m also doing iRacing, less buttons for sure

Anyone know how to set the mouse cursor in msfs2020 to the thumb stick on a Virpil flight stick? I'm at a loss

Very thought provoking. Made me wonder if combining totally blank panels where the instruments go (to act as the ultimate reach limit) with gloves containing small actuators that can simulate the feel of knobs and switches might be possible so that you aren't limited to one model of aircraft. The software would track your hand and "tell" the gloves when to simulate a particular touch item such as the switch, lever, knob etc. that is at the position where the hand is.

5:34 that is good point about the syncing between the VR headset and real-world state of a motion sim It's always been a bit of a brain stumper as to how the VR Projection interacts with any motion sim. I would assume that in the absence of a feedback loop, the VR headset would incorrectly interpret the pitch or roll of the motion sim as an actual projection and incorrectly pitch or roll the image (causing motion sickness). For the augmented reality perspective ; Redbird is attempting to develop an AR flight training device, that from the purview of the FAA, would constitute a BATD (ref, "Redbird Demos Mixed-reality Flight Simulator", Matt Thurber, ainonline, 2022-07-25). 4:10 from a hobbyist perspective, the only other manufacturer that I could think of would be DOF Reality that might be slightly cheaper

VR...how can you use your Virpil sticks/Honeycombs/Thottles you can't see them?

I use Mobiflight in combination with the Behringer Midi Board - that is a Solution for small Money pockets

How do you solve the problem with selectors and encoders?

How i find this and how match this price?

I've taken a passing interest in VR for simming but it doesn't yet appeal to me for the high cost of a headset and the problem reasons you've given. So i'm sticking with head tracking at the moment. However, a short while ago, I saw a KZfaq vid from a flightsimmer who'd positioned a screencam pointed centrally at his yoke and flight panels and then placed a clipped image from it as an overlay to match the area below the panel coaming on his VR headset. This was a very early version (the camera only gave a B&W image at low res) but he demonstrated that it felt far more immersive than having to search for various switches, etc. on his panels. I've spent the last hour searching for the vid in vain. Has anyone else seen it? As most of us fly using the same panels and controls set up, it could be a good, inexpensive solution to the VR problems you've discussed.

In my opinion mixed reality is the solution affordable is not far away

I don't get why they aren't putting more into hand/finger tracking in VR. The oculus quest 2 is improving slowly. But it doesn't seem to get much love on PC (the OQ2 isn't even a named VR headset in Steam). i have no idea if there is an API for it to integrate finger tracking in PC games/apps. But all these sim's use the original interfaces that are build for fingers and try to replace their activation trough buttons 😢 instead of adding tonnes of crap to your desk/seat. Just add finger tracking for VR. Ofcourse there is no real money to be made that way for periveral makers.... But do software developers really care about that?

Hi I know this isn't on topic but i don't know the way to ask you a question. Do you have instructions on the pmdg dc6 bravo tq setup.? Ive followed all your others Thx Lee

Doing 99% of my flying in the 737 and with limited space, panels are of no interest to me. What I would like to have though is a way to be able to see and use my own hands in VR and preferably with haptic feedback. That would increase the immersion quite a bit. I know there are already some products that could possibly give me that experience. However my understanding is those are so far not fully compatible with MSFS but when they are if that will ever happen, it will be an instant buy!

G'day Mork, you rabbit on any time you like - that was so refreshing original and touched on some thoughts I've been expanding upon for over a year now. Id love to discuss with you personally if that is possible. Let me know how. Cheers, Michael

Force feedback yoke should be widely available.

My brother built this at home. You basically just need some plywood, the knobs, one or two arduinos and the mobiflight connector - then you place your cockpit exactly in reference to one of the knobs' position in the virtual environment and boom - there you are. Cost factor might be around $100-300 depending on the amount of equipment you want to build. The openxr hand tracking enables you to see your hands in VR. Everything is very easy and well-documented, my brother had zero programming experience.

The big problem is that the VR interface in MSFS is not good enough to Interact with the controllers. X-Plane nailed that. I was hoping this was the first issue the Asobo VR platform team would address, but now the team does not exist anymore. If it does not work with controllers, it won’t work with hand tracking.

By the time we've spent 1000's on all this Gucci Sim kit...may as well have flying lessons 😂

I am a VR Pilot!!

A motion cockpit without G factor is not reality at all in fact it could be a detriment. I'm a sixty year ATP and flight instructor and for me the value of a flight sim for myself and my students is purely learning and practicing procedures. VFR or better yet IFR On a side note, I learned flying originally in a Champ with a stick. Switching back and forth from a stick to a wheel was never the slightest problem. I prefer a stick with a flight sim. the smaller the better. Being able to locate it below the desk has been a major improvement for me.

Come on Mark, you know there's a mouse in every Airbus and Boeing cockpit. Hehehe...

Has anyone ever used Leap Motion hand tracking?

which setup is that? FS2020 and Oculus 2?

Have you asked yaw for a review unit? 🤪

I don't have problem with mouse

I beleive forced feedback joystick or yaw should be more availables and cost less . To my knowledge , only one company is doing it at a pretty high price.

Here is my thought on what you talked about, you may not agree but if you sat inside a real cockpit and put a vr headset on, you are basically making yourself blind to the switches and dials. So flying VFR with a vr headset you can set a high level of immersion, but if you fly IFR it is like you say the mouse is your only friend and without some very scientific setup your not going to get anything near what you are asking for. This is the reason I am happy not to go down the VR route, tobii with a highend panel in front of you with devices is the only realistic way for home setups imo :)

Id pay to use that 172 fir an hour😅

I use the quest 2 and the vr controllers. Never ever the mouse.

Hi Mark. I’m not a VR enthusiast but I do definitely see the future of this. I use four monitors.Simply speaking a sheet of 1/2 plywood with switches, knobs, dials that could be reached by muscle memory with the center point of vision controlling the switch that is being used, and its authority. Is this not a simple way to get away from hand held controllers that ruin your immersion? Always do enjoy your vids. Ps msfs and others for over 35 years. Biggest wanna be ever, lol. Alex

Apple mixed reality with Vision Pro will solve all these problems.

well Dan Church (Flyinside dev) dveloped hand tracking years ago and it worked fantastic with all threee wih fsx Flyinside and xplane but Xplane 11 broke the api and he never went on with it such a waste of talent

LEO BODNER. Thats the name you should know instead of scammers like Knobster. I think they are calling their customers knobs when they see a $200 price tag for a dual rotary encoder.

VR pilots need....wait for it.... a life.