This type of innovation is what makes me excited for the future of VR/MR, thanks for your contribution to the space!!

A glimpse in to the future of VR. Thanks Russ!

Thank you, Sir! For all your brilliant work of putting your own personal knowledge gathered the hard way into very straight forward and easy to follow informational videos, Thank You!

Been a vr user since the original oculus rift. I just purchased the quest 3 from a quest 2. Night and day diffrence adding color pass through. Mixed reality is where it’s at.

Another great video Russ. You are a great innovator with regard to Flight Simming. This looks like the start of something great. It will surely be developed further.

This topic interest me as someone that has a lot of money in peripherals and needs to be able to see them. It's hard to correlate their relationship and what's happening in VR in a smooth manner, rather than having to constantly rely on muscle memory.

Thank you again for your fantastic videos! Just what I was looking for.

thats really awesome! cant wait to see where it will go in the future!

This is a way forward. Optimal hand tracking and virtual cockpit interaction, with haptic feedback, would be another piece of the puzzle.

Thank you so much for posting this. I just got my flight sim put together about a month ago, and stalled on buying 3 large (and expensive) displays, hoping that mixed reality would come out in a year or 2. It was more important to have a realistic and tactile interaction with my sim, in terms of the instrument panel. I'm in on the Meta Quest 3 because of this video. I'm sure the quality will improve over the next couple of years with new technology, but this looks good enough to give me the best of both worlds now. Plus, I can always re-gift it to the kids if something better comes along later! It will be interesting to watch how low prices on TVs and monitors drop over the coming year.

Thanks Russ for bringing this to our attention. I've seen so many videos of the Varjo XR-3 MR and wondered when the technology would make its way down to the consumer level. I'm in the process of setting this up now to try it out. I have the Quest 3 and RealSimGear Cirrus SR22 cockpit. I'm so used to using muscle memory for all my flight planning it's going to be weird seeing my actual cockpit in MR. Cheers!

Inspiring video and ideas, I really think that mixed reality will be the future of flightsimming and increased immersion, if we can use and see our own hands within the simulation that will be a huge change.

Good evening from the Netherlands, This video convinced me to finally order a new PC which is heavy enough to fly DCS and BMS in a Mixed reality Home made F16 cockpit. Thank you for helping me out with your cool progress. Regards Maarten

Thank you Sir for this presentation. That is exactly what I‘m waiting for the last 10 years !!!

Fantastic Video! I know what I'm getting (myself) for Christmas this year.

This is pretty fantastic!

Interesting stuff ...amazing technology 😮 Thanks for the video Russ 😀👍

Excellent Guide Russ. Thanks a lot. Really looking forward to FalconBMS getting support for OpenXR in Q12024. Finally I can find my buttons properly.

Trailblazers for next gen pilots!16 year old me would be extremely happy to see the progress. Now at the airlines. This really make me happy to see what available for the next generation of pilots.

this is pretty unreal. Extremely excited for the forthcoming chroma keying functionality in Reality Mixer, that may very well convince me to switch from my Reverb G2 to Quest 3. I don't fly but i do have a VR-only sim racing rig and being able to key in just my steering wheel, shifter/handbrake, and/or button box without the other surroundings is super enticing

This is extremely awesome.

Beautiful!

Thanks Russ I didn’t realize this was available.

This could be a total game changer for DCS sim pilots. What you mention at the 7:30 mark about home built "simpits" and not wanting to change over to VR hits the nail right on the head. That's the exact boat that I'm in currently.

Super interesting video here Russ, thank you for sharing. I had a feeling it wouldn't be long before someone developed a solution given the pass through on the Quest 3 is as you say, getting there. I may give this a go once I get the Somnium VR1 and XR-4 in my hands. Exciting stuff. Best wishes.

I'm just starting out in VR and find your KZfaq video very interesting. I have a home flight sim and have just purchased a Quest 3 and want to put the two together hope to see more videos to come.

Thx for this video! I've been trying to figure out if quest 3 had passthrough or not, but didn't know how to phrase my question. Definitely buying one now to use with DCS and my Virpil controls.

Amazing! It gave me the idea to use OpenKneeboard to create a solid color PDF that gives me a passthrough window to my physical kneeboard for taking notes. That works in openXr as opposed to reality mixer, which is is steamVr only.

Great ... never seen something like this ... very impressive !!! Ciao from Italy ...

Good info, just bought the quest 3 a week ago and hope it gets support with mix reality for MSFS2020 and DCS soon.

Mixed or Augmented Reality is defiantly the future of flight simulation. I have always loved the idea of a simpit, a full 1:1 recreation of a real cockpit (or something comparable) that would feature all of the cool bells and whistles that would you see in the virtual cockpit. This naturally allows it to be interacted with without a mouse or pointer. However, this (used to) come with essentially one of two options. One, VR. This removes half the purpose of an actual simpit because you can't see your controls and makes it much more difficult to navigate the panels (it is possible, but takes a lot of practice.). Two, surrounding projector / monitor setup or even 2D head tracking (Track IR). This allows you to view and navigate the simpit while easily viewing the virtual world you are flying in. However, this results in losing the emersion that VR brings that myself and others have come to love, primarily the ability to use and experience Propper depth perception while flying. Mixed reality combines the best of both of these solutions. Once these consumer level mixed reality solutions become easily available to the general public, I know and hope that it will truly change the flight simulation world by allowing simpits and projects alike to become available to more people, increasing their immersion and experience that it provides.

this is what I looked for immediately after trying vr flight sim a single time, obviously needs to happen for all the nice gear to be visible and work

Very good video, thank you.

Russ, great news.. Encouraging.. Thank you.

I have a full a10 pit for DCS. I’m excited to get this going. Santa is bringing a quest 3 for Christmas and this is going to be one of the first things I do. I been waiting for a decent passthrough application before I got into vr.

Thank you for this. Mixed reality really does change the game. There's an outfit in Switzerland which has created and approved a mixed-reality Sim for helicopter training, using a version of the Vario headset. I have only seen photos of it, but as I recall, the panel had instrument cutouts with some kind of pattern in them, which is recognised by the Sim, to insert an instrument.

Great work! This is where VR needs to go! I don't enjoy flying with my headset often simply because if I'm trying to be a "serious simmer" (for lack of better word) I have so much hardware that I can't use in VR, which makes me less efficient flying. Let alone the touchscreen interfaces making operating aircraft faster than any VR or mouse setup can possibly offer currently. This is the future for sure

Thank you for this video. I was not even aware of this possibility, but that is because I've switched to MSFS2020 from Xplane11. This is great and exactly what I was wishing for. Even just the possibility of taking hand notes in RL or looking at the charts and maps, rather than having a separate window in the VR cockpit I think is a lot more user friendly. I've been struggling with that a lot. Shame that this is not available in MSFS but this alone may persuade me to switch back to Xplane.

Thank you very much. That is interesting

Great video…. thanks !

Guess what I'm trying this week! Definitely a step in the right direction!

Love it. Thank you. I am in the same boat when it comes to VR vs real Control panel. I just finished building one and got Quest 3. I love flying MSFS2020 but own X-Plane12 as well. Unfortunately neither one is Steam version so I am crossing my finger that someone will come up with a tool to simplify this process and allow non Steam version of MSFS2020 to be used with mixed reality with Quest 3

Nice. I have also “dreamed” of a way of combining VR in the sim. I built a really nice IFR sim that mimics the panel of my actual Cessna 182. It uses AirManager and a separate screen for instruments. I also have a RealSimGear Garmin 530W so haptics would be very welcome. I mostly use it to practice IFR stuff like approaches, holds and standard attitude instrument flying. My idea was to build a custom XPlane cockpit that puts all the knobs and buttons at the same position you see them on the screen. This is much more elegant.

That is pretty cool! This is getting really close to what I am hoping for! Really hope that external cameras will be supported soon and proper chroma keying for physical greenscreens. How did you find the resolution of the cameras hold up with actually being able to read the instruments, looking at physical checklists and charts, taking notes etc. ?

pretty great in my driving rig too

Hi Russ, thanks for all those videos! thats's great and amazing! I am using A320(FBW) with MSFS and have HW(Airbus Joystick and saitek panels + tablet connected to emulate the FMGS). As you refeering to Xplane, would the Quest 3 mixed reality works? thank you

Certainly a good option and as you said, this will only get better. But, for me, the ideal setup would be for the system to track my hands and fingers so that I can interact with the virtual cockpit. And yes, I have a diy cockpit but I would gladly ditch the hardware if it meant that I could seamlessly interact with the virtual one.

Another viable option is to use InfuseVR where you can see 3d representations of your hardware in VR

Thanks for your wonderful input! would like to see more input on FS2020

This is a step in the right direction. Flight sim with VR is amazing because of the 3D effect, specially when flying over mountains!. But with no visibility to the real hardware buttons is really the problem. Only the yoke and throttle are easy to control. I really think the future is with AR, where you could control those nice 3D cockpits renders that comes with the planes you use, with your hands. Good eye tracking within the headset combined with AR like the direction that is taking the Vision Pro headset is really what will change the game. Air Manager is great. But being able to interact with the game cockpit while still in VR, without having to use a mouse, would be even better!

Great video, Russ, very interesting. Will the recently released “Steam Link” Quest app improve or streamline the set up/use of this kind of setup?

Russ, I applaud your efforts to make flight simulation more realistic. There has been a lot of discussion about how x-plane generates weather and how the aircraft responds to the weather. Much has been said about wind shear, microbursts etc, but I have not found anything about how the pilot's upper body responds within the aircraft relative to the instruments. It is this response that affects the pilot's ability to twist dials and read displays in turbulance. This is a very important aspect of flying in a real aircraft. From the seat up, a pilot's body is flexible and responds to external forces differently than the aircraft. Modeling this correctly would greatly improve realism. This is especially important in MR. Your thoughts?

Thank you Russ, Great video. Can you tell me what you use for a touch screen monitor. I have been looking for a 14 - 15.6" and just not sure yet.

Love this. Can reality mixer let you draw a specific shape to match your cockpit? And can you have multiple boxes at the same time? My thought is to draw a non-box shaped box for the panel and the yoke, then another for the throttle, and another for the radios and gps. And letting the sim draw itself in the spaces between them.

This is the technology I have been waiting for, before taking up VR/MR. I want it for my space sim games, so that I can see my keyboard and controllers at all times. For me though, I think I'm gonna hold off for a little while, until the process is simpler. Currently, I'm using triple screens @ 5760x1080, and TrackIR 5, in a dark room, with a stack of power amps and a massive 7.1 Hifi audio system. In effect my gaming rig is a devastatingly powerful 7.1 Hifi setup with a computer attached to it. :P Please don't ever let anyone tell you headphones are the way to go: they are garbage compared to 20K worth of all-analog 7.1 system. :)

This is a great video Russ! Thanks for showing it and the useful application of everything. Now all that is needed is a Varjo Aero type headset with pass through at a reasonable price. Unfortunately for me I am not willing to downgrade my visuals to that of a Quest 3 from my Aero. Yeah there is the XR-4 coming out but that is not a reasonable price. lol But I am pretty sure Varjo will have integrated the type of pass through you were talking about. Keep up the great work!

This is IT!!!!! I hate the mouse! Hardware peripherals mixed is indeed the way to go!!! Thanks!!!

Works in MSFS 2020. Might want to use the Steam VR beta.

Good idea was wondering how this would look.

Encouraging! My next step is VR!

Good afternoon Please tell me what could be the problem, when I launch Reality mixer on my Meta Quest 3 glasses, it’s like my cameras don’t work. The area in which the room should be visible is simply gray. What could be the problem?

That is so cool. Thx for demo. What is that flicker to what looks like a glass cockpit (20:39) in video that pops up once in awhile? Still trying to figure out what all that clutter is in background (middle right in screen) of your cockpit? Let us know about Microsoft Flight Simulator and if it works with that someday.

1:09 The default Air Manager panel has the vertical speed stacked above the altimeter. You can easily change it but it bugs me whenever I see ppl using it without fixing that

How soon would you venture to guess will there be a hand glove/passthrough of hands so you can manipulate the different instructions/controls without headset grips directly in Sim? Or is there? First time viewer and enjoyed tutorial. Thank you

When I run reality mixer dcs starts getting super choppy and loses frames like crazy. Even if I have the passthrough window closed (I have it on a toggle). Are there any settings in steam that can help this?

Improved hand tracking is probably the way of the future for most people. Professionals are going to use professional cockpits and for the rest of us I would very much prefer to just have reliable hand tracking and be able to use the in-game cockpit, which changes with every plane.

What setup do you have there ? Could you list your components you use in this video please ? I’m considering a quest3 and desperately need to update my old plastic pedals and yoke. I have MSFS so do you think the Mixed Reality pass ugh software would work the same (I assume so since it’s just a hole on the desktop ?). Would also like to know how immersive it is and how much it models reality. Thanks.

Excellent explanation! Reality mixer has the option to "add or remove" games to a profile. Were you able to do it? If affirmative, how? Thanks big time!

Great video as usual. Thanks Having to use Steam VR is a major problem based on my experience with Steam. Do you see a way to achieve acceptable MR without having to use Steam?

I think this is really the future. I have been wanting something like this for a while. I don't care about seeing my cockpit (it's ugly since it's 100% VR) but I DO want to be able to see things like my notepad, or my iPad, or checklists... so the idea of just carving out a section down on my thighs so I can have real-world things in my lap is appealing. I'm already on a Varjo Aero though, and I'm not ready to dump that for a Quest. But maybe the next Varjo consumer version will have cameras and support this...

Would the Quest 2 be able to do this? I know it's passthrough is Black and white. But I wouldn't mind if this would work

Hey, Russ. I like your approach to this technology. I have a Rift, but the Quest 3 seems more forward to what I'd like. Iwas informed that because of my meds I can no longer fly via Commercial nor Private certification. Is the Quest 3 a viable surrogate?

When Reality Mixer is on, are those pixels no longer being rendered in MSFS? That would be awesome if having all IRL gauges would cut down on the rendering requirements of the sim and therefore result in not only a more interactive experience using XR, but also a higher framerate in the sim.

Russ, I'm getting ready to buy a VR headset. I tore my old sim (HUGE THING) apart and I'm building the frame for my VR simulator tomorrow. 2-24" Touchscreens for Air Manager (love my knobster), Honey Comb yoke and throttle quadrant. I'm building this to use for instrument flying practice (Private pilot with instrument rating). You said the resolution could be better. Is it good enough to see what you need to see to practice instrument flight. I practice on steam gauges with a pair of G5s and the Garmin 540, mostly. You know if the Air Manager guys have any intention to model the Avidyne IFDs for Air Manager? That's my real world flying IFD.

Russ, how does the mic and audio work for thinks like PilotEdge? Does the headset have a mic?

I’m curious if the occluded area decreases the load on the PC’s GPU thus giving a higher fps performance. If so, that’s a selling point. Turn up the settings !

Looks great, but a little out of my ability to totally understand what most of your video demonstrates. Many things you talk about are probably things most computer guys take for granted but I have to really dig to figure out what certain things are or how they work. I was very happy to figure out how to set up 4 monitors, air manager, and the knobster. You have gone to a higher level.

this is MASSIVE

This augmented reality is getting better indeed. Would be great to be able to create the vr part along the windows of my cockpit shell. So only the outside view is vr. Well maybe soon…

Question for Russ: I think that neither the Pico 4 nor the Quest 3 have eye tracking. Do you think that the large sweet spot afforded by the pancake lenses obviates the need for eye tracking in MFS?

Can you use Steam Reality Mixer to overlay on non-Steam games? I have DCS standard install, so I need to be able to run the Steam Reality Mixer over the non-Steam DCS. It would be prohibitively expensive to buy all my DCS aircraft and assets over just to get them in Steam

Thanks Russ, decided the quest 3 aint up to it for my use,

Thanks again for introducing more innovative solutions to us, I have followed you through Air Manager, Knobster and much more. I recently completed a home cockpit with triple monitors and using Air manager on a separate PC and two monitors and love it. I just purchased a Quest 3 and love it as well but have been torn between the amazing features of the Q3 but not being able to use all of the work and money that I put into my build. This looks like it could be the answer. BTW your VR view seems very clean and stable. Mine is very good but yours appears to be somewhat better. Do you know yet what impact this software will have on performance. May I ask the details on your gear, CPU, GPU Ram etc. I run I7 10700 K and a recently purchased 4060TI and 32g of ram.

What is your view of the Pimax Crystal with passthrough

I'm wondering if it would be easy enough to (or not ?!) to have in the pass through area, the air manager screen could be passed through right around the six pack area (as you've done here, nicely) but then I don't want the keyboard, or the white wall, to be passed through. I guess it would require one to "draw" only the things to pass through, but I wonder if that would either complicate it more, or else not be as efficient (i.e. hit system performance more than just drawing perfect squares or rectangles for the pass through).

Just spent the last hour playing warthunder with my mosty DCS f16 styl sim and damnnn so cool! the immersion levels are high! I tried MSF2020 via VD but it was soo jittery, any tips?

@Russ Barlow. Hi Russ. I'm certain you may not remember, but we have had this debate before and I stand by my previous argument. It was some time ago, but we debated which "system" would be best for VR flight sim cockpit interactivity. I do not agree that mixed reality is the future of VR flight sim. There are several negatives which I will mention below. My belief is the use of VR hands, in combination with real world yoke, throttles, rudders, and whatever knobs, switches, flap levers, trim wheels, and equally important, KNOBS, where the real world peripherals are SYNCED with your VR cockpit will be the dominant means of interaction. There are several advantages to this type system over a mixed reality system. 1. The break of immersion with mixed reality is awful, and even though this is expected to improve, you will still be forced to use, often very expensive peripherals. 2. In mixed reality, the user will still need to purchase the real world screens, gauges, and displays, of all the various cockpit equipment and avionics. AND, you would be "locked in" to those devices. In a VR hands, used with synced real world peripherals, 100% of the tactile feel is achieved, WITHOUT the need for real world screens, gauges, or displays. 3. In mixed reality, your peripherals are, for the most part, fixed, and aren't conducive to use with a variety of aircraft. With the system I am referring to, while still not totally adjustable to every kind of cockpit, it is considerably more flexible and can be fairly easy to move around to accommodate various cockpits. This system IS currently being developed and has the potential to totally change how we interact with our Sims. Simple kits will be produced for various aircraft and many of those parts will be usable for other aircraft. For example: A consumer would purchase a General Aviation "panel". Then purchase a Piper Warrior package. This package would include (assuming they already had basic GA yoke, throttle, and rudder pedals) 6 single knobs for the 6-pack, a flaps handle, a tank selector unit, 4 dual rotary knobs, a trim wheel,..... you get the idea. A user could purchase as much, or as little, as they choose. Personally, I feel I would only "need" the rotary knobs, the trim wheel, and the flaps handle as the tactile input on those items is significantly more important than needing to "feel" the landing light switch. I would just "push" the virtual switch with my virtual hand. Just hearing the click it would make would be satisfying enough. Now exactly HOW does this system work??? The user sits in their VR cockpit. Through 3rd party software they import the individual 3d model of the real world knob. While holding the real world knob, they see their virtual hand holding the virtual knob. They then place the knob on their GA panel exactly where the VR panel knob is and "lock" it there. .... They then proceed to do this with all of the peripherals they purchased. And whallaa... When the user reaches out with their virtual hand to turn the knob to adjust their altimeter they are actually grabbing and turning a real world knob because their real world knob and virtual cockpit knob are in the exact same location. The system is still in its infancy, but as you stated, the advancements are coming very fast . There are a few KZfaq videos of the first of these systems being used and it will only continue to get better and more accurate. I'd love to hear your thoughts on this system after getting a good look at it. As always, I love your videos and keep up the excellent content! Thanks.

Великолепно!!! Спасибо огромное!!!

Is it possible to run it via link cable without having to deal with virtual desktop?

Ross thank you for the video. I did quite not understand , looks like it does not work with msfs 2020 , even if you have bought it from steam ? Am I right ? what is the name of the program you mentioned ? Thanks

Does Reality Mixer work with a green screen? I have been thinking about hanging a green screen behind my HOTAS and around my cockpit so that I can do what you're doing without the window boundaries

Given this uses SteamVR, it would probably work with DCS World as well?

The Quest 6, will have 8k passthrough with perfect object separation, and MSFS2030 will have photorealistic scenery that surpasses current graphic mods.

Hi Russ, I'm not sure if this has been asked already but will Reality Mixer work in a similar way with Quest 2, albeit with a lower resolution of pass thru?

Is this possible with Htc vive pro 2?

Russ loved your mixed reality demo. I’ve spent about 20 hours trying to get it to work. I have nice pink box I can move it around and resize it but no video passthrough. I think it’s one small thing I’m missing the program isn’t very intuitive. Not giving up😊

Was the footage from the first 20 seconds of this video from the Quest3 or another professional solution?

Great POC Russ. I've been trying to set it up with my Quest 3 and when I add the boxes all I see is white noise graininess. I go to camera configs and the cameras won't initialize and the option that you have that said "Quest, Pico, of Vive XR Use Solid Color..." just doesn't exist on my camera config screen. Any tips? I'm super interested to try this out.

Does the quest 3 use more gpu or cpu resources? Since it has its own computer on the headset I’m hoping it might be a good middle ground for computers with the 20xx series gpu

Russ please advise What system pc do i need to run MS2020 VR currently running amd ryzen 7 3700 8core cpu and amd 5700xt which is out dated updating to amd ryzen 9 5950xt stil 3 years old am I missing the boat as this is still outdated by 3years however 570MB and cannot get MS windows 11 install due a software glitch old system adding this new chip will start me in a newer direction. or with all the new chips coming out this week processing power is over the top. what software you using today to run VR3 on a budget head sets are expensive so 55inch monitor for now 2 side monitors 32 inch monitors for now just have joy stick fine with me..x-plane 11 at moment your advise greatly appreciated