Update on ChromaTrack Ovberlay that provides hand tracked Mixed Reality Portals to allow real world interaction during VR games.
Пікірлер: 76
@marcofernandes98396 ай бұрын
Making amazing progress towards a truly workable mixed-reality sim setup! Thanks for all the sharing, Russ!
@fwdskillzone1767Ай бұрын
This is amazing! Thank you for everything you do for the open-source community. Could you please create a step-by-step guide specifically for X-plane? I've followed all your videos on mixed reality, but I haven't been able to achieve any of the results.
@StevenMalatesta6 ай бұрын
Thanks for sharing this update Russ. I’ve been following these videos with much curiosity. Thanks for pushing the edges for us in this space.
@SimHangerFS6 ай бұрын
Another step towards immersion 😊 this type of program indicates the next exciting development for Virtual Reality.
@tangocharlie92915 ай бұрын
Thanks for spreading the news on MR. I think the PERFECT simulation experience will be a physical panel with working knobs, customizable gauges of all manner, paper charts & kneeboards, with VR green-screen type setup for everything outside the cockpit. I have a pimax Crystal and may dump it for the Quest3 if this is possible… It would be FANTASTIC to be able to build *MY* plane’s instrumentation, combined with the MSFS VR world.
@scheffchoch15 ай бұрын
Very nice will definitely try it out. Also looking forward to the MR faceplate of the Pimax Crystal and use it with that!
@user-ev5ur7fw4t6 ай бұрын
Great stuff thanks for sharing.
@soczkiens6 ай бұрын
Downloaded the software and donated for author, thanks!
@mxcollin955 ай бұрын
Fascinating add on!
@CardboardSim6 ай бұрын
If i had a Vr setup, I would definitely use this!
@exodous02Ай бұрын
I have a shoulder injury and my left shoulder hurts if I have my hands together on a keyboard too long so I have a split keyboard that is shoulder length apart so this would be perfect for me. Wish Immersed would do something like this, I'll suggest it on their forum.
@BigDuke6ixx6 ай бұрын
Wow, that's impressive.
@samturner44063 ай бұрын
Hey Russ, dd you see reality mixer just released a feature in beta that allows us to use a usb webcam in lieu of the headset passthrough cameras? This could be a game changer, especially since most webcams are better resolution than headset cameras.
@MegaIronica6 ай бұрын
This is amazing! Projects like this and like the new UEVR injector make me super excited about the future of VR and MR. My question is, I use the cable commection for flight and race sim. I guess for now, there is no way to use it without a Virtual Desktop. I know there is a way to bring hand tracking to DCS but I dont know about MR.
@terjehelle6 ай бұрын
Thanks for this video. What system (PC and headset) are you using?
@father-sonflightsimulator38385 ай бұрын
This looks really useful particularly for VR users flying IFR on a network like PilotEdge. (ie: CRAFT notes, ForeFlight) What headsets is this software compatible with? I use a Reverb G2.
@Lorian6674 ай бұрын
Wow... AR is really comming!
@scheffchoch14 ай бұрын
Hi Russ. I tried it out in my physical home cockpit and it is absolutley the right direction for me! Did you find a way to run it in OpenXR (VDXR) over Virtual Desktop? Or is it bound to SteamVR at the moment? MSFS seems to run quite a bit smoother in OpenXR than SteamVR.
@HCarter1114 ай бұрын
@srbarlow - I recently picked up a quest 3 and am trying to get it working like in your setup. My pass-through video looks terrible compared to yours. Do you find the bright images to be blown-out contrast-wise? I'm really thinking of exchanging this one for a different one.
@user-ru3hq7dt8z4 ай бұрын
it's great !!!! so The controller is displayed on monitor when I do hand-tracking, do you know how to remove it? I can't see the controller in your video
@keepsyouincontrolАй бұрын
hi, couple questions. you said that you can bind to joystick. however i cannot get it to bind. i can get it to work with keyboard, but not joystick. any ideas? i also noticed the colors from this app, to virtualdesktop do not match. do you know what RGB setting you had to change them too to match?
@salsa2k6 ай бұрын
Hey Russ, another great video thanks. Do you know if is possible to create a portal in one specific area.. for example for the A320, the light toggles are on top, and I have my button box on my desk, but I would like to when I look to the top it shows the portal with my button box. I know it could be confusing to look to the top and toggle the buttons in front of you.
@rbarlow5 ай бұрын
Sounds like rreality Mixer is what you want. Did you see my two previous videos on that. It creates toggle-able fixed portals.
@atrem652 ай бұрын
Hi Russ, how do you get the combined picture into obs? Steamvr window doesn't show the portals for me in obs
@christopherduarte36096 ай бұрын
quick question: are you able to select a knob in vr and then use knobster once it's selected?
@86abaile6 ай бұрын
It's very cool, but I wish people would stop developing for steam VR and instead develop for openXR.
@rbarlow6 ай бұрын
They tell me that Steam makes layering video streams possible while the current OpenXR standards do not. I agree that SteamVR is a performance hog I’d rather avoid but the purpose of my mixed reality videos is to show the potential and hopefully stimulate development of OpenXR mixed reality tools.
@sirgriffinman6 ай бұрын
It's totally possible to do video letters but you have to build it yourself instead just using the steamVR api@@rbarlow
@frankfrati46 ай бұрын
I can't even run DCS in steam with a layer because it ends up being so jittery and choppy that it's unusable. I haven't tried Microsoft flight sim but I imagine that will be a similar situation. My system is not top of the line but it's no slouch. I have a 3090 with an I-9 9900k and 64 gigs of RAM.
@Buddy308Ай бұрын
Russ, both the Hand Portals and Color Panel apps require Virtual Desktop. I therfore deduce that the new Pimax Crystal Light will not be compatible with either of these options. I cannot determine from the Pimax website OR Web searches if the Crystal Light even has passthrough cameras on its basic model. Is my assumption that neither program works with the Crystal Light correct? If so, do you know if there is any hope of getting such functionality to work on the Crystal or Crystal Light?
@flysurfer25012 ай бұрын
Hi Russ, please allow one more question. You are so good in figuring out all of these details. Did you find a better pass through VR glass as the Quest 3? Is it worth spending more money on a Varjo XR3. As far as I see it here, the Quest 3 is pretty much good enough for all our needs? My goal would be to fly fully IMC in X-Plane to the minimum with it. Do you think its better to buy higher quality VR pass through glasses? Honestly I don't find better reviews on this topic. Do you heard of better pass through glasses which are worth it? Thanks a lot Russ. Appreciate all your contents. Christian
@rbarlow2 ай бұрын
Wait for the XR-4 and Quest 3 is not bad to play with now but just not that easy to setup now … need better software like Varjo Lab Tools but for Quest
@SimForYouDIY5 ай бұрын
Have you tried this with a glass cockpit in Msfs. My G1000 screens freeze as soon as I enter VR. Still trying to get it to work 😢
@steeltribe39676 ай бұрын
Thanks for the update. I did download and use it and it is quite amazing. Unfortunately, steam VR slows DCS to a crawl in comparison to Open XR so it will not solve my MR challenges. Back to peeking through the nose hole :) I like where this is heading for the future.
@rbarlow6 ай бұрын
Thanks for the info! What cpu/gpu? Yes too bad it is not OpenXR. Yanking and banking is definitely more demanding than straight and lever and pattern flying.
@steeltribe39676 ай бұрын
i5 12900k and a 4090 but even so this is a lot to ask especially via Virtual Desktop. Wish there was something like this for the Pimax Crystal :)@@rbarlow
@Flying-4-Fun6 ай бұрын
Hey Russ, can you do a how to setup this in VR? I have a Meta but my pink shows up. I can’t get it to pass through. I donated $20 when I downloaded it
@Flying-4-Fun6 ай бұрын
Will this work with openXR? I own msfs20 & steamVR has to much baggage
@bobbydigital185 ай бұрын
Do you have Virtual Desktop? I do, I have the same problem, i think we have to change the virtual environment to passthough on VD. Not sure though i will let you know when i test. My son is currently using the quest
@davecollins40146 ай бұрын
Is there a way to put Microsoft FS 2020 and Xplane 11 and 12 onto the Steam platform? I bought all these directly from Microsoft and Xplane. Thanks. Really enjoy your videos and appreciate all your research!
@Myrdraall4 ай бұрын
De you need the Steam version of Virtual Desktop? And does it work IN VD like if you're just using your desktop in a VD environment and want to grab a coffee or do you have to use the passthrough VD environment?
@hejmollermads6 ай бұрын
Great video. Do you know if there is being experimented with having the virtual hands and arms follow you real arms? That could be used for giving even more immersion when you hold the flight stick and flick switches. I know you would have to align you own IRL stick/yoke/wheel with the simulated one. I mostly use race sims, and when I have my IRL wheel aligned with the virtual one, my brain accepts the virtual arms as my own. But I would love for the arms to actually follow my own. That would be magic for the immersion.
@rbarlow6 ай бұрын
I think Meta even has full body tracking for the Quest 3 but the game developer would have to take advantage of the technology in the game.
@shaneastwood63646 ай бұрын
Pressing 2 hands together in japanese means gassho or to appreciate its used lots especially in traditional martial arts like karate
@Jasperjvr20105 ай бұрын
Hi Russ. I am clearly missing something. I cannot even run virtual desktop without it forcing me to use my controllers. Then in steamvr it only works with controllers. How do I get steamvr to use hand tracking and not the controllers?
@paulgrenga1256 ай бұрын
Apologize for the inconvenience... I can't seem to find the download link. Could you post it in the "Update on ChromaTrack Ovberlay that provides hand tracked ..." section above? Thank you
@FlightSimandOtherStuff6 ай бұрын
Hi Russ, How do you get rid of the controllers. Mine is working fine (by the way no need for the steam version of MSFS I use the store version, launch the sim, then steam vr from VD and ask the sim to to go into vr - hey presto_) But when I am in VR I always see the controllers graphics from steam, can't seem to get rid of them and they block the view. This only happen here not in reality mixer by the way
@rbarlow6 ай бұрын
There is a setting in VD controllers that says hide controllers if inactive (I think)and after they sit still they will disappear. Then just use the hand gesture (praying hands) to activate the portals.
@shaunwallace86034 ай бұрын
This is the issue I'm having and I can't find the "hide controller if inactive" anywhere. When I set the controlers down, the portals snap to my hands but the controller graphic is still there. Any update on this?
@williammclean83865 ай бұрын
Thank you Russ. Can we use reality mixer and this program together?
@rbarlow5 ай бұрын
Possibly but not sure performance would be very good.
@Test-Pilot11943 ай бұрын
I have used quest 2 with msfs before. Now, the only problem I did have was using the vatsim window, for chat and frequencies. I’m very good at knowing how to reach directly for my controls in vr, my brain to hand is very good. Is there any other way of having my vatsim window….maybe in the cockpit with quest 3? Or would I have to go down this route with the pass through windows to see the specific monitor with the vatsim window. Thank you.
@rbarlow3 ай бұрын
I think this is what you want vpilot.rosscarlson.dev/ and sim-efb.com/ Read here www.reddit.com/comments/rnvzb6?
@Test-Pilot11943 ай бұрын
@@rbarlow thank you very much. I shall check this out. I enjoy your informative video’s. Keep up the good work.
@rbarlow3 ай бұрын
@@Test-Pilot1194let me know your results. I haven’t tried this myself.
@Test-Pilot11943 ай бұрын
@@rbarlow il be sure too. I’m looking at picking my quest 3 up in the next weeks. And it an ability I need with vatsim. I don’t really fly without vatsim. And I want to be in vr too. Il let you know of my findings. 👍🏻
@jrhur105 ай бұрын
Will mixed reality on the quest 3 allow usage with a motion simulator? Or will pass thru windows not move with the motion ?
@rbarlow5 ай бұрын
As long as you have the physical cockpit on the motion rig I think it should work( of course you need motion compensation for using vr on the motion rig).
@jrhur105 ай бұрын
yes I do. Thanks for the quick reply and all your awesome videos!@@rbarlow
@soulofastro6 ай бұрын
It would be great for VRChat when you’re drinking with friends.
@rbarlow6 ай бұрын
Cheers!
@jpr73656 ай бұрын
Hi Russ, do you have the MS store version of MSFS, or do you need to have MSFS installed from Steam?
@rbarlow6 ай бұрын
First you need to launch SteamVR from the Virtual Desktop settings menu ( press menu button in Virtual Dexktop). Then you need the Steam version of MSFS and you need to launch it from steamVR library
@FlightSimandOtherStuff6 ай бұрын
I have the store version and it works fine, no need to launch it from steam. @@rbarlow
@jpr73656 ай бұрын
OK thanks, I have the MS store version so it won't work for me.@@rbarlow
@peterbachinsky19244 ай бұрын
Russ: I'm too stupid to live. I've just acquired Meta Quest 3 VR goggles. Do you think I can connect them to X-Plane 12? HELP!!!
@AlMcGigi6 ай бұрын
Is it mandatory to use Steam VR ?
@86abaile6 ай бұрын
Machine translated from the website: "It does not work with standalone HMD solids because of apps that utilize the VR overlay function of SteamVR." So it seems that it must use steam VR, which is annoying for DCS and MSFS pilots hoping to get a little extra performance by eliminating steamVR.
@KevinMick5 ай бұрын
Has anyone in here been able to get the Pico 4 to work with this?
@KevinMick5 ай бұрын
Pico 4 doesn't support hand tracking on PCVR, so doing this on MSFS VR is not going to work with a Pico 4.
@venec6 ай бұрын
Too bad it's only for SteamVR, the overhead of SteamVr is too much for my liking, the Oculus runtime works way better for me personally on the Quest 3.
@zsavage18206 ай бұрын
I wouldn't give 10 cents for anything QUEST... RIPOFF and their goggles are overpriced glorified phone stuck in some shitty goggles..
@rbarlow6 ай бұрын
Don’t sugar coat it tell me how you really feel😀. I have owned lots of headsets including Aero and Crystal and for $500 Quest 3 isn’t a bad experience and has allowed me to experiment with Mixed Reality. Not a Meta fan for other reasons so I did compromise myself a bit when I bought it. Thanks for your comment.
@KevinMick5 ай бұрын
Can you go over your VD streaming settings in the Quest 3 as well as VD settings on PC. I can't get VR to work with this when I use the runtime SteamVR but it goes straight in using VDXR. @srbarlow
@zaelu3 ай бұрын
I am quite baffled... on the site of the developer of this program there is no instruction or link to a youtube video where it shows how exactly one is supposed to use the program. In this movie and previous on this channel... same thing... lots of explanations of what it does... but... Maybe I am retarded... How?? I have steam VR. I have Virtual Desktop. I am in MSFS in VR... I start this program and all I can see are two solid pink spheres/circles no passthrough no nothing. It's amazing :D Was it supposed to be so easy to understand and use that nobody thought to explain how? edit. Found one!! kzfaq.info/get/bejne/e512nbF7z9-2aJc.html