I’m convinced this is black magic and you’re a class 3 dark wizard

🚫 playing a n64 in vr ☑️ playing vr on a n64

James this is absolutely insane and I love it so much. Keep doing what you are doing, seeing you push the N64 to way beyond what anyone possibly expects it to do is just so cool. My 8 year old N64 obsessed self would be losing his mind seeing this stuff. It reminds me of when Sega was talking about launching a VR Headset for the Mega Drive/Genesis. Seeing you pull this off and get an N64 rendering two cameras for VR is just so wild. If I ever meet you I am going to buy you a beer big fella.

“If the project dies I want it to die soon…” *awkward pause where everyone is thinking of Portal 64*

I love James' "because I can" style of development.

If your original game has an art style like that, consider me intrigued 👀

That Oculus dev kit is old school, I love it.

Around 1994, I think, I got to participate in a VR demo at Epcot Center and the cast member mentioned it was running on Silicon Graphics hardware. Your project really reminds me of that demo. It’s really astonishing what you’ve achieved here.

attempting to play the n64 with the powerglove on vr is peak "this will be gaming in 10 years" but in 1996

if you and kaze emanuar teamed up the biggest n64 projects would become a reality

Portal 64 getting pulled seems to have freed you and given you permission to play with other ideas. I'm glad I'm able to witness this journey.

That demo reminds me of the Aladdin virtual reality at Disney world from the 90s, nausea included. Awesome work!!

I can imagine it now... Super Mario 64 in VR on the original (slightly modified) N64. That would make for one heck of a video.

For those wondering how it looks: pull the video up on your phone or other relatively small screen at 7:30. Put a piece of paper or something in the middle to block the other eye from seeing. Get as close as you can to the phone and try focusing far away, and the two eyes will merge and you got some 3D. It's not bad at all for N64, very blurry though! That would have been crazy in the 90s / early 2000s!

There are a few things here that might make the framerate usable 1) batching, by only copying an object to the RDP onece and rendering it the two times needed, it lowers the ram bus cost per draw call. 2) Alternate eye rendering, There's 0 chance of a decent reprojection algorithm running on an N64, so an ok alternative is to render 1 eye 1 frame and re-use it the next. Halving the per eye framerate but also halveing the horizontal resolution needed and draw calls. Overall for vr, it's recommended to use the resolution mode that gets closest to your target framerate

I just realized there is what appears to be a cannon in the background lol

Welcome back 64 daddy

There is no reason this should exist and it does and I love it

hell yeah, im excited for the game you are making!! my friend couldn't believe me when I said, "just cuz nintendo ain't making new games for n64, doesn't mean people can't create whole new ones 30 years later, as long as the console lives, so will its fandom"

Using what amounts to a compass to determine direction on a VR headset is a simple yet genius idea that I certainly never would have come up with

"Shut down my port of Portal, will ya?! Fine! Gonna go even nuttier!" Bro this is inCREDIBLE.

The VR stuff is really cool, but the exciting thing is the art style of your new project. It's absolutely beautiful.

As a daily VR user today, its rather awesome to see that same tech being powered by my childhood console. Given modern VR tends to need at least 80 to 90 FPS to prevent motion sickness in the average player, I pre-emptively winced when you said you could only poll every 60 frames and render even less! Still a very impressive tech demo, and that artwork looks lush! Can't wait to see more of your original title!

After Portal 64 I'm glad you still manage to create groundbreaking n64 stuff

Already working on another wild project. Very cool!

I'm so glad that you've picked up so quickly to pivot to an original project. I'm super excited to do what you do next. Loving the creativity and hearing about the struggles along the way

seeing a dk1 in 2024 feels crazy, its not even that old but it feels so vintage, great work!

Actually the best channel on KZfaq, can't believe this is real.

Dear James, thank you for clearly showing how incredibly inept and useless I am. I am surprised that I can even dress myself in the mornings. Your ability to just dream up something and then make it reality is nothing short of amazing. I’m sorry for wasting air.

This is mindblowing. It's also so poetic to me?? to have the N64, a landmark console in the 3D space, connect to VR, the modern horizon of 3D work, agh. its beautiful!! The scene especially looks so beautiful, I like the pixelated look a lot. I think its charming and cozy.

I'M SO GLAD TO SEE YOU BOUNCING BACK!! You rock. Keep rocking!

this is amazing

Happy to see you keeping up making stuff after what happened. This is a sick project!

Amazing to see how you linked these together! I can almost imagine an alternative universe where the virtual boy and powerglove did well and the N64 was a super VR experience.

I love it that people as smart as James are out there doing amazing work like this. Just made my day.

That desert scene is pretty amazing looking by N64 standards. Not to mention having it running at a full 640x480 + being rendered twice thru VR without it becoming a total slideshow.

Oh man I’d love to try this with my DK1. This is amazing!

Man, it’s crazy how much VR has progressed since the DK1.

So many old consoles are learning new tricks. What a time to be alive! I can't wait for the next thing the 3ds pulls off!

yoo haha, this is so cool. A couple years ago i messed with trying to connect my n64 with an Oculus thru the use of some "basic" virtual desktop stuff and couldn't even fully get that to work, let alone actually having it word like this. Keep it up!

i love your vids btw!

Dude, you are amazing!! The 3D looks great and I really enjoyed your explanation of the gyro/accelerometer/compass!

Your ideas are great, but don't let them take down your project again. The whole Portal thing you did was nuts, I hope your inventions will come to our homes

Awesome project! We will add this to our weekly retro gaming roundup.

this shows that even cheap pcs should be able to play vr

I'm so glad that what happened to Portal64 didn't make you quit. You have talent.

Excuse me, but you missed the perfect opportunity to make a Virtual Boy emulator for the N64.

Glad to see that you are still making N64 projects.

The very notion that the n64 can do any of this AT ALL is completely wild, we could have been vomit inducingly VRing it up with Mario all this time damnit!

I love this project! In the future when you have audio quality issues, it would be helpful to run it through a low pass filter to cut out that high-pitched ear-drill.

If you haven't checked it out already, there's a video of someone turning their WiiU into a VR headset on KZfaq. In that video, they talk about a new research paper's algorithm that applies error correction to the gyro magneto and accelero sensors to make a much better approximation, and it's also computationally cheap. Just thought I'd let you know.

The real milestone is 65,536 subscribers. You'll be there soon :)

I'm just shaking my head quietly. Shine on, you crazy diamond.

Wow! This could be the 90s VR experience we all dreamed of 30 years ago.

It’s hard to believe what I see. Exceptional work.

This is incredibly cool! I think a good compromise would be to ditch head tracking, that would remove much of the issues with motion sickness, and just have it be a 3d experience instead. There would be the added benefit of being able to have many more headsets work if they dont need tracking.

Seeing you with that headset and glove on, almost made me cry... the prophecy is almost fulfilled.

your audience loves this, even more than portal64

Glad to see you're still performing magic despite Portal 64's cancellation, this project looks insane.

I'm glad KZfaq served me this, great channel

If there were still any doubts about your skills and this channel's future beyond Portal 64, there aren't after watching this video. Absolute madness.

you should have 6million subs man, this is awesome stuff!

Steam says my old i5 laptop isnt powerful enough for VR but this guy can get it working on an N64

Dk1 oculus is such a nostalgic thing to see. VR has come such a long way and still isn't anywhere near the end of innovation. One of the few consumer technologies that have made leaps and bounds in progress. Everything else has been at a standstill with nothing but gimmicks for at least the last 10 years

Amazing! It would be cool to see this with a more stripped back environment, aiming for 60fps! I wonder if it would even be possible to get 60fps in the higher resolution mode if the poly count were low enough, maybe relying more on untextured vertices etc...

Very cool experiment! My thought on making the sensor fusion more responsive is that you might be able to do more to mix the gyro with the accelerometer. That is, instead of applying the smoothing directly to the trauma from the accelerometer, you could apply the transform implied by the gyro reading to the existing gravity vector estimate, and then smooth the current reading with the transformed version of the prior estimate. That seems like it should retain the smoothing effect for the gravity vector, but reduce the damping it applies to head rotations. It might also let you get away with a slightly lower damping factor. Just a thought. Very cool work! I do think people really underestimate how immersive VR can be even at low resolutions like this.

You are so insanely smart dude. N64 is my all time favorite console.

Love to see you going for another project after what happened with Valve.

If you make this open source, id absolutely give this a shot

He's back, thank goodness 🎉

The way you explained the physic maths behind everything caused me a epiphany as to why math is important to technology. Less math equal VR N64, MORE MATHS equal go to moon. Thanks for the video.

That thicc ambient occlusion though

You need to add time travel and warping the rendered image to match predicted location at the last second. There's quite a lot you can do about the lag. If the warping of the rendered image isnt possible on N64, you can still do the time travel stuff, just remember to interpolate based on estimated head center's rotation, not the headset's :)

This is even cooler than portal

Saw this mentioned on spawn waves channel. Cool stuff

Interesting fact: 3D Missile Command on the 64-Bit Atari Jaguar was originally intended to be a VR Game but the option was abandoned because the head set technology at the time was too heavy. It would work with today's technology.

Glad that events have opened a portal into even more amazing 64 updates. Would love to see a for show image of you with the 64, Power Glove, and Occulus. Also, what about dual n64s... I imagine that besides the hardware hurdles, the latency result would not be worth the squeeze.

Great to see your projects continuing. Super excited to see what your original game concept will be.

Now that you know you can read them, the sensors from a DK1 would be great candidates to transplant into the Power Glove and make it work properly :)

Watching this had thinking about the "virtual boy"; former developers of Nintendo stated that they did had a headset that is comparable to the "htc vive pro". They even complained that the Virtual Boy was so bad it was nicknamed "virtual dog".

I was fooling around with seeing what this would be like by synchronizing the inputs of 2 characters while playing my Jet Force Gemini co-op mod. I would cross my eyes after moving one of the characters to the side to simulate the interpupillary distance. I would readjust the character positions as needed (the mod includes support for a key bind that sets the position and rotation of other players to those of player 1, so readjusting was easy), and then I could kind of play "in 3D". It was pretty cool. Jet Force Gemini supports top and bottom split screen (and 4 corner split screen for 3 and 4 player sessions); of course, I was using the side by side split screen mode instead so that I could achieve the stereoscopic effect. Slightly related: it really bothers me that so many documents discussing stereoscopic vision use example images arranged so that the left eye view is on the left. I'm trying to see the stereoscopic effect without relying on hardware; you'd think the researchers would have figured out that the right eye view should be on the left regardless of the choice of viewing method made by members of their audience. edit: breh, the example video output you show at 7:30 has the same problem! edit that part to be R|L, damn!!

The lag between moving your head and visuals updating is what’s going to make you feel uneasy most 😂 cool project tho !

You went from portal on the n64 to something I like even more: VR! Also wow the stereo overloap looks so small on the DK1 7:34 x)

Very cool. A while back I was thinking about what VR would look like if it was released back int he day for either PS1 or N64 using an analogue video signal so I've got a few alternative rendering ideas which could be experimented with. 1. half-vres stereo: Render field 1 to the left eye and field 2 to the right eye. This would give higher horizontal resolution but lower vertical resolution than your approach. 2. full frame mono: Render full frame to both eyes, gain resolution but lose stereoscopy, might be worth the sacrifice at such low resolution. 3. R/B stereoscopy. Green to left eye, blue to right eye. Gain resolution but sacrifice colour information. Multiplayer would be cool too but would require even larger sacrifices, I'd probably sacrifice stereoscopy so you can have head tracked multiplayer at an acceptable resolution.

re 8:00: All VR systems deal with latency between tracking and display. What you do is extrapolate the pose (rotation in your case) based on rate*latency so that the image is in the right place when the light reaches your eyes.

Yo, using older tech to simulate VR is so dope! Will your new game be released on the Quest?

My lesser peasant mind wouldn't be able to have such a crazy idea and less to make it work, this is one of the craziest N64 modding/coding stuff I've ever seen, so sad that Valve shut your Portal project down but it looks like it will be just a mishap in a long journey of cool stuff. By the way, it looks like it wasn't that difficult either, you look like you pulled it off in a weekend, yet very impressive for a N64.

The way you combat tracking lag is through prediction. Extrapolate 1st and 2nd order motion (velocity and acceleration) by a few milliseconds ahead of the current orientation. Exact value depends on the specific motion-to-photons total system delay and in your case is obtained experimentally. PC, console and mobile headsets all have to do this because there's no framerate high enough to have unnoticeable motion lag without any remedial code - which is below 2 milliseconds (human oculomotor coordination is extremely fast and precise). Also you can shave off a few milliseconds of total system lag by polling headset rotation immediately before rendering down to a millisecond. Which is why DK1 has 1000hz polling rate despite only being capable of 60 fps. To get a proper VR output you need to render the final picture over a lens warp mesh, which does the inverse barrel distortion of your lens (straight grid rendered over such mesh should appear straight after passing through the lens). And the way you deal with dropped frames is by producing a synthetic new frame mid-rendering by treating the last one as a skybox (you actually just shift and rotate it but that conveys the idea better). Speaking of DK1, besides extremely strong SDE, poor resolution, low framerate and lack of positional tracking, large part of the reason it's vomit-inducing is its full-persistence display, which creates reverse motion blur when your head is moving (your eyes move across stationary screen, causing visual blur). If you can hack the headset backlight to strobe at 60 hz with low duty cycle, it'll appear a lot sharper. The SDE can also be remedied by simply installing an M3 privacy screen directly in front of the panel, or a very matte screen protector.

This alongside Kanes redone SM64 engine could possibly allow native Super Mario 64 in VR

I absolutely loved this. Keep it up James!

That little environment looks great

Congrats on reaching 64 subscribers!

a digital circus n64 game is closer than we thought

Just heard of Spellcraft 64 on Time Extension, so if this beautiful desert environment is a proof of concept for _that,_ well.... definitely subscribing! ESPECIALLY since it sounds like you're shooting for a flexible magic system a la _Mages of Mystralia_ or something (one of, if not my, absolute favorite spell crafting systems ever in a game). Can't wait for you to show off more incredibly cool things. Honestly I don't know why it's taken me so long to sub anyway, but at least I fixed that today!

Excuse my English I'm French. I'm a very happy subscriber of your channel. Your work is amazing and I am fascinated with your projets

I think it would be WAY COOLER if you could send all of the sensor data over three N64 ports, because then they're all in use!

Just imagine... having had something like this at the time. Just imagine

This is amazing, this looks extremely impressive, I never thought anyone would actually do a real VR experience on 1996 hardware and the in-game showcase environment looks very well done by Nintendo 64 standards. I would love to see other consoles doing VR, I remember back in the Wii days when I was a kid I kept imagining how cool it would be to have something like that in games like Wii Sports Resort, I always wondered if this actually can be done one day.

This could have gone good with Portal... womp womp This is just as amazing though. You are an N64 magician.

this dude is a wizard and the N64 is his wand