Can't wait for tech like this to be integrated with google earth + street view. The possibilities are insane.

Imagine using this at sport stadiums. Combining the video of all the cameras and then being able to create "Virtual cameras" anywhere on the field. You will be to see the game from the players' perspective, then zoom out as the ball flies through the air, and as it descends the camera zooms back to a first-person perspective.

VR would be more interesting and immersive with this technology! Particularly when it is integrated with Street View

Man GTA 194 in the year 2997 is gonna be so awesome

Is that the ue 5 matrix city

What a time to be Alive!!

Imagine, Google maps where you can walk anywhere the street view is available, this is no longer the realm of sci-fi, this could accelerate the speed of realistic games development, virtual reality worlds are within our grasp, I hope sci-fi writers are alive to see this

NeRF + Diffusion-based Super Resolution will be insane! What a time to be alive!

Pretty certain that's the demo city in Matrix Awakens.

Google Earth about to go hard

I remember Nerfs popping up two years ago and everybody was convinced, this is the future of 3D. Except a few good use cases, I don't see this go anywhere. Certainly not in games. Nerfs can't be animated, the lighting is static, making it impossible to rearange or stitch Nerfs together and to be of any use in simulation, you would need to calculate additional meshes to interact with.

Didn't NeRFs get obsoleted by Gaussian Splatting?

Do NeRFs actually generate 3D information (i.e. 3D point positions), or do they just render a 2D image of the 3D scene from a requested view?

The mind of the beast being built in front of our very eyes.

the thumbnail is a bit misleading

did they take images of the ue5 matrix demo city for the city test

Level 3 - City Level 4 - Country Level 5 - The World Level 6 - The Solar System Level 7 - The Milky Way Galaxy Level 8 - The Universe is the final boss.

What a surprise that their solution involves buying a bunch of Nvidia GPUs 😂😂

Shouldn't they get to work with Google Street, collecting all its images to have a better experience there?

Imagine creating an entire video game level by stitching together some concept art. That's where this is headed.

Lol this is so weird! I remember playin Myst IV years ago and thinking, " what if you had hundreds or more images, maybe you could turn around and walk more freely.. No, the game looks too good, it would never work for a computer" It sounds stupid but i really tried to imagine a smart way to have pre-rendered images melded together some cool way. Awesome

amazing as always

Amazing insights on the scalability of virtual environments! The distributed rendering process is a game-changer. Looking forward to seeing how this technology will evolve and its potential applications beyond gaming.

Is there any way we could experiment with one of these technologies? I have a rtx 3080 ti, maybe I could experiment with a small scene, but I don't know how to access this

Now this is more like it!

Real world simulation before GTA VI

What a virtual time to be run a city alive 🎉

64 graphics cards for the city geometry. or have i got it wrong? 4:40

Everyday goes by I believe more and more that we are truly in a simulation

Can this technique be used for distributed cloud training of LLMs and Diffusion models by communities running consumer machines?

Is the rendered city a 3D model you could work with in like blender or something or can you chop it into a bunch of models?

There is millions of pictures of cities in Google street view... Can you imagine take that pictures to use to make a real world with this technic? WoW!! What a time to be alive!!

I want Euro Truck Simulator 2 to use this to make the cities ultrarealistic!

"Could It be hell?" "Oh, yes It can... At least one." Are you sure you wants to cross this timeline?

this would be amazing for engineering structures quickly for prototyping

This would probably be very useful for TV show producers trying to get the best shot for some kind of documentary

For gaming you still a lot more, namely collision detection :)

I swear, the videos about papers you posted years ago birthed the age of AI we are having today

I don't think NERF will replace game worlds. What I hope is AI will be able to take NERF data and convert it into modular polygon pieces, and that it will be able to either dream textures onto the 3d space or draw from a collection of textures and combine them in such a way to avoid tiling and create micro details.

Having human engineers develop this first is great for AI training data. I believe AI can further improve this technique!

Oh my goodness, the papers are getting progressively capable, almost as if we soon will be able to simulate whole realities. This is probably just like a DALLE 1, wait till we get to DALLE 4

I think it's kinda disingenuous when people suggest that this technology will soon revolutionize gaming; as far as I'm aware, you can't relight these scenes, ever. They can't respond to dynamic lighting.

It would be amazing if NERFs could somehow be used to navigate representations of the latent space in a stable diffusion model.

Thanks Duckter. We need this for Google Earth Street view. It still won't prevent me from going on 40+km walks. 🚶

The graphics on the thumbnail looked more exciting.

Crazy to think that you in the future we could have a game similar to GTA that would in somewhat real-time update based on construction work or changes. So next time you have a quest or something in that city it has changed based on a real time event. I know, it would be super complex but it's fun to play with the idea.

What a time to be alive!

Károly gratulálok az eddigi eredményekért. Szép dolgot építettél fel. 🎉

I would gladly see a game that recreates real cities in low fidelity. Even if that meant low poly, limited color. There's something about real-world placement of objects that just cannot be beat for some genres of games.

Who else thinks in few decades from now, we will have Racing games with the map size of entire continents. Real copy of our own world. Similar to Flight Simulator, but with real useable roads. Mind blow.

In the future, you will scan the battlefield in real time, run scenarios and perform the most cost effective offensive, defensive maneuver, based in all the chess pieces and others you can simulate... Perhaps we can get the most cost effective wars we ever seen. Also, play GTA or wtv in whatever city you wish... Plus Flight simulator will be even more accurate.. I hope Im alive when we get the option to just live inside the Matrix....

Let's build a new gaming PC. I need: 1 super massive Mainboard, 64 RTX 4090 graphics cards, 16 TB of RAM, a Intel 18900K i11 CPU and full access to Elon Musk's bank account.

That Cybertruck aberration is everywhere in those simulations...

how long would it take for having a worldwide NERF made by satelital image?

wait, so it's like gaussian splatting but works with just a couple photos and gives you a model of the world instead of a group of a bunch of different stuff that build the model? this is amazing, now anyone with i assume nvidia gpu will be able to do 3d scans of objects

Awwwww... it's Leguna Seca as well, the best racetrack to experience in racing video games!

That city looks like the one in the Matrix demo they put out a few years back.

What a time to be alive in RTX 9090 + GTA 7 era

This is the time we hold our papers extra tight and call our neighbors to join in holding their papers as this is a wild leap, such a time to be alive!

Reading the book Permutation City by Greg Egan right now, and in combination with the development of Ai and virtual worlds, and I feel more and more like that meme from Twitter is turning out to be real 1:1: "Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus"

Using Street view imagery on the satellite 3d globe view so you can zoom down to the street and get so many more details. Or just a 360 completely smooth view, and being able to move around freely in Street View and related things. Yes please.

It's starting to look like we live in a simulation.

That's all fine and dandy but I need the blender meshes or the assets to actually make a product with it. 😅

That citty is the matrix city thing form that unreal engine tech demo....

Nerfs have major flaws if the goal is to create an environment you can explore freely. Most importantly, there's no 3D mesh. It's all just predictive changes based on nearby shots. So if you want to explore somewhere that's isn't close to the perspective of a bunch of pictures, then you're out of luck. It works if you're moving through the environment like a train following a track, but the more free you make the movement, the more attractive just making a 3D mesh that doesn't require recalculating every possible position gets. Aside from that, Nerfs are completely static. You can't snip a hole out of it and put something interactive there, or even something that moves. I challenge anyone to come up with an idea for a game where you move through a perfectly frozen environment and don't interact with anything, that anyone would be excited to play - it's less flexible than Zork and more computationally complex than Microsoft Flight Simulator. The combination of those two problems make this seem like a total dead end for gaming tech.

What's that artstyle from your thumbnail called?

Imagine using inpainting to remove pedestrians and cars from Street View photos, then combining them with satelite data to generate a 3D model of our entire world

Why not use stuff like octrees and something like some sort of 3d mipmapping LOD kinda thing, streaming details at big enough distances that the popping won't show when the camera moves, and maybe with some crossfading to further mask the popping?

That is not a city digitized from photos but rather the Matrix City Sample from the Unreal demo. Or it is digitized from photos of it. But that sample is already so detailed that nerfs would just do a worse job anyway.

Mind is blown. Virtual worlds are coming so fast.

Good news for akt with distributed computing

This could eventually enable a mix between GTA and MS Flight Simulator! Maybe GTA7?

Dope!

I cant wait until AI is smart enough to compete with reaserchers in intellegence and competence. Imagen How fast an AI could make conclusions without the need to rest.

Super!

Hey guys, do you think it will be less GPU intensive with Gaussian Splatting ? Great vid BTW 👍

Mr. Anderson ... we meet again 😂

That 25km squared is the UE5 matrix city.

is it ready player one distopia ?

Send this paper to Google map or Yahoo Japan team, and see what the next generation game would become.

2:02 What's with the truck turning right on red, and snaking that blue car?

How long does it take for a full render? Or is this interaction in real time why it needs to be on multiple GPUs? I'd be interested in something that's capable of taking in data, identifying objects, simplifying textures for multiple uses throughout the larger model. It could literally generate entire open world maps from google maps data. Of course, this kind of tech could also easily enable stalkers with drones, or military action with drones, or many other potentially bad actors with drones. And two papers down, with just camera phones.

Best catch phrase in recent history.. "What a time to be alive" -Dr. Károly Zsolnai-Fehér

that sample size seems small honestly. let's get this scaled! =)

Rendering developers using the same techniques they always do. Splitting the problem into parts and multi-threading. The results are spectacular!

Whole Planet simulations are next! 😎🤖

The fact that nerfs can handle reflections blows my mind

Not sure about running out of resources when data centers are now a standard size datacenter is 100K GPUs. And have cooling units the size of an apartment complex.

Remember NIVIDIA open source this 😉

I found it astounding that in case of all that complex programming, rendering and simulation we can still use some analytical equations to describe the problem and what has been done. Because the equations, looking very academic, can only describe the problem and the solution on a basic level, but really not how data was handled and special methods were applied to reach this level of of the model, being described in the paper ;-) This brings me up to my conclusion if we still need papers in the future to make humans understand what is going on. Or we find another format that could be understood by not academic people.

Good technology for Special Forces training

I thought Neural Radiance fields were too slow compared to Gaussian Splatting

Ehm... Area scale like the square, so 6 km2 is 36 times bigger than 1 km2, and 25km2 is (25*25)/(6x6) bigger than 6 km2 😉

It is not particularly surprising that Nvidia comes out with a paper to be able to use a lot of GPUs to make NeRFs, but a cool advancement nonetheless.

SLI is BACK babyyyy. Kindaaaaaaaa

hold on to ur papers everybody

It kinda seems like a Matrix type situation is almost inevitable doesn’t it?

We Are Living In A Simulation

I'm still waiting for an open world racing game where the whole Earth is present. MSFS2020 but for cars.

Hello, im from brazil and recently there was a llm called sapiens chat, and they are claiming on a podcast with Sérgio Sacanni a lot of bold things, but im not sure if thats all BS or not. Since you are my only ai news source that i trust, given you are highly technical, could you please analyse if their claims are actually real. And if they are real i do think they should desearve more international attention!