And beyond that... darkness

@@orangehatmusic225 And now we are in the void.

@@dmellis Not me my friend... not me.

never underestimate the cameraman

I was so confused by that

That was my first thought. I cannot wait to get my hands on this

From my experience, a scene in VR looks at least three times worse than on flat screen. In VR, you'd likely notice the low resolution of the textures and all the blur and artifacts introduced by this technique.

this barely touches 144 fps on FHD you need way higher resolution for VR as of yet, still impractical more improvements will be needed

@@mihailazar2487 Understandable since this is an entirely new way of rendering. It’s going from the ground up all over again, back to the pong days. We’ve got a long way to go until VR implementations are relevant

@@mihailazar2487 dynamic foveated rendering would help

"WE"? Give me a break. We couldn't beat a primitive virus recently, collectively. There's no "WE". There's just 70% of total idiots, 28% average IQ and 2% that drive progress. There's no "WE"!

*all of a sudden

@@alvallac2171 and what a sudden it is

AI is and will continue to push our advancements exponentially

Ok now switch the sun off and go to moonlight. That's the only drawback with this method. Love it nevertheless.

because it is though. thats a picture of a real place. And clear faults in how the light rendering was done on those ceiling beams or whatever. Still awesome though.

​@@waitisthatcobaltit's also missing all relevant imperfections to be really realistic.

its cool. i like stuff like this. some people are trying to animate using this technology, as well as doing more creative stuff.@@haifutter4166

but yeah lol ofc. @@haifutter4166

@@haifutter4166 is is real, is nothing cg, it is a 3d representation of all the pixels of the image in 3d space as small faded elipses

Basically the same trick old games do to make mirrors / reflections, render the geometry flipped

In older video games, reflections were simulated by rendering a lower-quality, shaded clone of the player/objects in view. So, not a true reflection, but nobody can tell the difference. It's the sane principle here, just scaled up exponentially. The most realistic reflection possible would be to perfectly clone everything in view, flip it, shade it under the water and align it with the cameras point of view.

Thanks @Cloudef @FelixHelix for your replies. Hm, personally I think the 3D game analogy is not helpful here. In those cases, an explicit geometry was created manually by the modeler to recreate the illusion of a perfect mirror surface, as a feature. Here in the video, on the other hand, this is not a feature, but a an unintended consequence of the method, Gaussian splatting, to visualize the 3D point cloud dataset generated from the original image / video frames. For the pixels in the multiple frames that captured the pond surface from multiple angles and distances, the algorithm constructs point clouds of the object reflections which works because it is a perfect, smooth mirror, so each 3D points' coordinates can be accurately inferred (triangulated). To the algorithm, the data from the reflection is just more spatial data, it does not distinguish between it and the rest of the geometry. Whereas our brains naturally segment what we see here into "real object" data and "mirrored objects" data, because we know this is a pond and how it should look like. So, this mirror dimension actually not a feature of Gaussian splatting but a kind of cool "bug", that has to be manually corrected after scene construction. This can also be seen from the fact that the mirror dimension scene is incomplete. It only represents the objects that the splatting could infer from the data, not more. Right?

(In response to my last comment) If this is true, it would of course be interesting to see what it does with image data from a perfect mirror surface that is not a plane, like for instance a perfect mirror sphere. The point cloud data from the mirror objects would need a non-Euclidean transformation, then, I guess.

@@Irrazzo You'd be correct that the reflection is just a byproduct of the render, unlike specifically placed or modeled reflections prior. It's a consequence of the render being essentially so high quality that it is indistinguishable from reality, and the method of rendering is revolutionary; we will likely see this exponentially grow to be mainstream and have integrated support for games, or in the future, VR and movies. The splatting essentially renders every possible point as a blob of data in space, and they each weigh their data depending on the point of view of the camera as the environment is rendered. Anyway, I think you actually know more about this than me! I'm just rambling. Lol

Splats themselves are a form of compression. However there are often many distant splats that could be culled, skys and long distance scenery views..

only with sufficient conmpression though. imagine having a each scene be several GBs big, too expensive and slow

why not have such a service be open source? stop relying on companies like these.

@@kkon5ti *having each scene

@@amentco8445who’s gonna pay for the petabytes of data storage?

Gogle street maybe too soon, but Ai art generations are going into this direction, although even that require a huge computational power

It would glitch completely.

osito gominola

That’s just another method of photogrammetry, lol

What do you mean thought this was real?

Yeah its such shame that we can only get this working for static environments at the moment, Though who knows, im sure one day someone will come up with a solution.

@@Tea-oc3gh a month later and that is no longer the case as one can use the Unreal plugins that rebuild the gaussians as niagra particles that can be animated

*What were the settings (because "settings" is plural) *fed

stop

it's quite different. 3DGS models the scene as a bunch of point splats with associated 3D gaussians serving as texture. These splats are then rasterized, thus forming the image

Yeah I got an idea of the final rendering method, but I am just wondering about the process, if it builds on top of a NeRF training (like photogrammetric meshes build on top of a point cloud first) or if it trains the photos directly with points splats directly from the photos or point cloud? @@TheYellowBlueWhite

Btw is this lighter to run than NeRFs?

@@BenEncounters yeah it is lighter, but needs 24 gb vram gpu

@@OxTofunmi You can run it to 7k iterations on a 3060 with 12gb. Edit: And that's for training, it doesn't need much vram for rendering.

yeah those were the gaussians. dannng, right into the reflection

It's all light based, so I guess not. Could've been off it was mesh geometry based

Not with gaussian splats, no

With help of Ai

yes, eventually that will happen...but not without gargantuan memory footprint

That's the Gaussian Splatting referenced in the title, Gaussian Splatting doesn't use a mesh.

At the end when the camera goes into the pool you can actually see that the reflection is fake. The algorithm that created the space didn’t understand what a reflection is, so it rendered an entire map segment that mimics a reflection when viewed from above (the angle at which the scans were first taken).

Given that you can move the camera and take more photos to collect more data about the environment, I wonder what would happen if you captured a mirror and then also behind the mirror

@@Jamiered18 that’d actually be really cool to do with an endless mirror or even one of those mirror mazes. Imagine the weird alternate dimension the program creates

basically you start with a point cloud (or without but it will be slow to train, full of artifacts) and 3D gaussians with 3rd order Spherical Harmonics for color are used to fit the scene by referencing given images and backproping the renderer. The PNG vs JPEG lossless/lossy comparison is a bit weird here because the gaussian splats actually try to store way more information in a more compact way and generally, both methods can't store the ground scene

20 years ago was premiered a Minority report movie, I see today basically everything which was showed there, except hologram projectors or police spider robots and flying swat things. But even them in movie have a much better version of this. By the way Ai already predicting your next move or action, so clairvoyant prophets kinda already here.

*an Unreal (because "Unreal" starts with a vowel sound)

That's because this is like a 3d image.

But.. This is real...

git gud

​@@whatisrokosbasilisk80u a homo