"the normal map is very important! lets scrap it" proper game development right there

Your artifacts aren't coming from texture compression. They're coming from sampling immediate neighbors. You can sample more neighbors or just slide your points out so that the hardware texture filtering samples more points for you. And you're doing a heck of a lot of work to arrive at the built-in normalize node. Just normalize the vector after you scale it.

The thing is that the normal creation works very inconsistently well and you are basically reverting back to CrazyBump times What looks good on a bark might look really bad on a brick. This makes sense if you don't have normal maps available otherwise all these operations will just cost more performance but less memory

Wow! I only was doing this with two textures. First texture I put a difusse (rgb) and roughess (alpha) Second texture I use a normal map compresed in two channels (r,g) and derive z normal from this two vectors. And I put ambient oclusion in the blue channel and displacement in the alpha. Incredible with one texture!! Thanks a lot!

A cool approach to experiment. Though this technique is inefficient and also can't be used for a lot of surface to represent. You are basically following the old Photoshop process of creating textures for materials using Albedo. That approach is nice but will never produce better result. Also Because you are calculating them in shader instead of using Textures, it can be calculation heavy. I feel the trade off is not worth it. Also for the Normal, You can use a more detailed Height map and then process it with the mesh normals to get a more accurate detail normal.

Wow, you clearly know a lot about shader programming. Well done. I've never seen a tutorial like this :) Liked and Subbed.

"Speaking of which, would you like to explore more the world of photogrammetry" while moving camera around the object with the chunky mouse movement 👍 I like it

Should be titled "How to unnecessarily hyper complicate a shading process to achieve a similar result in an unoptimized way" or "How to have 10x shading complexity and bottleneck instead of 1x memory bottleneck" Brilliant 👍

Virtual Textures/lightmaps/shadows are there to not have the Vram issue, doing that is transferring one issue (who does not exist that much anymore) to an other (number of instructions), that being said I think it's a very interesting approach! I would completely consider that kind of solutions if I wanted to have a very very light project with tones of reused assets using the same textures set everywhere :) !

big fan of your channel sir. was thinking of a wayout to have simplified props and character shaders for metahuman and megascans. found a thing to keep in minds. brilliant

I love watching videos about optimization. I make games for phones and try to have the best graphics so I need this kind of stuff

Adding an alpha channel doubles the amount of space and memory, because unreal uses 16 bit for textures with alpha. Also, in photogrammetry, the normal and height are not derived directly from color, but from the 3d point cloud generated from hundreds of images. Regardless, this is a super fun exercise. The best idea for textures like this is to do two samples Basecolor, and normal. Because the blue channel is not needed in normal, you can put AO, height or roughness in there. Common technique is to put height into normal, and then have the AoRM texture. Still 3 texture samples, but not so much more space. Alternatively, if you care less about samples, you could do basecolor, then have height, ao and roughness in a second texture. Same amount of memory as your technique! Stepping issue is due to precision and sample distance, not compression I think. Compression would have larger blocks with smooth details in them.

Every interesting! Could be really useful for stylized/mobile games. Thank you for video!

Really good explanation thank you

Since it is a grayscale texture ( and normal Maps only need two channels) I'd create a proper normal map from the texture. Then put grayscale texture in the third Channel and use an alpha for roughness and metallicity. That would give the dimensional normal effect without compromise and still have a single texture. even more efficient would be to use a single gray scale image for the texture and then a constant variable for roughness and metalicity.

Thanks a lot for you wisdom tutorial!

this is amazing, thank you

Great quality video. Well done.

Your videos are amazing, pure usefull content

there is always a tradeoff between compute time and storage (in this case memory), if memory is a concern calculating many of the maps for a texture is a good solution, good video

finally next video from my master

I use a texture format called "ARM" which is 3 textures inside one. Ambient occlusion = Red channel Roughness = Green channel Metalness (or use for different mask if material is not metallic) = Blue channel. This reduces the amount of textures from 3 to 1 and simply uses RGB values. I use ARM textures almost everywhere because they save a lot of memory and still look really good. And because they still can provide a lot of material information or since their unused channels (such as metallic) can be used for various types of masks they can be used for many things. Also you can reduce the amount of texture samplers by setting the sampler nodes to "shared wrapped" mode. If you then sample the same texture multiple times it's seen as 1 sample.

What you built is a very specific shader for a very specific case. You sacrificed alpha and roughness because you didn't need them in this case. But it makes the shader useless for any other texture that needs them. For example, roughness may not match the albedo.

I could see a simpler and more performance-friendly version of this being really great for adding small, low resolution models for finer detail, but for more prominent models such as characters, buildings, major set pieces, etc. it would be better to have all the real textures. I think it is more practical to combine things like roughness, height, metallic, into a second texture, and use a separate normal and color map. This gives you three textures as opposed to 5, and is typically much better looking than pulling from 1 texture as you get all the benefits of PBR graphics, with the smaller memory footprint of less textures, but less shader math than your method. While I agree that many modern developers tend to waste performance on poorly compressed textures and using many B/W maps to achieve what could be a single texture, it is important to remember that these "extra" maps exist for a reason, and that reason is to simulate the tiny changes in a surface based on real materials, rather than the older method of just approximating everything with fake or baked-in texture trickery. These tiny surface details cannot truly be achieved for most assets with a technique like this.

I notice significant drop in detail and shading clarity on the derived version compared to the normal mapped version. Moreover about not being generalizable... you're absolutely right. MOST game assets won't be able to derive their normal maps because most assets won't be using maps derived from one source. Most assets will be using maps derived directly from a model rather than from texture, and many of these textures will be greatly different in resolution from each other. Normal maps in particular are often 2 to 4 times the resolution of diffuse or height maps, and roughness maps tend to have absolutely catastrophic results when not mapped exactly to the right texel brightness. I've had no end of trouble trying to paint my own roughness maps by eye or using color sampling from other maps, because it just doesn't map in a simple linear way that can be interpreted. The reason these maps are usually baked into stored textures is actually because deriving them at runtime is... a bad idea. You lose a lot of quality (as I noted in the comparison here, though you seem not to have) if done the simple way, and you lose a lot of processing power if you do it the correct way that image processors that actually produce _good_ textures do it. (most texture processors do not produce good textures, which is why some texture bundles are a more expensive than others... more effort, better technique, and not all derived from diffuse... though some are just scams too.) If you really want to cut down on your memory footprint.... you can just not use lighting. Bake your lighting data into the vertex color channel of your objects, derive a general edge modulus value from your normal map to use on the baked lighting, and apply a diffuse texture. If you want height data... sample your diffuse map at about... 1/4 ~1/8 scale and normalize the values, or just bake the correct shape of your models.... into the model. Vertices and polygons are generally less memory intensive than textures, so... unfortunately, having several million polygons in your scene is more efficient on memory than having highly detailed textures OR derived texture information. And usually... the extra polygons aren't needed, because you can get rid of hundreds of thousands, quite readily, for one normal map or parallax map that is significantly lower resolution and can be used across multiple objects, than the footprint of the high resolution mesh. Most mesh details aren't necessary at all, because you won't see their silhouette or deformation ever, and this is the real problem with modern development. No one... is optimizing, because they've been taught that they don't need to anymore. A little work in the front-loaded asset development workflow goes a hell of a long way toward making a game run better in all ways. Taking careful thought to how your assets will perform when making them can avoid the headache of trying to get more performance out of the game you've already put together.

The issue with moving the height map into the alpha channel of the albedo map is that the alpha channel is generally uncompressed, meaning it takes up as much memory as the RGB channels combined. You're better off having two RGB textures than one RGBA texture.

Excellent

embedding textures in the alpha channel is my life for the past year. In the Source Engine (the engine for half life 2, portal, CSGO, etc if anyone didn't know already) you have to embed roughness into the normal map's alpha or else you can't have both at the same time

Technically, you could just bake in Displacement (Which Normal data is usually inherited from) in an external 3D program, and simply use a roughness channel in the alpha channel of the diffuse. To make things extra crazy, you could retain a metal map encoded into the same map as the roughness at the cost of half precision roughness by making metal roughness features use the top 50% of the value range and diffuse roughness for the lower 50%. Then just feed this new encoded roughmetalness into a multiply node set to 2, and then a pair of IF nodes. For the roughness output IF node, just have it pass through for values less than 1, and have a roughness-1 version come through for values greater than 1. For the metal IF node, I would just set a 0 float for values less than 1 and a 1 float for values greater than 1. The biggest caveat of this method is that nonbinary metal textures would have to be crushed into binary ones. Another caveat is that every mesh using a different scale for the texture would have to be tessellated individually Another thing worth mentioning is that you would not always need one vertex per pixel on your mesh to retain normal data on the mesh faces if you use an angle based dissolve on the mesh. But you might need to. And you definitely would need a lot of vertices. This method really leans heavily on Nanite to do heavy lifting, but, so does using WPO inputs. This also lets you skip the unreliability and low quality of WPO and Derived Roughness maps and ignore the vertex shader. It is also an overall simpler shader graph.

our TA's have that Material Template that utilizes just the base color only (for optimization purposes of course) and it automatically converts to Height, roughness, normal. But we Environment Artists mostly use his other material template which uses (Base Color + Height, AORM (AO, Roughness, Metallness), and Normal Map. It just looks better IMO and easier to control. Although our TA recommends Atlases for environment to save memory. btw, AORM can be set to a lower size to also save memory, doesn't affect the final output that much.

Very interesting technique. I wonder how much calculation per pixel is to heavy. I guess it's a race between memory usage and calculation time and dependy on the current situation. It may be better to simply reduce a 4k texture to 1k or 2k to reduce the memory footprint and make usage of a ORM texture.

It's worth noting that in unreal using the alpha channel doubles the memory usage of the texture compared to using one without alpha. RGB uses DXT1 compression which is 64bits per block, while adding an alpha uses 128bits per block (64bit for the colour and another 64 for the alpha values). So purely from a memory point of view a 2k RGBA texture is as expensive as 2 2k RGB textures. Spreading it over 2 textures has the benefit of possibly giving you 2 additional channels to play with, though you have less bits per channel to work with. Another useful thing to note is that the bit count per channel is different. Simplified you have 16 bits per RGB value, split as 5:6:5 bits, meaning you get better quality out of the green channel.

amazing! i will pay for your tutorial

I am not very good in shading process but after all these years of development I understood one thing that whatever you do, do not ever mess with normal map texture.

"do they look the same?" my eyes: ummno? "good!" OY!

Interesting. Was not thinking that calculating textures in memory would be better then sampling a texture ... but it makes sense. Unfortunately i would not want to loose roughness textures. But would it be good to use the same textures across multiple materials? If for example i have a the same roughness texture i put on multiple materials. Currently i use AO, Roughness and Metal in one RGB texture for a material. But would it be a benefit to split this and reuse the same Roughness texture across materials?

This process saves memory, yes, but trades memory for compute. You can get better results by packing and atlasing to shrink your texture inputs into 2 textures anyhow and more objects at the same time. GPUs continue to gain routine op improvements to make PBR math faster and faster, so skipping these inputs can actually be slower on top of being more compute heavy. Neat experiment, and I do agree there was a loss somewhere for hefty ops in favor of brute force, but this method is not the way.

This reminds me of a project i did as a hobbyist before i got into the industry where I used height map generated terrain so of course had to calculate the normals in the shader. However, like any engineering problem this has massive tradeoffs. 4 texture samples per pixel? That could be a lot of bandwidth, granted there might be some values in the gpu cache lines depending on the tiling mode of the textures but I'd be curious running this through something like pix to see the overall cost. Also I wonder about the VALU cost of all the calculations per pixel across shaders running on a gpu unit. With block compression of normal maps and other textures to help with memory I'm not sure if this would be a net win. I could be wrong, but i need to see metrics. Creative solution though, and good for thinking originally

I use only 1 texture as even bump and normal map in unity, in many event games and small apps for phones etc. and always bake them in 3D + in Unity. It was way more efficient, but un unreal it look way more better (compare to standard shader in Unity, not other like uber etc.).

Nice approach! I also experimented with a different way of dealing with masks for color overlaying, in order to pack a max of 44 different masks in a single texture, instead of the classic 4 you can get by using only the RGBA channels. Maybe we can have a chitchat on that if you're intrested in making a video about it to share the knowledge

The one on the left looks more real, maybe it's the lighting, or something. The one on the right looks kind of off? Oh found out why. I wouldn't do this but I like the idea. Plus you can highly optimize textures already anyways. But it could be a good decision.

Wow, very interesting material! 👍But in theory, is it possible to pack multiple grayscale maps into one channel and split them in a shader? I saw something like that in Bungie presentation from GDC, they packed 7 params into 4 channels

The fact that he's doing this with purely math and advanced shader programming knowledge is genius enough for me, regardless of what anyone's opinion might be of the method

For the normal calculation, could you do a single small pass at bluring it to help with the steps?

Inaccurate methods for all other textures including heightmap, which supposed to have higher bit than the normal texture we're using for color maps. But will this solve the infamous vRAM limitation issues? because small inaccuracy but enabling more stuff we can use in a scene is subjectively better.

Another tip for normal maps: I had good success multiplying in a blur constant to my image kernel texel sampling so I could play with how much angle I wanted to matter. Anything below 4 doesn't seem to overlap much and allows you to fuzz your angles since sometimes you don't want hyper detail but other times you do. I also added a saturate before the derive normalZ because depending on what that multiplier is you can push in some dead values there also by accident. I didn't notice any appreciable performance change BUT this does speed up my asset creation pipeline because with a lot of things I can go over to a single CH packed texture instead of a CR and NH and that means less to do and less to keep track of.

Would love to see some benchmark to see how it performs

Awsome explanation!. Is this more expensive in performance than using the normal map?

i developing VR game. And roughness, specular, ambient occlusion dont stand out as solid material like rock or brick. O i dump them all. Only use diffuse and normal, and it look good, combine with custom data, optimize texture size, optmize uv, optimive vertex count, optimize lod, optimize mip map, optimize culling, etc.

First of all, splendid work! Love it!

molto molto utile grazie :D

Maybe i dont understand exactly how channel packing works in term of optimization compared to this method but wouldnt be a channel packed texture a similar gain of performance while using much faster workflow? If we lets say use a channel packed texture with Diffuse, Normal and Roughnes (or anything that's needed) you can just basically plug the texture add a fraction of instructions you use to the material and the work is done. Please correct me if im wrong, and thanks for awesome work :)

A good way to test this for real world cases would be to turn it into an asset package or addon that crawls through and modifies all files in a project. Then any project made in the usual pbr style could have a second copy made for testing and have this modification applied to all materials project-wide. Then it's just a matter of running some tests to see results of running things this way, especially if it's a realistic graphics project, as apposed to stylized. I'd be really interested in the frame times and vram usage in said comparison of a project running on the usual method and this one.

you are very smart, take my sub incredible value tutorial thank you

이런 방법으로 하나의 텍스쳐로 각각의 텍스쳐를 구현할 수 있군요. 어메이징.

One question Master?. Minute 5:59 Calculating the pendient (derivate X and Y of the neighbourpixel).... Is it very fast use in this step to use DDX and DDY???? I think that you can avoid repack the normals and normalize them. Thank´s. :)

Is this available for download somewhere? I don't think I'm savvy enough to create this, but I'd like to play around with it!

Hey! You could actually subtract the vectors by 1, then sign the result, then lerp with the signed result between your divided result and the default mult100 to avoid the if statement. ;D

This video didn't make me realize anything, but it made me relive an old idea I had with transparent pictures. Is it possible to remove all white from the picture, and translate it into alpha channel? This way, if there's a white background, the transparent picture will appear to be completely normal. If the background weren't white, the texture itself would look rather dark, or deprived of any light. I also would like to do this with any kind of color, not just white.

The only textures you need is albedo and some kind of spacial information, depth or height, if you know that it's metal or non-metal by default. Think of how real-life objects work. You can pack this information in one RGBA texture. Other maps can be derived at runtime. This is a basic long-running argument of "processing" vs "precalculation".

While yes this makes the files smaller, when running the game procedural textures blow up using memory somewhat comparable still.

Is there a particular reason you didn't use the "height to normal" node instead of all of the custom node setup in the shader graph?

Smart, very 😌✌️

This could save a lot of storage space in larger games which would otherwise have many textures, nice

Hey. This method of generating normal maps. Can it be used for generating a normal map between two blening materials? Lets say. I have a metal and a paint on it which is added using vertex paint. Is it possible to add a normal map on the edges like in painter using this method? Or may be you know other method? This is probably a good theme for a video btw.

Could you use your magic to make flow map based high map, like lava flow from mountain?

Very interesting. I have a personal question, are you italian?

I'm seeing an application for AI here. Not even a heavy AI. You've basically correlated aspects of a color map to other maps. Those correlations can be developed into models automatically. As you've shown it doesn't take that large a model to perform the correlation even when done by hand. A AI model could likely do it with an even smaller model, and likely consider pixels beyond immediate neighbors to drive useful results. The only area I'm not very familiar with is generating those high frequency features. My intuition is that is possible as well based on what I've seen other people do, but I wouldn't know how specifically to implement it myself. But with that you could compress textures pretty painlessly.

For the Roughness section, aren't you having to store the values that you're calculating from the original texture in a separate memory instance from the texture? Isn't that essentially having two textures in memory? Transient though one might be.

i made a material with paint chipping on the edges.. i did this by using vert color and a texture mask to randomize up the edge vert color so its uneven. Is there a way to generate a normal map for the edge of the paint?

Bro thanks for normal map and btw i try to make a material function for later on and engine crashed lol👍

Nice idea, but is it even worth it now that we are using virtual textures pretty much everywhere? It also needs manual adjustment pretty much for every material, I wonder if this workflow would actually pass and if this method is wildly used in any projects. Maybe for something like mobiles it would be more beneficial ?

Ideas on how to use this to fake or rederive the ambient occlusion map would be neat.

you might’ve saved some texture samples but you made the shader massively massively more processing intensive. sampling a texture is much faster than this. Is saving the VRAM worth taxing the fill rate like this? I would never recommend that anyone make something like this

Can anyone tell me what those light-blue BaseUV nodes are at 3:23? My materials always look like spaghetti and they look perfect for me.

How computationally expensive is this? I would assume it's a bit of a trade off, less VRAM usage but more processing time?

The biggest downside to this approach is IIUC going to be that it needs to use its own material. If all of your megascan materials share the same material and just vary by what textures they're using, it's typically going to be easier on the GPU to draw multiple actors using different material instances than it is with fully different materials. As with all things, there's a tradeoff to be made - if you're strapped for VRAM, deriving normals from the heightmap texture will certainly help.

Can you add parallex onclusion instead of displacement

Reminds me of ORM textures.

is there a derivative node in ue? if so, you can derive the exact same normal map from the bump accurately. right now your normal conversation isn't as accurate as it can be.

What will you do if there are some contrasting patterns on the texture, and/or materials with different properties? Rhetorical question.)

Will this cut down under CPU usage or GPU usage that a game needs to run... If so can you put a link under this video or video or at least to this comment about other videos you made that have this level of scientific efficiency genius so I could grow in your light and become like a like a UE5 scientist too! 👊😎'kapaw

Can you list the draw calls of both?

Would it not be more effective to just take the min between vector {1, 1, 1} and the derived normal? If statements are costly

Hi Visual, I am not 100% sure but tessellation should not work in UE5 shaders. It was removed in favor of nanite. The only tessellation like geometry is only available in the water Objekt and for the landscape. Other than that the video itself was very informative and the a very interesting approche. Love your videos a lot!

I think this is a great excercise, but not very useful. This is a super heavy shader just for abasic PBR material. Taking this and adding in even more shader effects makes it very hard to work with and very very expensive compared to just using 2 texture samples compared to your "complex" math and many many more texture smaples. From my knowledge, texture samples are very slow. I think another btter method would be using texture arrays with small (512) textures with high tiling and texture maps that define the geometry features such as AO and Cavity (can be combined in a single channel and seperated in shader, Edge Map, Curvature Map etc. You will have a much cheaper shader, smaller texture sizes, higher detail and can make a shader where yu can rotate a mesh and textures will render appropriately so a single mesh rotated will appear like a different object.

The one on the left has much more granular detail.

is calculating shaders in real time is more performant than loading textures?

Interesting, but i think i will not do that shit at least for the normal map and use another texture instead ahahaa, and yes, memory is an issue indeed, you cannot expect to import quixel megascans and to make a game with it out of the box

what are you using for text to speech?

You said RGBA = 2xRGB when compressed ... is that true? Or did you mean its just bigger by 1/3

the real proof here is to made a small landscape of some sort with a town and forest. after doin it normally, copy it and made one with your method. then put then to the test. which has better frames and results.

This is not very useful in the case of photogrammetry because you can capture normal map and roughness. You'd need a darkroom for that. If you need to capture something outside, you use a tent or just scan at night. For normal map you need a small metal ball and a light source that you can move. And software, of course. Substance Designer or Details Capture | Photometric Stereo. For roughness you need to take two pictures without moving the camera, one picture as usual and the other with polarising filter and polariser on the light source. This will give you two images with and without glare. The one without glare will be our Albedo. Now we turn both images into greyscale, count the difference between them and get something similar to rougness.

Its a good technique, but if you need specific normal map with specific features its not the wat at all.

Is such material not difficult for GPU? Runtime calculation. I'll have to test later🤔

But will that really help performance 🤔🤔 idk man… I’d say just save time and go the old fashion

can someone pls make a GLSL code or HLSL for this pls?

Why not working for me? I analized every sec video but cant find mistake in my nodes, my normal map is flat, could you upload somewere this material pls?

Putting it into Alpha makes the engine consider it as 2 textures pretty much

This dude sounds like Steven Hawking's synthesized voice. LOL