NeRFs: Neural Radiance Fields

NeRFs: Neural Radiance Fields - Paper Explained

Рет қаралды 34,377

Күн бұрын

❤️ Support the channel ❤️
/ @aladdinpersson
Paper: arxiv.org/abs/2003.08934
Full title: NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Paid Courses I recommend for learning (affiliate links, no extra cost for you):
⭐ Machine Learning Specialization bit.ly/3hjTBBt
⭐ Deep Learning Specialization bit.ly/3YcUkoI
📘 MLOps Specialization bit.ly/3wibaWy
📘 GAN Specialization bit.ly/3FmnZDl
📘 NLP Specialization bit.ly/3GXoQuP
✨ Free Resources that are great:
NLP: web.stanford.edu/class/cs224n/
CV: cs231n.stanford.edu/
Deployment: fullstackdeeplearning.com/
FastAI: www.fast.ai/
💻 My Deep Learning Setup and Recording Setup:
www.amazon.com/shop/aladdinpe...
GitHub Repository:
github.com/aladdinpersson/Mac...
✅ One-Time Donations:
Paypal: bit.ly/3buoRYH
▶️ You Can Connect with me on:
Twitter - / aladdinpersson
LinkedIn - / aladdin-persson-a95384153
Github - github.com/aladdinpersson
Timestamps:
0:00 - Introduction
0:54 - Goal of NeRFs
2:36 - NeRF network architecture
4:35 - The key idea you need to understand
6:17 - How NeRFs learn and work
12:15 - Intuition behind volume rendering
15:15 - Loss function
15:55 - Trick #1: Positional Encoding
17:36 - Trick #2: Hierarchical sampling
19:35 - Ending

Пікірлер: 79

@AladdinPersson 2 жыл бұрын

Thinking about doing a NERF implementation from scratch in PyTorch next

@markwindable 2 жыл бұрын

Awesome! Would be very keen on a tutorial

@emrecagataykose 2 жыл бұрын

Please do! I'd love to see it. Also, could you do an implementation video where you go over a new paper and explain your thought process on how to implement it at the same time? What are the alternatives for each of your decisions, how do you go about solving errors, etc? I haven't found any video where I could observe an ML engineer's process of reading and implementing a paper from start to finish (with all the debugging, researching, refactoring, testing, evaluating, etc.). I would watch it, no matter how long it is. And I believe it would be very valuable.

@simonedeangelis655 2 жыл бұрын

Looking forward to it, awesome content as always! Does the paper tell how much photos of a scene nerfs need to get a good 3D reconstruction?

@mizhou1409 2 жыл бұрын

Damn! That would be awesome!

@thousandTabs 2 жыл бұрын

Definitely, can't wait! Great intro to NERF, thank you for your explanations :)

@_bustion_1928 2 жыл бұрын

I am so glad that I've found your's channel. Started from zero, but now I am working on the StyleGAN on my own :)

@matejsirovatka 2 жыл бұрын

This is like the best resource i've found on this topic, would love the implementation from scratch

@AladdinPersson 2 жыл бұрын

That means a lot, thanks

@jeffreyeiyike122 Жыл бұрын

please i am having issue running it. please can you help out

@evgen8600 2 жыл бұрын

Thanks, Aladdin! It would be interesting to see your implementation :)

@antaoalmada1475 Жыл бұрын

Thank you for the great explanation. It would be awesome if you could make the NeRF from scratch video to better understand the formulas. Even a basic implementation would be great.

@Ali-wf9ef 6 ай бұрын

absolutely amazing explanation. Thank you.

@deeper_learning Жыл бұрын

Thanks for the simple explanation👏

@jakobringler5856 2 жыл бұрын

An implementation video would be fantastic!

@erfanshayegani3693 Жыл бұрын

Nice and simple explanation. Thanks

@sando_7 Жыл бұрын

Thank you for making such a great video :)

@joelu1758 Жыл бұрын

very nice explanation on Nerf in such an easy way

@user-pf4bh4wb2j 2 жыл бұрын

A code implementation would be awesome!

@mgaschenbeck Жыл бұрын

I like your intuition on the volume rendering. It helped me understand this topic. However, the description of T(t) is a bit misleading IMO. You say it is "How much light has been blocked up until point t." It's actually the opposite of that. My description would be "How much light remains up until our current point". If it's a large number, we have a large amount of light left and we will use a lot of the color when rendering. If it's zero, ALL the light has been blocked. Hope this helps some others.

@nazmicancalk2415 6 ай бұрын

It would be very interesting to see this from stracth implemented man! :) Thanks for your videos, your explanations are simple to understand and best!

@jiaminghu9260 Жыл бұрын

Nice video. Very helpful. Thanks

@sayeedchowdhury11 8 ай бұрын

did you end up doing the pytorch implementation? If not, please do, it would be great, thanks!

@dereknewman6106 2 жыл бұрын

Would love a code implementation from scratch!

@holoceo Жыл бұрын

Is (x,y,z) a coordinate of a point in the scene or coordinate of a viewer?

@FLLCI 2 жыл бұрын

Alright. This is very interesting for sure, like basically you do 3D reconstruction with Neural Networks... However, I don't really see use cases of this based on the fact that we need to do this for each particular scene. This can be easily implemented with the 3D reconstruction algorithms in practice.

@mattizzle81 Жыл бұрын

Yeah that’s the thing. Yes you do need special hardware but I’m happy with the results from my iPhone lidar scanner. No training needed and fast. Same or better results

@EricDongh2p Жыл бұрын

Perhaps there are missing data or need to create a scene from synthetic data. If you have all the data, sure other photometry methods work better.

@muhammadtarekrefaat4901 2 жыл бұрын

can you please make a video for the implementation

@ishandandekar1808 2 жыл бұрын

Hey Aladdin! love your videos. I made 2 projects related to deep learning and want to start replicating papers. I have some proficiency in Tensorflow (2.0). Wanted to ask how do you come about replicating papers i.e. how do you find these papers and start replicating them. Also, If I have replicated a paper, should I add this as my "portfolio project".

@xXxIMMORTALxXx Жыл бұрын

At 16:30-16:45 where you struggled to explain why neural networks work better with high dimensional data. Am I correct to say that neural networks need high dimensional inputs in order to avoid linearity, which will make the gradients vanish resulting in the networks stop learning? Btw great video explaining the concept in as simple manner as it could get for convoluted topic such as NeRF. Subscribed! :)

@4mb127 2 жыл бұрын

Trying to learn this, understanding and experimenting with the Fourier features is much easier with 2D images.

@ankanbhattacharyya8805 Жыл бұрын

Still I didn't understand how the training works. So, is it like we build dataset of lines (rays) and then train MLP on that? Can you please explain and show how to do it?

@beowes 3 ай бұрын

Thanks

@shikharsaxena9989 2 жыл бұрын

please explain about the creation of the dataset. How poses will be created from my own scenes?

@jayakrishnanvenugopal4179 Жыл бұрын

You can use COLMAP.

@talhaejaz7651 Жыл бұрын

This is amazing. Please do slam with nerf paper explanation

@miras3780 Жыл бұрын

Does anyone know, how does NeRF know what is occupancy of any point along ray? does it just minimizes the loss and thus learns both corresponding sigma and Rgb of the scenes ? To be honest, can not understand how does MLP learn the distance and corresponding RGBsigma of each point on the ray, intuitively it should know some sort of 'depth' of the scene(maybe through the other view pictures)?

@nitisharora41 Ай бұрын

Nice!

@Lerise 10 ай бұрын

here x,y,z is the camera location right ? not pixel location

@AlbertMunda 2 жыл бұрын

Good Job bro

@vishwak1998 2 жыл бұрын

Would love to implement of the paper too

@rabia3746 Жыл бұрын

Hey Aladdin, Could you make a video tutorial about TE-GAN ? This is for thermal image enhancement. Good luck and thx.

@generichuman_ 2 жыл бұрын

Hey Aladdin, if you did a paper walkthrough and implementation of GFP-GAN, you would be a god amongst men.

@claude_by3495 Жыл бұрын

Great video!!! Could you give us a NERF implementation from scratch in PyTorch?? Please!!! It would be greatly helpful to a graduate student!!!

@furkatsultonov9976 3 ай бұрын

Hey buddy, is there any field in DL that you have not put your head in ?! lol. Whatever new thing I begin and look for some tutorials, your videos keep popping up. Wish you all the best.

@muhammadzubairbaloch3224 Жыл бұрын

when you are implement NeRF implementation in pytorch. Because this is time of research in NeRF

@vanshshah6418 Жыл бұрын

please it helps us a lot.. can you implement it from scratch

@robertodelprete1075 2 жыл бұрын

Hey Aladdin, think implementing RetinaNet from scratch !

@MikeSieko17 7 ай бұрын

how accurate are nerfs

@ChizkiyahuOhayon 3 ай бұрын

Can you talk about BEV and Occupancy Network?

@deeper_learning Жыл бұрын

Yeah, that would be great if you implement it in pytorch

@muhammadzubairbaloch3224 2 жыл бұрын

Kindly implement the NERF . thanks

@yannickpezeu3419 2 жыл бұрын

Let's say we put a camera at each angle of the room. We will soon be able to create a 3D reconstruction of a video... That's something !

@akagamishanks7991 4 ай бұрын

Hey Aladdin, could you please implement this from scratch?

@mrayushbajpai 2 жыл бұрын

A little thing, I'd like to request......... Please don't use the white background. It hurts the eye, battery, and pretty much all programmers use dark theme anyways.