Many Rust content creators are making introductory content or covering application topics. I enjoy that, but at my current stage on my Rust journey I also really like this kind of well explained more advanced technical content. Thank you for sharing your experience and understanding with us. You are unique, effective and appreciated.

Man, I went into this having no opinion on the matter and came out convinced that you'd have to be crazy to use &Option

This is the Rust content we need!

Brilliant mid-level content, perfect for folks with some experience and looking to level up. I feel like I just leveled up.

Another big reason related to the Box issue why to use Option is if you use an alternate type internally like Result, or even a raw pointer (for some reason). It's trivial within a function to construct an owned Option that has a &T, but it isn't trivial to return a &Option from a function.

Option is pretty much just a pointer - the pointer arithmetics + a bunch of convenience functions. Even outside of rust it is the prefered type for nullable references (unless you want to use raw pointers which usually you shouldn't)

I think the following analogy is instructive. If you think of `Option` as a container, it makes sense to compare: `&Option` vs `Option`, with: `&Vec` vs `&[T]`. I think the analogy works quite well. `&[T]` is all you'd ever need to immutably access the contained items. If you had `&Vec` you couldn't really call `into_iter()` on it since you don't want to move out of the underlying `Vec`. You can't really do anything useful with it that you couldn't do with a `&[T]`. In the mutable case `&mut Vec` vs `&mut [T]` the distinctions are also similar to `&mut Option` vs `Option`. A `&mut Vec` would allow you to change the underlying `Vec` by for example: `push`, `pop`, changing capacity or even replacing it with an entirely new Vec. That seems like something you would almost never want to do. `&mut [T]` on the other hand makes much more sense in the majority of cases, since that will allow you to mutate any number of the items in place, but would never mess with the integrity of the `Vec`. That would almost always be what you'd want. The considerations about changing the underlying storage seem to match logically as well. I think the analogy mostly checks out. If anyone spots somewhere it fails, please let me know =).

Another reason: You're implementing a trait method that returns &Option but you don't actually have an Option in your implementation and just want to return None. You can't just return None; you now need a redundant empty Option in your type to return a reference to with the correct lifetime. Had it been an Option return type you could return None no problem.

The only case I return &mut Option is in traits when I want to directly expose the underlying storage for it to be modified. In this case, I can't just make it public (as said at 3:15) because it's a trait. I never encountered another use case.

The explanations in your videos are honestly the best i've found on youtube. I wish I could find more content just like this.

I really like the constant all black background. Perfect for bed time watching! Also this content is extremely useful to me as a beginner rustacean! Great explanations keep it up!

It's nice to hear opinions of experienced rust programmers. It really helps a lot to understand more about rust and have a better foundation when programming I'd really like in depth explanations on more advanced stuff too like niche optimizations and static code analysis

I'm the developer of Hurl, an Open Source cli tool in Rust: I've one year of Rust experience, and still on my learning journey. Your video are crystal clear and invaluable, on essential concepts. Please keep up the good work !

Every time I watch a video of yours I feel there's still so much left to learn about Rust. The quality of your videos and the explanations you're giving really help others like me understand the nuances of Rust. Thanks for the video. Keep up the good the good work!

Another thing is that using the ? operator on &Option only works if T is Copy. You need to do .as_ref() to turn it into Option anyway if you want to use it in that case 🙂

Please keep making these. They're wonderful for an intermediate Rust dev.

This is my new fav channel

Again, a good insight gained from your videos. Had the realization after watching that this also applies to (&K, &V) vs &(K, V) in many situations, for the same reason of not exposing how you store the data. To return a ref of tuple, you are saying "I have a tuple which I can lend to you" whereas a tuple of refs can be freely constructed from any data source that allows you to get refs to both sides of the tuple i.e. "I have a K and V somewhere, and I'll lend them both to you as a pair."

Your videos are extremely high quality! Please make more! I'd happily to pay for an in depth course.

I'm a beginner Rustacean, with tons of background in other languages. These videos are very helpful.

Thanx, great stuff! Real world problem for a Rust beginner like me - not just repeating stuff from the Rust book. Looking forward to more!

13:08 Rust actually *guarantees* memory layout of Option to be optimized. Check option documentation, paragraph about representation

What a perfect explanation, thank you so much! Yesterday just found out that sizes of Option and &T are equal and thought that it's some fancy compiler optimization with null pointers. And you confirmed it

This is the first KZfaq video I'd be watching on coding and I'm so glad to come across it. So expository! Thanks for sharing. You're amazing.

We need more videos like this. I’d suggest something related to parsing Real life crazy JSON documents with a few possible different types for the same fields

Been thinking this for years, glad someone finally said it

First Arc, now Option, what else is there to come? I want a trilogy!

Been doing rust full time for 2 years. This is a *great* video, keep the amazing work

Wow, great follow up to your previous Rust video! Loving the super focused in-depth style you have

The all-zeroes Option None is the closest thing I've seen to a NULL so far in Rust. I'm a pretty seasoned developer, but still dipping my toes in the crab-infested waters. My initial instinct was to say "of course you just want a real Option" but I had no idea there were so many reasons to reinforce my gut.

Excellent deep dive into option. It answered questions I had long time ago about designing API when Option is needed. Huge thank you. Please make more.

I'm quite new to Rust and really enjoying how you explain (to me) more advanced topics - it's enticing me to spend some real time with the language. Thank you - and sub earned!

15:12 “Option of a reference has no space overhead over a regular reference. It’s essentially just a reference that can also be null.” Wow! Really cool that Rust has safe and ergonomic null pointers without any overhead.

Finally, some advanced Rust advice! Thank you!

This was so helpful! I've banged my head against the issues you discussed many times but never took a step back to think about _why_ before. I stopped to think when you said to at 1:48 and I figured out the basic issues around intent and ownership. You also explained some things I would not have thought of (especially the niche optimizations).

Your visualisations are really clean!

I love those topics. It improves my intuition about design choices!

I just discovered your channel today, great stuff by far. Subscribed!

this reminds me of scott meyer's books, but for rust and in video form!! very well thought out and more attentive than most articles

This is one of the best explanations I saw about Rust, keep up with great videos like this, your way of speaking is amazing

I appreciate having the answer in the thumbnail.

Wow, that memory layout thing kinda reminds me of of NULL pointers in C execpt abstracted away such that you don't have to thing about null pointers! great!

Thank you, that was a very clear explanation! As a relatively new rustacean, even though I understood the difference, I hadn't trained my intuition about which way is superior.

Thx for this video! I already knew about the Null Pointer Optimization stuff but I'll now know which to use!

Another great Rust video!! Thanks for making this!

This video was something I definitely needed to see. Thank you for making my code better. I really appreciate it.

Im not ur audience but just for fun, i never touched a single like of Rust. Still i understood almost everything. Great video, very well explained

Great content. Extremely useful overview. Hope you will keep on releasing such great videos. Thank you

The Rust community has flocked to your channel, and now you are doomed(?) to produce only Rust content for the rest of your life, lest your followers ask "WHY U NO RUST!?" in every non-Rust video!!!!! j/k But seriously, you have a knack for explaining Rust concepts. This channel is going places. Thanks for the great video!

This video was amazing and incredibly informative

I'm instantly subscribed. Thank you for this video !

loving you rust videos. instantly subscribed

i love your channel, keep up with those type of videos, we need more of them :)

this video is useful and valuable! I enjoyed learning about this and will employ it personally from now on

Very cool detailed explanation. And also, I tend to agree with this pattern.

Two and a half minutes in, and you got yourself a new subscriber

Never seen a programming video like this one. Super good 😮

This is good and honest content, keep it up!

God I love your videos. Please let me support your content with a patreon or through KZfaq so it continues to flow.

That’s hot! Well done, it’s really nice to see beautiful explanations and animations to make information structured, reliable and easy, thanks!

U r the guy, whose gonna help me through rust

Love it! Thanks! I need more of this!

Really good video as always, thanks for your detailed explanations, I'm looking forward to the next video :)

There was a pretty good opportunity for a Sum 41 reference in this video

Thanks for the video!

Glad to find someone who likes null pointer optimisation as cool as I do x)

Absolutely golden. Loved every second of it! I knew Rust was blazingly fast, but the niche optimization part blew me away

clearly the answer is &Box

Me, an assembly programmer: "I like your funny words magic man"

Great video, it was interesting to see how rust stores option reference None too.

Great video! I will definitely be putting this to use in the workplace.

Option memory representation not just simply "niche" optimization, but also a bridge into void* for FFI. None is actually null pointer and Some pointer to actual data when passed into languages like C/CPP.

great rust content man.

You videos are amazing really They are very depth and help a lot in understanding Thank you for all your videos

Excellent explanation :)

another banger video 🔥

about the last point you made on "being honest about what function needs" kinda contradicts an idea of "abstract in, concrete out" - making function parameter generic instead would result in more flexible API anyway, thanks a lot for this content. I'd have never figured this out myself!

There's some other advantages to Option which I don't think you covered. 1. It should be slightly more performant to check whether the Option is None, because there isn't a reference that has to be looked up. 2. (This one isn't relevant to discussions of API design because it's about how the data is stored and not how it is exposed) If you do end up storing your data in Option, that should use only 8 bytes of storage to store the pointer, or null. But Box will be 8 bytes to store the pointer to the Option, however many bytes to store the option state (None / Some) plus padding, and the size of T, which has to be reserved so that the option can still be turned into a valid Some. That can be WAY more memory used. Especially if you have a lot of Nones.

really well explained. i'm gonna go refactor some stuff

thank you for this great video💯

Truely master piece of an educational video!! 😊 You probably come from the C++ side, actually that reasoning directly applies to std::optional as well.

I hacked together a tiny game and ended up using Option in a place where it's probably not intended and as a result the "correct" reference type in some places of the code ended up being &Option. It was a tiny tile based game, a clone of 2048, in the terminal, using a fixed size array of tiles as a game model and just using stdin/stdout for input. Instead of going for something like Box with a Tile trait, I just made a Tile enum with all the possible tile states. (For such a simple game one could easily just have gone with some integer type, but enums allow me to make bad states unrepresentable so I went with that.) It ended up being the case that all tiles except the case of an empty tile shared some common simple behaviour like being able to be incremented to the next higher state. Also in the game most tiles can slide until they hit other tiles. Empty tiles shouldn't really slide along and they don't count as hits. So again the empty tile is a bit special. Given that I'm already in enum land, the design to go for would probably be to create a two state wrapper enum like `Tile{Empty,Occupied(SpecificTile)}`. And then the SpecificTile enum would containing the details of the specific non empty tile. The benefits of that is that code becomes super easy to read and I can define the semantics however I want. This is definitely a legitimate option. But, since this was just a hacky project to get up and running that I don't intend to ever touch again, I decided to just use an option instead. This is not as semantically clear. In fact, it can end up being quite semantically confusing. Later down the line in a serious project I might be in a state where I maybe have an empty tile and maybe not. That would be represented with Some(None) and None respectively, whose meaning isn't exactly self-evident. I'd advice against using this for anything half serious. There are a few upsides though when iterating quickly. You get a bunch of convenience methods for free. You can use stuff like flatten to ignore all the Nones, and you can use `if let` constructs and more, without writing any boilerplate. It lets you do some things in a quickly and dirty way. Don't do it ^^. But alright. Given that I went with this *bad* design choice, the corollary was that &Option actually is the type that should be used in certain places, since, this particular Option actually was part of the data model and I would sometimes want to mess with the Option itself. My overall conclusion after thinking about it is that it's just a bad decision. The semantics become confusing. I didn't shoot myself in the foot or anything and it does technically work. But if I want to make it useful I'll probably have to end up rewriting that bit eventually. Even when you've been programming in rust for a while, sometimes it can be a bit tricky and take some work to remember how to reproduce the behaviour you want of some of the standard library types. You have to implement a bunch of traits. Understanding what to do and how to do it is a lot trickier than using the existing types. But semantically, it's probably better to give up some of the convenience for clarity. There is some benefit to use a standard library type to quickly iterate and that's exactly the convenience. But it's good to make types and states have a clear unambiguous meaning and that is what the type system allow you to do if you use it right.

Thanks, this is great!!

If I need ownership, I like to use `impl Into`. That way, the caller can simply `call(42)` or `call(None)`.

even if you dont think you need to offer a mut accessor, making the field pub is often still the best option. the way i look at it, its not about whether you expect people to modify the field, its about whether modifying that field will violate some invariant. in many cases, your struct will still make perfect sense if someone decides to replace that field without going through your code, so you may as well let them and save yourself some pointless accessor methods

14:20 mind blown.

Please correct me if I'm wrong. What I learn from the video is that we always want to take the reference to a Data if we can. So the point of this video is not about &Option and Option. but in fact about &T and T so writing something like &Option should be okay because it can coerce into Option

Awesome content.

I saw the thumbnail and the reasons seem immediately clear, I had just never thought about it. I have not the need to sit through this entire video, but i'll at least comment on it.

Wow easily earned the sub well done .

Hahah, I literally paused the video after the first second and thought "That first variant looks completely useless and misguided". Then I hit play and you say exactly my thought. :D

Nice video... I 99.9% agree with you and that .1% is filled with "A long enough timeline will provide at minimum one counterexample to any rule".

10:05 - I'm pretty sure the only "reasonable" way to do that is to construct a new Option and leak it (e.g. with Box::leak(), called on a second box enclosing the whole mess). Which is obviously terrible and bad and wrong, but it is at least "safe."

Thank you

Thank you.

I kinda just always used `Option` without thinking about it most of the time; I always just assumed that if you can get away with returning an owned type for something you always should

You explanation is absolutely clear and you can make a video shorter definitely. Thanks for Rust content, btw.

video felt a lot shorter than it was, great stuff!

Before watching, my take in one sentence is: One converts easily into the other. &Option converts into Option, so the latter should be used in function signatures so that they're less restrictive. This is the same idea as why &[T] should be preferred over &Vec, or &str over &String. Just as with Vec and String, and for essentially the same reasons again, the argument does not transfer to mutable references. &mut Option can be useful, just as &mut Vec and &mut String can be. These can be used to insert something into the Option, or push something into the Vec/String, respectively, whereas Option, and similarly &mut [T] and &mut str, don't allow the same. This observation is also closely related to the fact that &T is covariant in T, but &mut T is invariant in T.

Another point wrt parameters would be that you can bind a generic type to Into and allow the caller to pass a &T directly without having to wrap it!