C++ Weekly - Ep 374 - C++23's out_ptr and inout

No video

C++ Weekly - Ep 374 - C++23's out_ptr and inout_ptr

Рет қаралды 17,726

Күн бұрын

☟☟ Awesome T-Shirts! Sponsors! Books! ☟☟
Upcoming Workshop: C++ Best Practices, NDC TechTown, Sept 9-10, 2024
► ndctechtown.co...
Upcoming Workshop: Applied constexpr: The Power of Compile-Time Resources, C++ Under The Sea, October 10, 2024
► cppunderthesea...
Discussion: github.com/lef...
T-SHIRTS AVAILABLE!
► The best C++ T-Shirts anywhere! my-store-d16a2...
WANT MORE JASON?
► My Training Classes: emptycrate.com/...
► Follow me on twitter: / lefticus
SUPPORT THE CHANNEL
► Patreon: / lefticus
► Github Sponsors: github.com/spo...
► Paypal Donation: www.paypal.com...
GET INVOLVED
► Video Idea List: github.com/lef...
JASON'S BOOKS
► C++23 Best Practices
Leanpub Ebook: leanpub.com/cp...
► C++ Best Practices
Amazon Paperback: amzn.to/3wpAU3Z
Leanpub Ebook: leanpub.com/cp...
JASON'S PUZZLE BOOKS
► Object Lifetime Puzzlers Book 1
Amazon Paperback: amzn.to/3g6Ervj
Leanpub Ebook: leanpub.com/ob...
► Object Lifetime Puzzlers Book 2
Amazon Paperback: amzn.to/3whdUDU
Leanpub Ebook: leanpub.com/ob...
► Object Lifetime Puzzlers Book 3
Leanpub Ebook: leanpub.com/ob...
► Copy and Reference Puzzlers Book 1
Amazon Paperback: amzn.to/3g7ZVb9
Leanpub Ebook: leanpub.com/co...
► Copy and Reference Puzzlers Book 2
Amazon Paperback: amzn.to/3X1LOIx
Leanpub Ebook: leanpub.com/co...
► Copy and Reference Puzzlers Book 3
Leanpub Ebook: leanpub.com/co...
► OpCode Puzzlers Book 1
Amazon Paperback: amzn.to/3KCNJg6
Leanpub Ebook: leanpub.com/op...
RECOMMENDED BOOKS
► Bjarne Stroustrup's A Tour of C++ (now with C++20/23!): amzn.to/3X4Wypr
AWESOME PROJECTS
► The C++ Starter Project - Gets you started with Best Practices Quickly - github.com/cpp...
► C++ Best Practices Forkable Coding Standards - github.com/cpp...
O'Reilly VIDEOS
► Inheritance and Polymorphism in C++ - www.oreilly.co...
► Learning C++ Best Practices - www.oreilly.co...

Пікірлер: 96

@skeleton_craftGaming Жыл бұрын

Malloc returns NULL if it cannot allocate the memory. Therefore it is theoretically possible that the address pointed to by that pointer is a null pointer

@skeleton_craftGaming Жыл бұрын

Also why can you not delete memory that was malloced?

@Raspredval1337 Жыл бұрын

@@skeleton_craftGaming it calls a destructor; destructors only exitst in c++, not in c. Plus, smart pointers are meant to hold AN instance of an object, not a block of instances: that's why you sholdn't really use a smart pointer wrapper over a heap allocated buffer. In theory, the whole block should be deleted, but that smells like a potential UB since the smart pointer calls delete instead of delete[ ].

@KingKarEl100 Жыл бұрын

@@Raspredval1337 the deleter issue has been resolved and the correct deleter is used by the smart ptrs type since c++ 17 (possibly since c++14)

@N.... Жыл бұрын

@@KingKarEl100 This is not true. If you are not using `std::make_unique` or `std::make_shared` then you must absolutely take the time to implement the proper custom deleter for the memory you are wrapping in the smart pointer. It is not possible to autodetect what the correct thing to do is. Keep in mind a pointer given from a C API might not have been allocated from the heap at all, for example it may be from an internal stack array and must be returned by a corresponding release function that marks it as no longer in use.

@oracleoftroy Жыл бұрын

@@skeleton_craftGaming _unique_ptr_ handles NULL just fine. And _unique_ptr_ can also delete memory created by _malloc_ or any other allocator just fine as long as you tell it to use _free_ or whatever other release function is appropriate. However, it uses _delete_ by default, so you have to explicitly tell it to use something else.

@AlfredoCorrea Жыл бұрын

1) a well crafted C API will have a corresponding free function, e.g. call it`free_data`, to work with `get_data`, so you don’t have to guess how to free it. This might also incidentally prevent the use of `make_unique`. 2) probably the difference between `out` and `in_out` can be appreciated more when the pointer is `shared_ptr`

@florianwinter4892 11 ай бұрын

Inout can be justified by C APIs that replace or reallocate pointers and use a single in/out argument for that. Whether such an interface is good or bad and whether there should rather be separate in and out parameters is arguable, but C APIs like that do exist, and sometimes we have to use them.

@zanfur Жыл бұрын

Reset does not implicitly call release; it destructs any original object. Release doesn't destruct the object. That's why it's used for inout: it's assumed that the C API has taken ownership of the pointer.

@warzchoantark3327 Жыл бұрын

release() call releases the ownership, but doesn't delete the object. So, using inout_ptr in your code would lead to memory leak if you don't save raw pointer value from unique pointer and don't call delete explicitly somewhere later

@N.... Жыл бұрын

It is meant to be used when the C API takes the old pointer and frees it internally and then replaces it with the new pointer (or when it does not modify the pointer at all)

@ABaumstumpf Жыл бұрын

The description of out_ptr and inout_ptr are written as cryptic as if being unreadable was the goal.

@AgrossMX 8 ай бұрын

the point of these if that if you standardize their usage then it would be easier to reason about the lifetime of raw pointers from C (or anywhere else for that matter) from the perspective of the compiler and any tooling built over it.

@LemonChieff Жыл бұрын

```c int* result = malloc(sizeof (int)); *result = 42; // dereferencing NULL on malloc failure ``` Here is a fix: ```c int* result = malloc(sizeof (int)); if (result) *result = 42; ```

@bloodb0ne Жыл бұрын

Quite useful, but the crazy part is that you can set a breakpoint inside the decltype'd lambda :D never thought about that 👍

@jewulo Жыл бұрын

Lambdas are just classes with overloaded () so it should not be a surprise. I think?

@redcrafterlppa303 Жыл бұрын

I never thought that a block {} could exist between it's just to bizarre to the point that I see c++ as a dying ship adding every weird syntax just to stay relevant and becoming totally unusable and unreadable in the process. You really need a phd to fully understand even 1 line of the std lib. C++'s downfall is that it has too much obscure niche syntax and is adding more every 3 years and it's never removing syntax to stay backwards compatible.

@cppweekly Жыл бұрын

Understand the lambda and understand C++

@__hannibaal__ Жыл бұрын

When i swimming inside STD Oceans, I found thinGs never found in __official__ Documentation_____.

@florianwinter4892 11 ай бұрын

@@redcrafterlppa303It's been a while since I've seen a university from the inside, but my impression has always been that actual academics with actual PhDs hate C++. If this has changed in the last 2 decades, then maybe it is a good sign and not a bad sign for the future of the language.

@driedurchin Жыл бұрын

"Let's just hope that's doing the right thing" is the most C++ thing I've ever heard.

@sneakystabbalot6735 11 ай бұрын

Thanks!

@cppweekly 11 ай бұрын

Thanks!

@tonydelamancha5513 Жыл бұрын

microsoft has something similar in wil::out_param. but it has an issue were you can't use the pointer in the same direct scope, like inside of an 'if' statement. does this have the same issue?

@stephenhowe4107 Жыл бұрын

What Alfredo Correa said below. Suppose the C get_data function lives inside a DLL or shared library and is statically linked with the C library. Then the wrong free() is invoked in the lambda. The free() called is the statically linked version from the C++ main() and not the free() statically linked version from DLL or shared library. So Alfredo is right. There should be a free_data() and this should be in the same location as the get_data() in order for the correct free() to be invoked. uinique_ptr should not use a lambda.

@verylongtrain Жыл бұрын

The smart-pointer should use a lambda, which calls free_data() in that case.

@stephenhowe4107 Жыл бұрын

@@verylongtrain Good call

@lostphysicist Жыл бұрын

Sorry, Jason. This isn't the right approach. Not because your code doesn't compile or causes undefined behavior directly as is, but because of an implicit rule that you should never free an object outside the place that allocated it. This is actually a well-known rule in the plugins world. Meaning: the correct way to do this is have your C code provide the deallocation function, and then make your unique_ptr's destructor call that function. The reason for that is that C API usually crosses compiler boundaries, which means that a compiler that compiled the C part doesn't have to be compatible with the one running the C++ part except through the C API, which means that a compiler that adds metadata to malloc or new that doesn't violate the standard will lead to the other compiler doing undefined behavior. See for example how the memory size in new[] calls are determined on delete[]. It's compiler dependent. So allocating in one compiler and freeing in the other is generally undefined behavior.

@Zettymaster Жыл бұрын

to put this into more real terms: instead of doing a free() on the allocated object - the implementation of said free wont necessarily be the same as the one used for actually destroying the object. say, if it comes from a memory pool, now you'd accidentally free part of the memory pool, instead of just marking it as unused. so the "better" code would look something akin to : std::unique_ptr ptr: im mainly thinking of stuff like the Vulkan API in the back of my head, with say a VkDevice, which you need to free with vkDestroyDevice

@lostphysicist Жыл бұрын

@@Zettymaster Precisely. Many libraries with C APIs have such interfaces.

@michaelkovaliov8877 Жыл бұрын

let's say that you have a c function void allocate_device(int** dev); and a free function as you suggested void dellocate_device(int* dev); // or maybe 2 pointers? you may use the deallocate_device as your free function inside the lambda, thus achieving the same thing basically, no?

@j_ondrusek Жыл бұрын

Funny how you used the endl and then corrected it after a cut 😂

@cppweekly 11 ай бұрын

:) took 20 weeks for someone to notice that. Yeah, I was very confused during the recording of that video. It took a couple of reedits to get it together and at one point I had no idea why I wasn't getting output!

@Raspredval1337 Жыл бұрын

but isn't it a bad c practice to return a preallocated mem block instead of wanting the user to provide one? Seems like a smelly solution for an even smellier problem.

@platin2148 Жыл бұрын

I think you typically have functions being called with NULL first then it returns the size it needs and then you pass the right sized allocation.Or they allow you to give/get the size of the buffer directly

@claytorpedo Жыл бұрын

What is missing from this example is that most C APIs that allocate like this also provide a paired deallocation function, so there is no need to know internal details of how the alloc/dealloc works. I haven't looked into out_ptr/inout_ptr and I'm curious why they were chosen over unique_resource, which seems more explicit about what it's supposed to do.

@TsvetanDimitrov1976 Жыл бұрын

most of the directx functions allocating buffers, textures, etc. work like that

@tikabass Жыл бұрын

It's a very commonly practiced bad practice.

@platin2148 Жыл бұрын

To return internally allocated memory yeah. But COM and also DBUS sort of do it but indirectly..

@florianwinter4892 11 ай бұрын

Does this work for non-pointer types, such as file descriptors or other handles of type int or HANDLE (which is a pointer, but thats's an implementation detail...)? For example, I have written a generic handle class with a unique_ptr-like interface that can manage ownership of any kind of "handle" a C API might return, and I have used it for all sorts of handle types used by the Windows API, OpenGL, libCURL, etc., and I can see std::out_ptr as the solution for a lot of ugly code for obtaining the raw handles from C APIs. Smart COM pointers are another (Windows-specific) use case.

@florianwinter4892 11 ай бұрын

(Of course, I can just try it myself. This was rather a comment on the issue you brought up of "having to know what allocation function was used". I think this was not so much designed to be used with unique_ptr and malloc, but rather with custom smart pointer-like objects that also call C APIs to free resources. But of course, it does work with std::unique_ptr with custom deleters for pointer types)

@Ariccio123 10 ай бұрын

As a fun fact, you can annotate HANDLE parameters with SAL and have the static analyzer do all your checking because it is indeed just a void**

@Wizatek Жыл бұрын

All i hear is that you need to use these things if you're doing sketchy stuff

@peterbonnema8913 Жыл бұрын

If I read the docs right, the reset function also uses the deleted to delete the original piece of memory. So I would've expected two invocations of your custom deleter function.

@ivanp_personal Жыл бұрын

malloc can return NULL - that's what you've missed in C function, you have to check malloc result before you use it.

@phantom_stnd Жыл бұрын

Trying out a new c++ feature util you run into undefined behavior challenge

@_RMSG_ Жыл бұрын

It would be nice if we could interface with C safely by not interfacing with C 😅

@willwatts8103 Жыл бұрын

How is this palaver with out_ptr beneficial? Would it not be easier and clearer to declare a lambda that does the deallocation + transfer to the unique_ptr? I realise it is deliberately a piddly example, but the faff:meat ratio to use this seems unfavourable.

@N.... Жыл бұрын

It reduces the amount of code noise you have to contend with. Without std::out_ptr you currently have to write stuff like this which leaks a raw pointer into the enclosing scope: void* p{}; some_c_api_make(&p); std::unique_ptr ptr(p, some_c_api_unmake); Avoiding that variable leakage is nontrivial prior to C++23. With std::out_ptr we can avoid the extra variable: std::unique_ptr ptr(nullptr, some_c_api_unmake); some_c_api_make(std::out_ptr(ptr));

@dwarftoad Жыл бұрын

Once you create the unique_ptr and do whatever interaction with the C functions is needed, you can pass/store the unique_ptr the same as any other unique_ptr without having to remember when or how to deallocate the memory created in the C function, perhaps in a separate part of the code with no knowlege of the C API (no need to #include in for example). In the past we'd just have to include the C header and make sure to use the correct C API call in the destructor of any object storing it to deallocate the resources, plus checks if it is valid/set or ownership has been transferred to or from another object, and don't forget to null the pointer when neccesary, etc. etc. Or write C++ wrapper code that does this. Not a big deal but it is useful, if you are otherwise using unique_ptrs a lot.

@platin2148 Жыл бұрын

They are now going full crazy aren’t they? When relative pointers based on more horrifying templates?

@oracleoftroy Жыл бұрын

No, this is a great addition, and it makes working with C apis in a safe way a lot easier.

@dwarftoad Жыл бұрын

Nobody's forcing you to use this feature.

@Runoratsu Жыл бұрын

13:37 long. Nice.

@eudorian111 Жыл бұрын

But why would you not just call free() after calling get_data on a pointer?

@N.... Жыл бұрын

Because that's prone to mistakes and isn't exception-safe.

@eudorian111 Жыл бұрын

@@N.... But won't this example will ultimately call free as well? How does it provide more exception safety?

@oracleoftroy Жыл бұрын

@@eudorian111 // the C way SomeCStruct * p; auto error = allocate(&p); // use p; release(p); // Try to use the C api safely in C++ before C++23 SomeCStruct * temp_p; auto error = allocate(&temp_p); std::unique_ptr p(temp_p); // use p; // C++23 std::unique_ptr p; auto error = allocate(std::out_ptr(p)); // use p; Plus I have a utility that looks like: template struct deleter_from_fn { template constexpr void operator()(T *arg) const noexcept { fn(arg); } }; template using custom_unique_ptr = std::unique_ptr; This lets me write: custom_unique_ptr p; auto error = allocate(std::out_ptr(p)); // use p; That way it is really easy to add unique_ptrs to C objects and not have awkward layers trying to wrangle the two API's together.

@vorrnth8734 Жыл бұрын

get_data does not need to use malloc. It can use a different allocator.

@N.... Жыл бұрын

@@eudorian111 You wouldn't free/release the data until you were done with it. The process of using the data before then may result in an exception being thrown.

@lukasz2345 Жыл бұрын

One more reason to learn rust I guess :/

@User-cv4ee Жыл бұрын

Is pointer to pointer a common thing in C? Most function I have seen return a handle/pointer.

@X_Baron Жыл бұрын

I think that the idiom used here is that the C function is supposed to return an error code, and that's why it has to take a pointer to a pointer as an argument (to "return" a pointer also).

@N.... Жыл бұрын

Yes, it is very common, especially in graphics programming where the graphics API uses your allocator internally and then gives you back pointers from within the structures it has allocated.

@AlfredoCorrea Жыл бұрын

Yes, it is common. anything that creates a new object in which you want to abstract the heap and/or abstract the bunching of object building and object allocation together uses this idiom. a (niche) example I have handy is the whole of the MPI (Message Passing Interfcae) API.

@sirhenrystalwart8303 Жыл бұрын

You see a similar thing in c++ with a reference to a smart pointer. Both are providing two levels of indirection which is needed to do something like reseating the pointer.

@Ariccio123 10 ай бұрын

SAL has entire groups of annotations just for handling double pointers for exactly this reason!

@TsvetanDimitrov1976 Жыл бұрын

Using unique_ptr looks like a code smell to me, since the only reason to do it is to abuse the dtor with a custom deleter. It's probably just a biased opinion, but to me it's a lot clearer to read something like int* p = nullptr; ctor_func(&p);..... dtor_func(&p). I might wrap the whole thing into a separate class that calls the c api functions in the ctor and dtor, but that's about it.

@N.... Жыл бұрын

std::unique_ptr automates the process of "I might wrap the whole thing into a separate class that calls the c api functions in the ctor and dtor" and is much less prone to mistakes, all you have to do is provide a proper custom deleter and everything is handled for you with a consistent API and exception safety.

@TsvetanDimitrov1976 Жыл бұрын

@@N.... It automates calling the custom deleter yes, but doesn't automate the construction. Also, those c api functions are usually a whole family not just create/delete, so it makes perfect sense(to me) to wrap them in a class.

@N.... Жыл бұрын

@@TsvetanDimitrov1976 Fair, but even with a class it may still make sense to use `std::unique_ptr` as the class member.

@dwarftoad Жыл бұрын

unique_ptr sometimes lets you avoid having to write a destructor or maybe any constructors at all, which can be nice.

@TsvetanDimitrov1976 Жыл бұрын

@@dwarftoad sure, but let's assume i want to create a dx12 texture. I'll need a descriptor describing it which i probably want to keep as invariant, also need to create the resource views this texture will be used with, and probably some additional things. I surely can keep all of this as separate objects, but they are intrinsically coupled with the texture i'm creating. Why should I keep them separate instead of creating a class that keeps all those things and their lifetimes coupled with the texture itself instead of writing a wrapper, with a proper ctor and dtor? Keep in mind that the wholoe "c api" boundary is most of the time caused by c++ breaking abi all the time, it's c++ internally in the library.

@KingKarEl100 Жыл бұрын

This feels like an unnecessary abstraction offered by the language. Since C++ 11, smart ptrs already offer you the ability to safely manage the lifetime of ptrs from a C library. The extra ptr types add more complexity to a problem thats already solved

@WenirR Жыл бұрын

It's not an additional pointer, it's a helper /thing/ to reduce boilerplate

@KingKarEl100 Жыл бұрын

@@WenirR actually they did introduce additional PTR wrapper types for this. The video shows one of them called "out_ptr_t". I understand it's used to reduce boiler plate but I don't think this is beneficial to the language. This same thing can be done by declaring a raw PTR, passing it to the c function then giving the raw ptr to the smart PTR with a custom deleter to manage the lifetime. We're doing almost the exact same simple steps. Maybe it's the example used in the video but this feels like a pointless addition

@WenirR Жыл бұрын

@@KingKarEl100 that's why I wrote /thing/ - I don't really care how they achieve it

@sergeiborodin9254 Жыл бұрын

That looks like a horrible crutch that will makes code even less comprehensible. Can't see any reason to implement such thing in a first place. C functions have so much nuances so almost every time you have explicitly create some kind of wrapper around C functions with memory allocations for C++ . Explicit custom wrapper class is 100 times more readable and usable. You can add things like validity checks, mutex sync and errno wrappers for try-catch. I already imagine this thing with Linux API. My rule of thumb: if you have to deal with raw pointers and C-allocations in C++ - you have to deal with allocation errors, store boundaries and write converters to C++ types.

@gregdee9085 Жыл бұрын

Ppl adding crap to C++ now because they're looking for something todo... C++ was mature ages ago, no amount of new features will help a programmer with bad judgement....

@MarekKnapek Жыл бұрын

Now the joy of `get_data` using `malloc` from one instance of CRT and your app using `free` from another version of CRT. Haha, Fun Will Now Commence. Let me explain: If you are interacting with C-style API, it is probably implemented inside other DLL. Windows has no concept of libc, C run-time library (as it is known on Windows or CRT for short) is just another library, just like any other library, Windows has no special treatment for libc. So, programs could either statically link against CRT (each EXE and each DLL could link against different version). Or dynamically link against CRT (it is also known as VCRedist) also each EXE and each DLL against different version. Or each application could distribute its own local copy of dynamically linkable CRT. It is a mess. Windows' dynamic linker also works a little bit differently than Linux's. So, to be at the safe side you need to: Either call `get_data` and `free_data` using the same DLL, do not `get_data` using DLL and `free` in your EXE. This works in all cases (static CRT, dynamic CRT, different versions of dynamic CRT). Or, if you are author of both EXE and DLL, set both projects to use dynamic CRT. Then you are fine to `malloc` in one and `free` in other, as both modules are sharing single CRT state (think of set of global variables) in separate CRT DLL. Also it is a good idea in general to link against dynamic CRT as it could be updated by Windows Update independently of you, fixing bugs and improving correctness and performance for free.

@vorrnth8734 Жыл бұрын

Usually the get_data also comes paired with a free_data. Use that like you are supposed to.

@dwarftoad Жыл бұрын

If the library provides a free_data() function, then you can just give that to unique_ptr e.g. std::unique_ptr p(nullptr, &free_data); (Or call it from the lambda instead of free() if you want to do it that way.)

@MarekKnapek Жыл бұрын

@@dwarftoad Yes, that was my point. If you use `get_data` from a library, you should also use `free_data` from the same library. Unless documented otherwise.

@MarekKnapek Жыл бұрын

@@vorrnth8734 Yes, that was my point. If you use `get_data` from a library, you should also use `free_data` from the same library. Unless documented otherwise.