The Heap: what does malloc() do? - bin 0x14

  Рет қаралды 145,282

LiveOverflow

LiveOverflow

Күн бұрын

Introducing the heap by looking at what malloc() does.
=[ ❤️ Support ]=
→ per Video: / liveoverflow
→ per Month: / @liveoverflow
=[ 🐕 Social ]=
→ Twitter: / liveoverflow
→ Website: liveoverflow.com/
→ Subreddit: / liveoverflow
→ Facebook: / liveoverflow
=[ 📄 P.S. ]=
All links with "*" are affiliate links.
LiveOverflow / Security Flag GmbH is part of the Amazon Affiliate Partner Programm.

Пікірлер: 84
@cyancoyote7366
@cyancoyote7366 7 жыл бұрын
I lost it at the "This is real fucking magic" part xD
@wuchta7812
@wuchta7812 7 жыл бұрын
me too thanks
@BitBerlin
@BitBerlin 6 жыл бұрын
"This is all real fucking magic!" - LiveOverflow 0:54
@billybabcokcs8224
@billybabcokcs8224 6 жыл бұрын
Best educational channel on youtube, thanks a lot dude
@ashikombat
@ashikombat 7 жыл бұрын
Thank you. I am happy that this is not web app related :p
@LiveOverflow
@LiveOverflow 7 жыл бұрын
+ashikombat some more vids of this series are coming :)
@cyancoyote7366
@cyancoyote7366 7 жыл бұрын
awesome
@stratan9707
@stratan9707 7 жыл бұрын
I f-ing love these vids. Can't wait for more!
@saspect2462
@saspect2462 3 жыл бұрын
Γειά σου ρε stratan master των pwners !
@Occcc12
@Occcc12 7 жыл бұрын
Thanks a lot. Much appreciated stuff :)
@DeRobyJ
@DeRobyJ 4 жыл бұрын
Much needed video, thanks!
@DavidThomsenPhD
@DavidThomsenPhD 5 жыл бұрын
My favorite part, "this is all real fuck magic!"" Hilarious!
@NghiaNguyen-bg9bz
@NghiaNguyen-bg9bz 7 жыл бұрын
Thank you ! Have a nice day :)
@manuelavendano3891
@manuelavendano3891 3 жыл бұрын
Damn! You really know your shit dude; guess from now on, I'm taking more seriously your videos than a lot of my school lectures for my educational career
@smrtfasizmu6161
@smrtfasizmu6161 2 жыл бұрын
If you are OK with the string being unmodifiable, you don't have to allocate new memory for the char* in the struct. You could have just assigned a string literal to the char* and that string literal would be placed in the executable file, after the section which contains the CPU instructions. However, this memory is read only, the program is not allowed to modify itself (at least not this part of itself). Which means that there is no need to allocate additional memory for the char* in the struct if you want that pointer to point to a read only string. If you want the char* to point to a string which is modifiable (a string which is placed on the heap in this instance), then you need to allocate additional memory in which you will place the chars to which char* will point to.
@Simrasil_
@Simrasil_ 7 жыл бұрын
I had to dive pretty deep into this stuff when I wrote a VOL plugin for the HDF5 file format last semester at uni. Still really interesting to see your explanation video on it. it's comprehensive as always :D
@LiveOverflow
@LiveOverflow 7 жыл бұрын
+SonKomischerTyp cool! What is VOL?
@Simrasil_
@Simrasil_ 7 жыл бұрын
LiveOverflow It's the Virtual Object Layer which catches the API calls from HDF and forwards them to the actual objects in memory. I wrote my plugin to optimize execution on a cluster so it used shared memory. Quite a cool project to work on :)
@LiveOverflow
@LiveOverflow 7 жыл бұрын
+SonKomischerTyp ah awesome. Sounds like a cool project. So you also had to deal with concurrent access and make it safe?
@robl39
@robl39 3 жыл бұрын
This channel is amazing!
@BurningApple
@BurningApple 6 жыл бұрын
CGP Grey easter egg? Love it :D
@ImGuti
@ImGuti 7 жыл бұрын
Love the videos, thanks for taking the time to teach me all of this !
@allwhatyouwant
@allwhatyouwant 6 жыл бұрын
I like your old videos as well as your newer ones! Also, is it right that I think from your accent that you're german?
@BryceChudomelka
@BryceChudomelka 3 жыл бұрын
great channel
@nekoneko9822
@nekoneko9822 7 жыл бұрын
sehr gut !
@dyslexicunt
@dyslexicunt 7 жыл бұрын
Might be worth pointing out that you can perform pointer arithmetic in this case on the struct as both items happen to be 4 byte aligned. You can't rely on this, however, if the struct contained members which are not aligned, unless the struct is packed. It might seriously confuse people in the future if they try to use the same techniques on an unpacked struct! I know _you_ know this, but it might be worth mentioning for people who are learning this in a later video. Your videos are exceptionally well presented. Keep it up :)
@LiveOverflow
@LiveOverflow 7 жыл бұрын
+dyslexicunt yeah thanks for pointing that out! Actually didn't think about that during writing the script :D I would hope nobody would rely on something I say this blindly. Instead I hope I could make the heap a less scary beast for exploration :)
@dyslexicunt
@dyslexicunt 7 жыл бұрын
That's ok, just thought it might help someone in the distant future who's screaming at gdb wondering why things aren't where they thought they are! (I spend a fair amount of time doing that myself) Looking forward to the next vid :)
@Borgimanio
@Borgimanio 7 жыл бұрын
I don't understand what you mean by packed and unpacked structs, can you maybe give some examples/hints?
@dyslexicunt
@dyslexicunt 7 жыл бұрын
+Borgimanio Sure :) Say i delcare a struct like this on an x86-32 bit platform: struct foo { u32 age; // 4 bytes u8 first_name; // 1 byte u32 *ptr // 4 bytes }; Looking at this, you would expect the struct to be 4+4+1 (9) bytes big and the elements contiguous in memory. You'd be wrong, however. The compiler would, in this case, silently perform this on your behalf: struct foo { u32 age; // 4 bytes u8 first_name; // 1 byte *+ 3 bytes of padding* u32 *ptr // 4 bytes }; Why does this happen? Because unaligned memory accesses are _bad_ for performance. The compiler will help you and pad out elements in your structure like this to ensure the next access (*ptr in the above case) will fall on a natural address boundary; a natural address boundary on 32 bit simply being an address evenly divisible by four. However there are some occasions where we dont want the compiler to 'help' like this, and we want it to leave our data packed exactly as it we've delcared it. This, is a packed struct. Nore simply put it's a struct which you have explicitly told the compiler you do NOT want to pad out. You do this with a compiler directive which, in gcc for example,looks like this : struct __attribute__((__packed__)) bar { u32 age; // 4 bytes u8 first_name; // 1 byte u32 *ptr // 4 bytes }; This explicitly informs the compiler not to meddle with the alignment and to keep the elements exactly contiguous in memory. Let it be said that you generally want to avoid doing this for the performance reasons given.
@Borgimanio
@Borgimanio 7 жыл бұрын
+dyslexicunt Thank you very much for your explanation! What could be possible reasons that someone doesn't want it to be unpacked? Perhaps if one has many structs and wants to save memory and doesn't care about speed?
@dedkeny
@dedkeny 3 жыл бұрын
He said "This shit is truly magic..."
@LemonChieff
@LemonChieff 6 жыл бұрын
I use malloc a lot. I had a basic idea of how it worked so this both clarified somethings and makes a lot of sense
@ashwinkafle1771
@ashwinkafle1771 5 жыл бұрын
I didn't expect you to be using ad block. Great tutorial BTW.
@buhaytza2005
@buhaytza2005 3 жыл бұрын
2:24 :):):) best explanation ever ;)
@riasgrimori6505
@riasgrimori6505 Жыл бұрын
The website of expoit exercises is not working 😢
@coolwinder
@coolwinder 7 жыл бұрын
Can you pls put link to playlist in your video. Great job btw
@LiveOverflow
@LiveOverflow 7 жыл бұрын
+Бојан Драшко ah yeah makes sense. I will do it in a couple of days
@Indic4Zone
@Indic4Zone 3 жыл бұрын
is this house of lore in phrack issue 67?
@thecrazzxz3383
@thecrazzxz3383 2 жыл бұрын
3:33 So the dword that stores the size of the chunk is counted in the size ?
@thecrazzxz3383
@thecrazzxz3383 Жыл бұрын
yes
@typedeaf
@typedeaf 5 жыл бұрын
I cant figure out why the stick figure waves his arm!
@Lunarixus
@Lunarixus 6 жыл бұрын
You should make a follow-up for this and explain what kalloc() does on iOS, good video anyway
@casperes0912
@casperes0912 5 жыл бұрын
Yes please
@radhekrishnapatel2843
@radhekrishnapatel2843 6 жыл бұрын
Which IDE are you using?
@lksw42439
@lksw42439 6 жыл бұрын
The dark looking one is Sublime
@silasalberti3524
@silasalberti3524 7 жыл бұрын
Isn't dlmalloc wasting incredibly much space? I mean: On your illustration it took up half of the space just for logistics
@sontapaa11jokulainen94
@sontapaa11jokulainen94 4 жыл бұрын
In this program yes but in larger programs the mallocs would allocate a lot more of space (for example malloc(1000)) so the 8 bytes (blank space and size) aren't that big of a deal.
@AntoshaPushkin
@AntoshaPushkin 6 жыл бұрын
God, it would have been so nice if your channel had existed 5-6 years ago when things like this one looked like magic to me
@redgek
@redgek 7 жыл бұрын
Nice &CGPGrey
@LiveOverflow
@LiveOverflow 7 жыл бұрын
+Dima Kalchenko ;)
@billybabcokcs8224
@billybabcokcs8224 6 жыл бұрын
I thougt malloc returnedt void though
@mehoneybadger999
@mehoneybadger999 4 жыл бұрын
cant see text editor text proprly ,they are all dark mode
@smrtfasizmu6161
@smrtfasizmu6161 2 жыл бұрын
If you allocated the memory of the size of the entire struct then why did you allocate memory again for char* name? 8 bytes for char pointer (or 4 bytes on a 32bit machine) have already been allocated.
@shobhitdogra7924
@shobhitdogra7924 2 жыл бұрын
Pointer is a variable like other variables but what makes it different is that it points to a memory location and that memory locations contetnt can be read by dereferencing the pointer. So if it was a char arr[some size]
@smrtfasizmu6161
@smrtfasizmu6161 2 жыл бұрын
@@shobhitdogra7924 On my PC I can write f2->name = "Hello World"; and it compiles and runs correctly. I can also write char* text = "Hello World"; and it compiles and runs correctly. I am using gcc compiler. I see what you mean though, now I am a little bit confused as to why it works, because if I try to do the same thing with ints or any other type, it doesn't work, I get a segmentation fault. For instance, if I write int* arr = {1, 2, 3}; printf("%d ", arr[1]); This will not work, and it should not work, arr is a pointer, it is not an array. I can use that pointer to manipulate an array or read from an array but the pointer itself is not an array. It is just an 8 byte number which is a memory address of some other variable, or struct, or of another pointer. If I write the previously mentioned code, compiler assumes that I wanted ar to point to the memory address of 1, because 1 was the first number I wrote. However, I can write char* text = "Hello World"; and it will compile and run. I don't understand why I am not required to write char text[12] = "Hello World"; char* ptr = text; If I wanted a pointer to Hello World. In other words, what you are saying makes sense to me but that's not how my compiler works.
@anon_y_mousse
@anon_y_mousse 2 жыл бұрын
I don't know if you ever figured this out on your own, but in C a string between two quotes gets placed in memory somewhere, depending on platform and compilation options will tell you where, and you're merely assigning a pointer to this space that was allocated for you. So char *s = "Hello, World!"; will already have the space allocated for the string. In most cases you can't modify that space, and that is indeed standard behavior, but you can do char s[] = "Hello, World!"; and make it modifiable, but at a set length.
@smrtfasizmu6161
@smrtfasizmu6161 2 жыл бұрын
@@anon_y_mousse By now I know this. When posting previous comments I probably thought you have to write char string[20]= "this is a string"; And then you could write char *ptrToString = string; Or something to that effect. I was confused why writing a string can work in any other way than this way. char string[20] part takes 20 bytes on the stack in which you can put characters and the null character in the end, then char* ptrToString can point to the place in the stack where this array of char begins. So, writing char* s = "some text" ; did not make sense to me. In my mind, it would be logical that you first have to write char string[10] = "some text" ; and only then char* ptrToString = string; This is how int, double, float etc. work. You can't write int* arrayPtr = {0, 1, 2, 3, 4}; You have to write int array[5] = {0, 1, 2, 3, 4}; int* arrayPtr = array; I know by know that you are allowed to do this with char* And the example from the video makes sense now. When you are just writing char* string = "This is a string"; in the main function, compiler can put the chars and null in the end on the stack, no problem. If you make a strut with char* on the stack, compiler can again put those string bytes in the current stack frame (which means these bytes are lost after you exit the function). Still, no problem. However, when you are putting a struct on the heap then the compiler doesn't know where to put this array of bytes which represents a string. That's why in this case when you are allocating a struct on the heap you have to provide a place in memory where to put the bytes of the string. You have to either malloc that memory or just set the char pointer of the struct to be pointing to some place in memory where you have already stored the bytes of the string.
@smrtfasizmu6161
@smrtfasizmu6161 2 жыл бұрын
@@anon_y_mousse But I still don't know the difference in modifiability when it comes to char* str = "some text; And char[] str ="some text"; I haven't done much C in my life so I never noticed that the first one isn't modifiable. If it really isn't modifiable (I have no reason to assume you are lying) then why is that?
@selinaisgreat
@selinaisgreat 4 жыл бұрын
Why is 0x10 used for 8 bytes? 0x10 is 16?
@sontapaa11jokulainen94
@sontapaa11jokulainen94 4 жыл бұрын
Because the whole row is 16 bytes (including the blank space at the start and the size itself).
@goustune
@goustune 7 жыл бұрын
I don't get it. I do a malloc(16) but if I look on my heap I see a 0x21 (so 33). Shoudn't it be 0x20 ? ... Ok nevermind, I didn't watch until the end huhu
@LiveOverflow
@LiveOverflow 7 жыл бұрын
0x21 has the last bit set (as opposed to 0x20). That is the `prev in use` bit, which means that the block before it is in use (is allocated). Which means if you ignore that metadata bit, the size is still 0x20. I have kinda ignored that for this video, to keep it simple. Just keep watching the Heap videos, it will be explained :)
@deepncsu
@deepncsu 5 жыл бұрын
@@LiveOverflow only vid on here which actually explains the internals on malloc. Great job! A noob question. Why is the block size double of the allocated bytes?
@deepncsu
@deepncsu 5 жыл бұрын
Shouldn't it be allocated-size+8 bytes (2 words)?
@zombiedude347
@zombiedude347 4 жыл бұрын
Did you pronounce the abbreviation for character the same as "char" (to burn)? Abbreviations are normally pronounced the same as the unabbreviated syllables aside from a vowel shift from change in stress.
@anon_y_mousse
@anon_y_mousse 2 жыл бұрын
Finally! This has always bothered me when people say char, like to burn, instead of care. It's an abbreviation of character.
@Sahuagin
@Sahuagin 7 ай бұрын
from my understanding your use of "the heap" here is not correct. malloc() calls do not give you memory from the heap, there is no heap (in C). as you explained, malloc() asks the operating system for a chunk of arbitrary memory, which you then get a pointer to, but that memory does not come from "the heap", it comes from wherever the operating system decides it comes from. "the heap" is from C++ where the compiler does set up an actual heap for you, effectively doing a single malloc()-like request for one single large chunk of memory at the start of the program. it then manages this chunk of memory for you, and when you request a `new` object, the memory allocation is taken from the heap (the previously allocated chunk of memory). and there's a whole big family of algorithms and problems that come from maintaining this heap. but this is C++-specific and does not exist in C. (unless you manually implemented one or something, using malloc()).
@sanchopansa1950
@sanchopansa1950 6 жыл бұрын
good video. but going way too fast. hard to follow.
@batuhanbatuhan6445
@batuhanbatuhan6445 3 жыл бұрын
I dont like intros. Distracting time wasting and yours is waaaayyyyyy too long mate. I understand you make money from hype but our attention is our money. Your customers are kids spending their money on junk. Those who spend mindfully would like videos that explains as much as possible in as little time as possible without overwhelming the learner so they can get to the next thing to learn instead of getting hypee candy fatigue.
@amandamate9117
@amandamate9117 7 жыл бұрын
instead of "anyhow" say "anyway"
@mohamedayad9646
@mohamedayad9646 4 жыл бұрын
ع المطبخ
The Heap: How to exploit a Heap Overflow - bin 0x15
12:11
LiveOverflow
Рет қаралды 102 М.
Did you believe it was real? #tiktok
00:25
Анастасия Тарасова
Рет қаралды 53 МЛН
你们会选择哪一辆呢#short #angel #clown
00:20
Super Beauty team
Рет қаралды 48 МЛН
تجربة أغرب توصيلة شحن ضد القطع تماما
00:56
صدام العزي
Рет қаралды 32 МЛН
THEY made a RAINBOW M&M 🤩😳 LeoNata family #shorts
00:49
LeoNata Family
Рет қаралды 33 МЛН
Garbage Collection (Mark & Sweep) - Computerphile
16:22
Computerphile
Рет қаралды 236 М.
demystifying the secret structure you've been using all along
8:03
Low Level Learning
Рет қаралды 94 М.
First Stack Buffer Overflow to modify Variable - bin 0x0C
11:21
LiveOverflow
Рет қаралды 192 М.
All Rust string types explained
22:13
Let's Get Rusty
Рет қаралды 153 М.
A simple Format String exploit example - bin 0x11
10:01
LiveOverflow
Рет қаралды 167 М.
Pointers and dynamic memory - stack vs heap
17:26
mycodeschool
Рет қаралды 1,4 МЛН
How processes get more memory. (mmap, brk)
6:50
Jacob Sorber
Рет қаралды 70 М.
Did you believe it was real? #tiktok
00:25
Анастасия Тарасова
Рет қаралды 53 МЛН