Is Your Python Code Unsafe? GIL’s Hidden Secret

Рет қаралды 5,456

Күн бұрын

In this video, we take a deep dive into the inner mechanics of Python's Global Interpreter Lock (GIL), which is a must for all threading and multiprocessing devs, and expose a subtle nuance, which could potentially make your code unsafe. Join us as we explore the impact of GIL on threading and multiprocessing, and discover how it affects the performance and scalability of your Python programs. By the end of this video, you'll gain valuable insights and learn proper threading practices that can help you avoid potential pitfalls.
0:00 - Overview
1:15 - Python threading example
3:50 - Threading
5:46 - GIL (Global Interpreter Lock)
6:24 - CPU bound vs IO bound
7:40 - Purpose of GIL
8:21 - Multiprocessing
9:25 - GILs downfall
Source code - github.com/daniel-boctor/Dani...
WE HAVE A DISCORD NOW! / discord
MUSIC CREDITS:
LEMMiNO - Cipher
• LEMMiNO - Cipher (BGM)
CC BY-SA 4.0
LEMMiNO - Nocturnal
• LEMMiNO - Nocturnal (BGM)
CC BY-SA 4.0

Пікірлер: 49

@TheArtikae 6 ай бұрын

Answer for your crimes: Why have you created an anime OC for the GIL?

@AlexSwanson-rw7cv 3 ай бұрын

She's his GILfriend.

@thatoneglitchpokemon 3 ай бұрын

@@AlexSwanson-rw7cvdeem me speechless 🤣

@VeryaliveWoW 8 ай бұрын

The GIL protects pythons internal interpreter state, that does not mean that you cannot introduce race conditions of your own as in this example. The job of the GIL is not to ensure that data structures that you choose to use are atomically locked

@t0prar 8 ай бұрын

☝ this. Copying a value, sleeping and then applying it is a logical error, not a data race. No language will protect you from that. Also clickbait title. If you're talking about a vulnerability. Please refer to a (possible) CVE. >.

@DanielBoctor 8 ай бұрын

I see a lot of you are mentioning the clickbait title, I wanted to address this, and apologize. I'm still trying to find a balance when it comes to video packaging, and my titles and thumbnails often change frequently within the first 24 hours after publish. I'll work toward packaging that better represents the contents of the video going forward. I'm sorry it was viewed this way 😔. I take responsibility, and it won't happen again going forward.

@VeryaliveWoW 8 ай бұрын

@@DanielBoctor The title may be a little sensational but hey, that's youtube for you. I think the video is well produced and covers an interesting topic, I just wanted to add some context :)

@DanielBoctor 8 ай бұрын

@@VeryaliveWoW ayyyy thank you for the kind words, I appreciate it!

@almightyhydra 4 ай бұрын

@@DanielBoctor I've noticed a number of creators do this and really don't understand the point. Is it simply to bait people who have already watched the video into clicking it again as they think it's something else? Surely that's likely to lead to viewers getting annoyed, ultimately resulting in less viewership?

@naumbtothepaine0 5 ай бұрын

Never know stuff that I thought unrelevant in my Operating System course could have so much potential impact on coding lol

@EgonFreeman 2 ай бұрын

THIS is the sort of thing we should be teaching people. Remember: the Law of Leaky Abstractions _is a thing._

@camelotenglishtuition6394 5 ай бұрын

Amazing explanation

@kolaysgames 2 ай бұрын

the 5ms constant is all you need to add some wildly difficult heisenbugs to your code.

@ally6438 6 ай бұрын

Fascinating thank you

@DanielBoctor 6 ай бұрын

Glad you liked it 😊

@EgonFreeman 2 ай бұрын

I saw someone making the argument that you _can of course introduce these bugs yourself if you really want to._ But the thing is... someone, somewhere, will eventually write code that does this _without meaning to_ - and _that_ should be the key take-away. **Forewarned = forearmed.** If you keep little things like this in memory, it may take you less (sometimes much less) time to realise that it's time to rewrite the function to change how the steps are performed, eliminating the risk and/or bug. Thankfully, most small stuff doesn't need to multithread... but optimisations happen and are getting more advanced, right? Is it possible that your code _might_ be multithreading without you writing it that way?

@greyshopleskin2315 4 күн бұрын

In Haskell, and I guess some others functional languages, I think the compiler might make your code run in parallel. Not really sure, but I think it can do it. Or at least, even if it does not, it is possible. So yeah, it is possible to have your code to run in parallel without you realizing

@TheControlMastr 8 ай бұрын

1:47 what you did here was so amazing 😭 He just made everything a for loop function in .5 🎉

@dracula5752 3 ай бұрын

great video i mean first mistake trying to use multi threading in python if you need speed just use go lmao.

@tiwongemhango955 8 ай бұрын

Love your videos. Hope to see you with a mil+ subscribers one day 🙏🏾

@DanielBoctor 8 ай бұрын

Thank you!!

@CielMC 5 ай бұрын

Great video, but where did you get the Python waifu image from?

@DanielBoctor 5 ай бұрын

I generated it with AI lol

@ionrael 7 ай бұрын

I am studying this at operating system subject of my career and its very interesting. A single process can have multiple threads and the data can be accesed between threads and thats why without GIL your code can have different results. Multithreading itself is a threat to security due the way it works (if not handled properly). Thats why web browser can't be multithreaded, because that could mean a tab runing one page could acces information on another tab. Thats why browsers use multiproccesing instead, F.E. Firefox creates one proccess per tab (and that is also why JavaScript is singleThread only, not because they are lazy to implement multithreading but because its more secure). Processes are managed by the OS, they can also share information and they are organized in a hierarchy tree (father processes could have sons and so on) But I'm not sure if the OS give each procces its own thread or they are executed in other way.

@aurkom 8 ай бұрын

What did you use to create that blue haired character tho at 0:18?

@DanielBoctor 8 ай бұрын

I used an AI image model - specifically live3d.io

@aurkom 8 ай бұрын

Oh thank you, I am quite curious to know the prompt you used. @@DanielBoctor

@DanielBoctor 8 ай бұрын

@@aurkom LOOOOOOL I don't remember the exact one, but it was along the lines of 'blue hair, yellow shirt, blue jacket.......' It was nothing crazy.

@aurkom 8 ай бұрын

I see, thank you! What I found interesting was the python logo in the character. I was surprised thinking that even that logo came up in the generation. I am assuming it was edited later. @@DanielBoctor

@DanielBoctor 8 ай бұрын

@@aurkom oh yeah - I placed that python logo there after the fact, should have mentioned that lol

@oxi2118 8 ай бұрын

High quality vid, thanks a lot for this!

@DanielBoctor 8 ай бұрын

I'm glad you enjoyed, thank you!

@denysholovatyi3149 Ай бұрын

background music too loud

@igorgiuseppe1862 8 ай бұрын

great video. i clicked because of the waifu, but still didnt figure out why she is there lol

@DanielBoctor 8 ай бұрын

LOOOOOOOL it's just a play on words because GIL is also a girl's name

@bitpilot79 8 ай бұрын

Your voice is too low compared to the background music volume. Sometimes it's hard to understand what are you talking.

@DanielBoctor 8 ай бұрын

Thank you for pointing this out, I appreciate it. I'll lower it going forward 😊

@Proprogrammer001 8 ай бұрын

Can I use AI to make a 24 hour version of this video but replace everything you're saying with some gibberish because its so soothing but packed with crazy good info at the same time. I want this to run in the bg while I work but not get tired of absorbing info / info saturated. Even the bg music choice was so good. Don't appreciate the clickbait tho :/

@DanielBoctor 8 ай бұрын

I appreciate the comment, it means a lot In regards to the clickbait, I apologize 😔I'm still trying to find a balance when it comes to video packaging, and my titles and thumbnails often change frequently within the first 24 hours after publish. I'll work toward packaging that better represents the contents of the video going forward.

@ivanleonelvera7540 2 ай бұрын

If Gil ends up doing nothing and the only thing that prevents race conditions is the operating system, why Gil even exist? Let's let all the task to the os and the code can run 4x faster.

@0maxekinge0 8 ай бұрын

Rewrite it in rust 🎉

@cancername 7 ай бұрын

"In fact, it is a mathematical certainty that my CPU will never perform more than 4 operations at once." That's true, but only if you ignore instruction-level parallelism and SIMD.

@HHHjb_ 4 ай бұрын

got an ad literally every minute(note this is more than the default rate by about 3 to 4 times) hence Im not watching this despite how interesting it is

@Zsombro42 8 ай бұрын

Nice video, tho that multiprocessing code is the textbook example of what not to do when you use multiprocessing. I also find your pronunciation of GIL horrific :(

@petrlaskevic1948 8 ай бұрын

How so?