The ins and outs of context managers and try-finally in Python

Рет қаралды 27,278

Күн бұрын

You can't forget!
Context managers (used with the "with" statement) and try-finally are both constructs in Python that help you ensure certain cleanup code runs even in the presence of exceptions. Becoming familiar with these constructs is important for developing good coding habits in Python. Don't put the burden of remembering how to do cleanup on yourself, use a context manager to let the object handle its own cleanup! In this video, we take a deep dive into the semantics of context managers and try-finally, including the basic examples, sharp corners (and how to avoid them!), as well as less common use-cases.
― mCoding with James Murphy (mcoding.io)
Source code: github.com/mCodingLLC/VideosS...
With statement docs: docs.python.org/3/reference/d...
Try-finally docs: docs.python.org/3/reference/c...
Pythonizing Imgui video: • Pythonizing Imgui (fea...
SUPPORT ME ⭐
---------------------------------------------------
Sign up on Patreon to get your donor role and early access to videos!
/ mcoding
Feeling generous but don't have a Patreon? Donate via PayPal! (No sign up needed.)
www.paypal.com/donate/?hosted...
Want to donate crypto? Check out the rest of my supported donations on my website!
mcoding.io/donate
Top patrons and donors: Jameson, Laura M, Dragos C, Vahnekie, Neel R, Matt R, Johan A, Casey G, Mark M, Mutual Information, Pi
BE ACTIVE IN MY COMMUNITY 😄
---------------------------------------------------
Discord: / discord
Github: github.com/mCodingLLC/
Reddit: / mcoding
Facebook: / james.mcoding
CHAPTERS
---------------------------------------------------
0:00 Intro
0:34 Without context managers...
1:00 Common context managers
2:01 WHEN does cleanup run?
4:36 Try-finally
7:01 With in terms of try-finally
8:11 How to write a context manager
11:38 Suppressing exceptions
12:53 Wrapping an external api
13:54 contextmanager decorator
14:51 "Actual" context managers
15:40 Decimal library
16:21 Limitations
18:51 Thanks

Пікірлер: 62

@Indently Ай бұрын

06:08 is the black magic I'm always excited to learn about in Python.

@syimyuzya Ай бұрын

or in Java (yes, you can do all these black magic in Java's `finally` too 😂

@obsidiansiriusblackheart Ай бұрын

2/3 python channels I watch in one space! I wonder if arjancodes will be here 😂

@jlp2011 Ай бұрын

Man, never knew of return overriding on finally. Such simple, deep, examples.

@DavidMoedinger 28 күн бұрын

I didn't expect it, but today the "try-finally overwrites any returns/throws if you return" actually saved me lots of debugging time :)

@mCoding 28 күн бұрын

Then making this video was worth it 😄

@glorytoarstotzka330 Ай бұрын

the timing of it is insane, I have a program that receives user input and it also polls updates from some API. I kept having random issues with starting a loading screen but something erroring and it never leaving the loading screen, or it stopping the polling but never starting it it then occurred to me that context managers are a thing and even if I don't need clean-up like closing a database connection or closing files, that I can use it for re-turning on the polling and for changing the screen away from the loading screen I then asked myself what happens if you do try: finally: and return in the finally block (I expected it to run the finally, clean-up but then prioritize the first return) this video appearing and these things being covered here 1 day after I implement them in a real project is quite the timing

@maninthebox1678 Ай бұрын

This video is brilliant, I've found that there isn't much information online which clearly and simply explains context managers in a way like this. Appreciate it.

@kezif Ай бұрын

I like how functionality explained using a bunch of prints

@pahan228_killer Ай бұрын

Thank you for the great video James! Clear and straight to the point as always

@bigutubefan2738 Ай бұрын

Really great advice as usual, James.

@malteplath Ай бұрын

There was a lot of stuff in this that I didn't know I didn't know. I thought I had a pretty good understanding of try/finally and context managers, but it turns out there is quite a bit more. Especially that finally pre-empts a return and variations thereof.

@quintencabo Ай бұрын

woah I did not know one with statment could have multiple things in it that is great!

@felipe5195 Ай бұрын

Hey, Excelente video throuought! Wanted to add also a sugestion I think that a vertical split would be a nicer look for both the terminal and the file!

@thefrey9588 Ай бұрын

crazy timing as I was just trying to figure out how to do the `with...as` thing with a selenium driver.

@vincentperrollaz5261 Ай бұрын

Very informative video as usual. I would add that disassemblling a minimal example like from dis import dis def test(cm): with cm: pass dis(test) was pretty surprising

@ramimashalfontenla1312 Ай бұрын

Super useful!

@japedr Ай бұрын

14:30 I would say this is quite useful if you have a complicated enter/exit logic with a lot of local variables used before and after "yield" (e.g. interfacing with a 3rd party library in a specific way) where also wrapping in a class would be overkill because that would be the only use of the class.

@deathdogg0 Ай бұрын

I used to be scared of your videos. Now they're a walk in the park. Thank you, and I'm glad advanced python isn't as scary anymore. Your explanations are great

@TigerWalts Ай бұрын

While I now prefer to deal with absolute file paths, sometimes a context handler for changing the current working directory is very useful.

@quintencabo Ай бұрын

I love context managers

@jeffreyh.1436 Ай бұрын

16:09 The code uses the proposition that pi = 3 + 6 * sum_{n=0}^inf ((2n+1)!!)^2 / (4n+6)!!. I don't know how to prove this. Thanks for the video!

@denoww9261 Ай бұрын

I'd like to provide a (somewhat hacky) use case for suppressing exceptions in __exit__: conditional context managers. In my case, I had an external API with a notion of "tasks" where the code should only run if the task is not done, and once the code runs successfully it should mark the task as done (so it doesn't do it again, and so other systems [or humans] can view which tasks have been completed). In the first line of the with block, I can check if the task is done and raise a custom exception if it is; then, in __exit__, I can simply suppress this exception and exit from the with block. If any other exception was raised it can propagate like normal, and if no exception is raised it marks the task as done.

@nicolasmaclean2895 Ай бұрын

I've got another one for ya! A while ago, I made a Transaction context manager that would run a stack of undoable actions. An exception while performing the actions would tell the Transaction to undo all of those actions, then suppress the error. The gui running the Transaction would just check the transaction's result and decide what to do with that exception. It was super handy for performing file operations.

@denoww9261 Ай бұрын

@@nicolasmaclean2895Nice!

@daineminton9687 Ай бұрын

nice, & thank you

@Nerdimo Ай бұрын

The reason I prefer the contextmanager decorator is just cuz it’s pretty straightforward to define a function vs. designing a class that needs to be instantiated at some point within the program.

@marckiezeender Ай бұрын

I just wish the yield expression returned the exception, instead of raising it.

@omgwtfafterparty Ай бұрын

@@marckiezeendertry creating your own decorator then :)

@marckiezeender Ай бұрын

@@omgwtfafterparty Ok. Here it is: pip install ez-context.

@GuagoFruit Ай бұрын

Context manager would be great for a machine learning program that you need to terminate early. Write the exit as a summary/save function and you're good to cancel it whenever.

@kezif Ай бұрын

Im ml you can use callbacks to save every batch. I think there is even already built in options for popular libraries

@shahiqzaidi Ай бұрын

Great video

@GuyMichaely Ай бұрын

Regarding 7:12 where you explain the semantics of with statements, is the finally necessary at all? It seems to me the code would be functionally equivalent if the finally label were removed and you dedented the last two lines

@Mertly Ай бұрын

It would not be the same; if you moved the last two lines out of finally, they wouldn't be executed if except is raised. But you are right that you don't need finally. Sometimes we want to ignore the exception, and sometimes we want to perpetuate it. We need a way to keep track of that. I store the error in `error` and check if it is undefined at the end of the try/except block to see if any errors were throw that we do care about: ``` throw = random.choice(["value","other"]) error = None fp = open("test.txt", "w") try: if throw == "value": raise ValueError("bad") else: raise Exception except ValueError: print("There was a value error") except Exception as err: print("Something else went wrong") error = err fp.close() if error != None: raise error print("Keep working") ``` Finally is useful as shorthand to do the same thing. Finally perpetuates errors raised in exception blocks: ``` fp = open("test.txt", "w") try: if throw == "value": raise ValueError("bad") else: raise Exception except ValueError: print("There was a value error") except Exception as err: print("Something else went wrong") raise err finally: fp.close() print("Keep working") ``` Finally does not block/ignore exceptions, it puts them off until it's done working.

@GuyMichaely Ай бұрын

I see, thanks!

@marckiezeender Ай бұрын

I wish that the yield statement RETURNED the exception instead of raising it, in the contextmanager decorator. I.e: exc_type, exc_val, exc_tb = yield val

@mCoding Ай бұрын

Copy the implementation and make the changes you want to see!

@marckiezeender Ай бұрын

@@mCoding I did it! And I published it on PyPI under ez_context

@norude Ай бұрын

context managers/try-finaly defer destructors basically the same thing

@fuexfollets5755 Ай бұрын

Discord server gang 🪃

@mCoding Ай бұрын

Much appreciated!

@plato4ek Ай бұрын

This should have been posted yesterday, on Pi Day.

@alfeberlin Ай бұрын

Python 3.10 here; use with decimal.localcontext(decimal.Context(prec=10000)): instead.

@Tomyb15 Ай бұрын

Dissapointed to hear you say that about the contextmanager decorator. It is by far the superior way to do simple context managers in a clean way, even with the try-finally. However, the only reason that you need the try-finally in the generator is that the decorator will re-raise the exception inside the generator at the yield statement for you to catch it and do something with it. In the example shown, as well as most other uses of context managers, you don't care about the exception. You just care about making sure the resource is released/closed. There could be a different decorator (or a parameter to the decorator) that disables the re-raising of the exception inside the generator so that an exception happening inside the with block won't prevent the closing code inside the generator from running. Functions will always look cleaner to me than peppering classes everywhere, so maybe I'm a bit biased.

@marckiezeender Ай бұрын

I just made a cleaner decorator. Instead of raising the exception, the yield statement just returns it. And you can suppress it by returning True. It's published on PyPI under ez-context

@anon_y_mousse 24 күн бұрын

I feel like more languages should have a `with` construct.

@darcash1738 Ай бұрын

just as it appears to me as a noob but try finally seems kinda useless. usually we want to catch our errors and say something, so it seems to make more sense to do try: pass except (Error(s)' types) as E: pass [then you could just write what you want to do regardless on a non-indented line] whats the point

@Indently Ай бұрын

Wuhu!

@mehdi-vl5nn Ай бұрын

"People often say that Python is easy to learn."

@yash1152 Ай бұрын

more like, easy to write

@YuvrajRaghuvanshiS Ай бұрын

The first two minutes of the video are meditating for some reason.

@mrtnsnp Ай бұрын

Ah, there is a bug in the π code. The pi() function is taken from the documentation of the decimal module I noticed, and contains the same bug. If you compare the term t to zero, then the loop terminates as expected once the additional terms become zero within the precision. The current test can never become False.

@mCoding Ай бұрын

The output shown in the video is from running the code shown in the video, so it does terminate. I don't follow the logic you stated in your comment, could you elaborate and give code that exhibits this infinite loop behavior you mentioned?

@mrtnsnp Ай бұрын

@@mCoding My bad. I missed the lasts = s at the beginning of the loop. Without that you have to check if t ≠ 0. Multiple ways to skin a cat I guess. Not I still have to figure out which sequence is used. n follows the odd powers ((2*i-1)²), d follows ((4*i+1)² -1), for i ∈ {1, 2, 3, …}. But I haven't found the sequence yet.

@norude Ай бұрын

Rust

@jullien191 Ай бұрын

Por que amigo?

@kellymoses8566 Ай бұрын

Maybe return shouldn't have been allowed in finally.

@rafacrazyboy Ай бұрын

I think it's shicking how "easy" it is to break python programs that use context managers, as shown with your example at the end. Definitely an important security issue IMO.