Every Python dev falls for this (name mangling)

  Рет қаралды 135,389

mCoding

mCoding

Күн бұрын

Don't fall for it!
The Zen of Python must not have been invented yet because this feature is confusing, not obvious, implicit, and much more. Private name mangling in Python converts variable usages like __var into _ClassName__var at compile time and there's no warning it's happening until you run into the trap.
― mCoding with James Murphy (mcoding.io)
Source code: github.com/mCodingLLC/VideosS...
Python docs: docs.python.org/3/tutorial/cl...
CPython mangle: github.com/python/cpython/blo...
SUPPORT ME ⭐
---------------------------------------------------
Sign up on Patreon to get your donor role and early access to videos!
/ mcoding
Feeling generous but don't have a Patreon? Donate via PayPal! (No sign up needed.)
www.paypal.com/donate/?hosted...
Want to donate crypto? Check out the rest of my supported donations on my website!
mcoding.io/donate
Top patrons and donors: Jameson, Laura M, Dragos C, Vahnekie, Neel R, Matt R, Johan A, Casey G, Mark M, Mutual Information
BE ACTIVE IN MY COMMUNITY 😄
---------------------------------------------------
Discord: / discord
Github: github.com/mCodingLLC/
Reddit: / mcoding
Facebook: / james.mcoding
CHAPTERS
---------------------------------------------------
0:00 Unexpected output
2:31 Private name mangling
4:23 Python does not have private variables
4:54 Mangling rules
6:51 Mangling at compile time
8:28 Compiler explorer
10:18 getattr and setattr
10:55 To mangle or not to mangle?
13:53 Thanks

Пікірлер: 382
@QuantumHistorian
@QuantumHistorian Жыл бұрын
Wow. I knew about name-mangling, but it's even worse than I expected. Maybe it has valid uses, but it definitely feels like a half-assed implementation of private variables that opens up far more traps than it solves.
@aantoniou96
@aantoniou96 Жыл бұрын
The point isn't to provide private variables, but to prevent name collision in subclasses.
@SeanBracks
@SeanBracks Жыл бұрын
@@aantoniou96 Not true. Read the docs. "it can be used to define class-private instance and class variables"
@thegamecracks1317
@thegamecracks1317 Жыл бұрын
@@SeanBracks Although the documentation does describe name mangling for creating "class private variables", it's not "private" in the conventional sense that it prevents users from accessing those attributes. The same explanation is also given by the doc's tutorial at section 9.6. "Private Variables", pointing out that the feature is only there for the use of collision avoidance. (yes, their terminology for this subject is all sorts of mixed up)
@AbelShields
@AbelShields Жыл бұрын
I had to extend some code that was written entirely like this and it was a nightmare. I ended up rewriting it in a new language. It's so infuriating, python is meant to be open and extensible and writing like this is like you're trying to write in C++, it really makes it worse
@QuantumHistorian
@QuantumHistorian Жыл бұрын
@@aantoniou96 And yet, it doesn't even achieve that, because you can re-use class names between modules further down the inheritance hierarchy. Not to even that it *is* used as a "soft" way of doing private variables in practice, because the two concepts are not a million miles apart.
@knut-olaihelgesen3608
@knut-olaihelgesen3608 Жыл бұрын
You make top tier videos showcasing advanced Python topics. There is just too much beginner videos lying around. Therefore I'm glad to feel challenged by watching your videos
@davefancella
@davefancella Жыл бұрын
I stopped at 1 minute. This name mangling bit is there to provide private variables. Python doesn't strictly enforce private variables, but I use a convention where one underscore means it's internal and you shouldn't use it, but it's readily available just in case. But seriously, I'm a Python developer who's never had this issue.
@NubPaws
@NubPaws Жыл бұрын
I remember learning about name mangling in my Intro to OOP class and we all just wished that we didn't have a question about that on the test. It's something that our lecturer told us to try and avoid and just use the convention of "underscore at the start means private - do not touch". Anyways, another informative and entertaining video from you :D
@j.u.g.y
@j.u.g.y Жыл бұрын
m_ for member.
@naitsab_33
@naitsab_33 Жыл бұрын
@@0neMadGypsy Thats the problem in Python. it doesnt have any way to declare some var as protected. and private doesnt really exist either, because you can use _classname__var to access __var. Its just convention to not mess with attributes (or methods for that matter) that start with an underscore. there is nothing stopping anybody from accessing those "private" attributes. while this seems kinda weird, its kinda nice to have this ability in a scripting language, which python is, because sometimes you want to manipulate some data without using the methods the library gives you and you dont want to do it the right way (which is probably to compile the library yourself with public attributes), because you just want a quick script to do some job, not some high quality solution that will stand the test of time until the universe burns out
@isawadelapradera6490
@isawadelapradera6490 Жыл бұрын
you can use _classname_var to access members of an inherited class. This helps solving some of the puzzles you'll inevitably come up with when trying to solve the obvious resolution order problem arising from multiple inheritance. In fact, this actually makes dunder variables VERY useful given the adecuate context. Note that this context is NEVER publicly accessing members not designed for it.
@cobalius
@cobalius Жыл бұрын
@@0neMadGypsy yeah what this^ guy just wrote
@dutchdykefinger
@dutchdykefinger Жыл бұрын
php had __construct class methods builtin as default and the such built in to the oop part,whenever you defined some var with a new class too, you would EXPLICITLY name them so they would get called lol using those __ because they used to be like that in php and java is just a convention thing people took with them i guess personally i wouldn't touch python with a 10 foot pole if i didn't have to languaged with forced indenting are retarded. i don't trust languages that don't use semicolons as line enders, the entire basic-like structure falls flat if it means having to import C libraries for performance lol
@kmor8829
@kmor8829 Жыл бұрын
I finally understand why they chose ‘Python’ for the name of the language! It wraps itself around your brain and squeezes all the sense out of you! :)
@vectoralphaAI
@vectoralphaAI Жыл бұрын
No Guido chose it because he liked Monty Python's Flying Circus
@kosaelectronics
@kosaelectronics Жыл бұрын
​@@vectoralphaAI 😑
@1234567qwerification
@1234567qwerification 8 ай бұрын
Yes, this is the official, or conscious, explanation :) He could have chosen 'Parrot' (I know, Parrot is now occupied) or 'Monty'.
@ProfMonkeys
@ProfMonkeys Жыл бұрын
I have definitely encountered this issue before and I am blown away by how much more complicated and weird the behavior is than I realized. My general approach has been to just avoid using the "feature".
@TomsonTheOne
@TomsonTheOne Жыл бұрын
I was aware of the initial quirks, but didn't expect it to have so many implications. Thanks for this video; you saved future me a bunch of time.
@CrapE_DM
@CrapE_DM Жыл бұрын
Except if you're always using these variables as if they're ACTUALLY private, none of these implications matter. Use them in a sensical way, and you won't ever have an issue.
@johnbennett1465
@johnbennett1465 Жыл бұрын
@@CrapE_DM if anyone else subclasses your class, you have to hope that they know to follow the rules. Otherwise you have left a hidden mine for them to step on. From personal experience, assuming outside users understand things like this is not a good assumption.
@very_unique_username
@very_unique_username Жыл бұрын
6:26 - this is golden, you've really managed to capture and portray everyones feelings in 3 seconds
@finnaginfrost6297
@finnaginfrost6297 Жыл бұрын
Least clickbait title ever, A+
@mCoding
@mCoding Жыл бұрын
By clicking the video you have fallen for the trap, as advertised. It follows that the title was accurate. Maybe I should put something about mangling in there just so it is more searchable....
@Bentroen_
@Bentroen_ Жыл бұрын
I knew about name-mangling, but I clicked on this video knowing I'd still learn something new. Had no clue about the mangling rules and was also happy you briefly showed the C code, which I was curious about!
@defenestrated23
@defenestrated23 Жыл бұрын
I've been writing python since 2010, and consider myself a bit of a python expert. I've used name mangling in production... a whopping zero times. I've seen it in libraries only a handful of times. I've used it while messing around and found it kind of useless as long as you follow good hygiene. The answer to the question "how to avoid clobbering implementation details" is indeed look at the source code. Python is so heavily dynamic that if you feel the need to use _underscore attributes in a subclass, you really do need to know what the parent class is doing.
@steven7936
@steven7936 Жыл бұрын
Well, if you have a variable that you don't want the user to have access too, I think making it private would be a usecase for doing so. The point of making somethng private is to make it more explicit that the variable can't be changed outside of it's scope.
@alexd7466
@alexd7466 Жыл бұрын
Same; almost never needed it in 12years of using Python. Beginners should avoid this topic completely, simply because they don't need it.
@alexd7466
@alexd7466 Жыл бұрын
@@steven7936 but that concept does not exist in Python; there is no 'private' and there is no way to prevent access (and there is no reason to do so anyway).
@DrDeuteron
@DrDeuteron 9 ай бұрын
Ive used it where I had euclidean a vector class, and then needed a Gibbs version (complex values for quantum angular momentum)...so with mangling I was able to add a few lines do decide if abs() -> sum(getattr(j, self)**2 for j in 'xyz') or if getattr(j, self)*getattr(j, self).conjugate() was required. very slick.
@atrus3823
@atrus3823 Жыл бұрын
Lately, due to many issues like these, I've been borrowing a lot of principles from functional, while still essentially using OO. For one, I try to make stuff immutable with named tuples or frozen dataclasses. Then I try to think of classes more in terms of types and its variables as defining its uniqueness. If something changes, it's a different instance of that type, so no counters. I've found that most of the time, moving counters out of classes makes for more readable, flexible, and loosely coupled code anyway. And if you learn how to leverage comprehensions, map, any, all, sum, itertools, etc., you often don't need counters. For things like widgets, use the ol' "composition over inheritance" mantra. You don't worry about stepping on any component's toes if you have to access them by their instance name anyway.
@ChristianBrugger
@ChristianBrugger Жыл бұрын
What about containers. Functional languages usually have copy-on-write optimizations for modifying them. In python you would need to shallow-copy any data structure (dict, list) when you add or remove anything, killing the performance. How do you get around this?
@ChristianBrugger
@ChristianBrugger Жыл бұрын
BTW I find this very interesting, as I also see the huge benefits of such approaches.
@xelaxander
@xelaxander Жыл бұрын
Yeah, I like that approach, too. I write a lot of data mangling code, which can get complicated really fast.
@volbla
@volbla Жыл бұрын
I'm a bit fuzzy on the details, but don't frozen dataclasses effectively enforce private variables?
@ChristianBrugger
@ChristianBrugger Жыл бұрын
Frozen doesn't make the meners private. It makes them read only. SO once an object is constructed it is immutable. The same as tuples behave.
@Mutual_Information
@Mutual_Information Жыл бұрын
I have avoided these headaches by never using double underscore prefixes.. Im likely to keep that habit.. I get the sense that as Python blew up, it’s gotten pressure to have features of other lower level languages.. static typing and private/public attributes. But b/c those were afterthoughts and the solutions were retrofits, we get these wonky implementations.
@isodoubIet
@isodoubIet Жыл бұрын
99 times out of 100 if you reach for the word "class" in python (and you're not using it just like a struct), you're likely doing something wrong. The language just wasn't built for proper OO support so if you're not building a general-purpose library type like pandas dataframe, the whole thing is best avoided.
@mCoding
@mCoding Жыл бұрын
I'm sure everything will be fixed in Python 4.0 🙃
@vectoralphaAI
@vectoralphaAI Жыл бұрын
@@mCoding I dont think so. At least not from how Guido described Python 4.0 when he was on Lex Fridman podcast.
@trag1czny
@trag1czny Жыл бұрын
great vid as always, was fun to watch with the editing 🤩 also loved the Star Wars references
@bereck7735
@bereck7735 Жыл бұрын
Hey sexy, how are ya.
@mCoding
@mCoding Жыл бұрын
Thank you as always 😀
@AlexanderVlasov
@AlexanderVlasov Жыл бұрын
Thank you so much. I've spent half a day once trying to figure out what the hell is going on when one of the classes used different implementations of one of the functions depending on the conditions (self.__run_command = run_command_locally if local_machine else run_command_via_ssh). It was quite hellish exercise, until I found it out in a hard way. Switched to single underscore.
@benj6964
@benj6964 11 ай бұрын
This is one of the many reasons why I tend to avoid OOP at all reasonable cost
@thespam8385
@thespam8385 Жыл бұрын
I love that for every rule you introduce, you sound increasingly embarrassed that they exist
@tuanazzam4995
@tuanazzam4995 Жыл бұрын
The funny thing is I fell for that in the worst place possible. In my interview.
@mCoding
@mCoding Жыл бұрын
May your career rest in peace lol. Just kidding, now you know, get back up and try again!
@davepruitt
@davepruitt Жыл бұрын
Any interview that tests you on that is absolutely stupid. Interviews should not try to trick you or test whether you know some inane detail such as this, because it has no bearing on how competent you are as a developer.
@tuanazzam5172
@tuanazzam5172 Жыл бұрын
@@davepruitt agreed. but it was just one question. there were more. other questions weren't that bad.🥺
@rb1471
@rb1471 Жыл бұрын
You just flipped my world and I'm surprised I never ran into this issue before. Must be because I rarely ever use classes.
@daemoncluster
@daemoncluster Жыл бұрын
I learned this a long time ago when I went deeper into the language syntax and semantics. If I use either _ or __ then it's because i dont want the property to be accessed directly outside of the class. The key things I learned were to bind it to the class in the constructor and learn how to intuit whether it should be used it or not. I only use getters and setters in certain circumstances, but if i expect the property to be accessed outside of the class, then there is no point in using either _ or __ and I completely drop the @property decorator. Personally, I abide by always binding properties to the class. I see no rationale for placing it outside of the constructor for the very reasons mentioned in this video. Regardless, this is A+ material and I enjoyed the deep dive as it was a refresher for something I learned years ago.
@DrDeuteron
@DrDeuteron 9 ай бұрын
changing instance attributes outside its own instance methods is a sin.
@user-hk3ej4hk7m
@user-hk3ej4hk7m Жыл бұрын
"Well, let me introduce you to the questionable feature behind all of this confusion: classes"
@alerighi
@alerighi Жыл бұрын
Or an even better solution: do not use inheritance from classes that are not interfaces (and thus don't have private fields) at all, and instead use composition, that is better in many ways. Avoid the problem completely basically.
@POINTS2
@POINTS2 Жыл бұрын
I have heard that the single underscore is preferred because you don't know how your classes are going to be used. Using double underscores ("private" variables) makes classes that have them unfriendly.
@skipfred
@skipfred Жыл бұрын
This is the most Python thing I've ever heard. You should know how your classes are supposed to be used (I mean, you designed and wrote them...), and should structure them such that they can only be used in valid ways - or at least such that it's difficult and obvious when you're using them in an unintended way.
@Victor_Marius
@Victor_Marius 7 ай бұрын
I've seen some JavaScript code that was using IIFEs to create an exported object containing any functions and classes you would like to expose. And for some object attributes it was mangling them with a random string created when the IIFE was running.
@timseguine2
@timseguine2 Жыл бұрын
Guido van Rossum is very clever. That ends up being the source of a lot of python quirks like this. This is definitely a very clever way of implementing private variables in python with minimal overhead. In practice it turns out that people just never use it because its behavior is too strange. It's also difficult to figure out what would be a better solution given the design constraints of the language (that every object is basically just a dictionary of key value pairs). Somehow adding real private scopes seems "unpythonic", although I can't explain why. I think a lot of the confusion end up because the behavior results from a "magic" naming convention. Python doesn't really like adding keywords for stuff like this, but I can't help but think if name mangling was somehow linked to a "private" qualifier, that this confusion would not exist. That would have a drawback too though, in that you would have to qualify all uses of the field with "private". That's essentially what python already does actually, except they spell "private" (confusingly) as "__".
@gloweye
@gloweye Жыл бұрын
Real privacy is unpythonic because it's assumed that the programmer isn't trying to fuck themselves over intentionally. Getting/setting an _varname is basically telling the library that you think you know what you're doing, and if you really want to shoot yourself in the foot, then that's your right.
@garriganstafford9113
@garriganstafford9113 Жыл бұрын
Great vid. Will say you missed a potential solution where you prefix the variables with a namespace like in C or ObjC. It’s not great and you do that in those languages of necessity, but it does solve the problem better IMO then manually pre mangling and you don’t have to follow the weird rules of that. Now it’s a bit hair splitting cause depending on how you namespace it’s the same _Class_Var is the same as just mangling but _Module_Class_Var or _pClass_Var is a technically not. It is still not a great solution considering Python is supposed to have built in namespacing, but as an ObjC dev I couldn’t help myself lol
@tonyflury2291
@tonyflury2291 Жыл бұрын
easy to avoid - don't use double underscores on class variable names if you plan to use them outside of the class.
@controlsgirl
@controlsgirl Жыл бұрын
This is so awesome! So thorough but simply shown
@AJMansfield1
@AJMansfield1 Жыл бұрын
The main thing name-mangling is legitimately used to avoid name collisions in code that needs to be able to subclass arbitrary types or otherwise inject some sort of extra management attribute into an object of an unknown type.
@DrDeuteron
@DrDeuteron 9 ай бұрын
use properties for getters and setters: class A: __count = 42 @property def count(self): return self.__count @count.setter def count(self, value): self.__count = value Now if you repeat that code in sub class: class B(A): __count = -999 @property def count(self): [etcetera] then: .>>> a, b = A(), B() >>> a.count, b.count 42, -999 each class has a truly private unmangled count read/write public attribute that don't collide.
@Rizhiy13
@Rizhiy13 Жыл бұрын
Always thought: no underscore == public; single underscore == protected; double underscore == private. Don't see what the problem is. Also, can be thought as: no underscore == stable; single underscore == internal, can be changed at any time; double underscore == don't touch me. In my ten years programming in python name mangling has never been a problem.
@shenlong3363
@shenlong3363 Жыл бұрын
I discovered it last year in an article written by Dan Bader, and I didn't even think about someone trying to change the attribute using the "wrong" name.
@castlecodersltd
@castlecodersltd Жыл бұрын
A really good break down of the issue. Thank you☺
@Kommentierer
@Kommentierer Жыл бұрын
Just don't touch underscored vars from outside. It is for a reason that they are considered to be private.
@HexenzirkelZuluhed
@HexenzirkelZuluhed Жыл бұрын
I did know about this, but the details get fuzzy after a while. I think the solution whenever possible is to use composition instead of inheritance, anyways.
@isodoubIet
@isodoubIet Жыл бұрын
Man, I thought I understood this. Thanks for disabusing me of that misconception.
@krokom9
@krokom9 Жыл бұрын
How does the run-time naming handle conflicts between mangled names? If the mangled name ignores leading underscores in class names, could you not have some class "_____A" (5 underscores) and another class "____A" (4 underscores). If both of these classes have some internal variable __x, how would their mangled names be handled?
@jakobullmann7586
@jakobullmann7586 Жыл бұрын
Your channel is awesome!
@mCoding
@mCoding Жыл бұрын
Thanks so much!
@josephmazor725
@josephmazor725 6 ай бұрын
This video seems like you rage quite a project and just had to vent in the most productive way possible
@quixadhal
@quixadhal Жыл бұрын
What SHOULD happen, if it were designed properly, is an attempt to access the "private" variable __count would throw an exception for an access violation. This would put python in the same camp as other languages that have public and private keywords for class variables. Instead, it just mangles the name and it's on the programmer(s) to play nice and not abuse or work around it.
@alexd7466
@alexd7466 Жыл бұрын
no, people should just learn the language instead of trying to copy Java. Why would you make something 'private' anyway? in some languages it makes sense, but not in Python: name conflicts are extremely rare in Python because it has Modules. instead of using getters/setters every real Python dev uses @property or writes custom descriptors. Learn to use the __set__ special method.
@bereck7735
@bereck7735 Жыл бұрын
This is a very informative yet, a very funny video, nice one james.
@Not_Today_1969
@Not_Today_1969 Жыл бұрын
Hi there! Could you please make a review of new features of 3.11 release of Python? Thanks for your work!
Жыл бұрын
OMG, thanks for the info! It was frightening before going to bed tonight.. :-O
@reimusklinsman5876
@reimusklinsman5876 Жыл бұрын
I thought the convention was that a single underscore is supposed to represent a private variable. I don't recall hearing anything about two starting underscores. Edit: just got to the point in the video where he confirms that
@Veptis
@Veptis 10 ай бұрын
I have been learning and using Python for 4+ years. And even teaching an intro class a few times. And I run into new issues all the time. Right now I struggle with relative imports just because I wanted to be fancy and skip copying code. Two days later I was forced to figure out packaging and refactored a lot of the script I had to now have a module of utils. Feels good, but I still haven't solve my initial issues of reusing a complex function elsewhere. Can't import that anymore because it imports the relative utils package now -.-
@yash1152
@yash1152 2 ай бұрын
1:23 what's that R🖊 icon shown near the line number 32's gutter in the pycharm?
@unperrier5998
@unperrier5998 Жыл бұрын
This name mangling is fine, it solves a real world problem. The only issue is with the rare side effects of the poor implementation, for example when modules a mangled (and sometimes not). But they're rare because mangling local variables is not an issue, after all it's already local, who cares of the name python uses to access the locals dictionary. That's why I think it's fine as it solves real world problem, as explained in the video. Speaking of the video, it focuses too much on those side effects that you will never hit, and if you do you can always look it up. I'm afraid that videos like this one deter programmers from python, thinking "wow this language is unpredictable".
@Jonathan-rm6kt
@Jonathan-rm6kt Жыл бұрын
I feel like a deep understanding of this would prevent tens of hours of struggling. But I know I’m going to have to learn this the hard way
@sevdalink6676
@sevdalink6676 Жыл бұрын
One more reason to choose functions over classes whenever possible 😀
@vivianyyd
@vivianyyd Жыл бұрын
I've never liked embedding semantic information into variable names, feels old-fashioned a little like Hungarian notation.. This feels like a funny way to build around something that is already not great. Are there channels similar to this one which detail quirks of other common programming languages like Java?
@JossWhittle
@JossWhittle Жыл бұрын
@mCoding this seems utterly horrendous. But can it be made even more horrendous by combining with __slots__ ? And if so, how would such a monstrosity work?
@mCoding
@mCoding Жыл бұрын
Indeed it can be made more horrendous using slots, you end up with mangled slot names. I dare not post such cursed dark magic in the clear web.
@JossWhittle
@JossWhittle Жыл бұрын
@@mCoding I've also just seen someone combining it with slots and metaclasses and... wow. Just wow. So hurried to see if they could, they never stopped to ask if they should.
@DrDeuteron
@DrDeuteron 9 ай бұрын
10:53. can you have a base class or mixin class _MyMangler: @classmethod def _mangle(cls, attr): return f""_{cls.__name__}__{attr}_" so internal getattr(attr, self) are replaced with with getattr(self._mangle(attr), self).... not that I would ever do that.
@Android480
@Android480 Жыл бұрын
If you treat it genuinely like a private field, then it kind of works. You just need to know that private things exist. But yeah I remember seeing the mangled name in my debugger for the first time. Wild…
@user-hf6zn6mu9f
@user-hf6zn6mu9f 11 ай бұрын
As I used to know in earlier versions of python when you tries to reassign any strictly private property in a class you will get the error!
@syllogism5843
@syllogism5843 Жыл бұрын
I feel like if you need name mangling functionality, name-space your private methods/members by manually mangling instead so there's no dark magic
@xakkep9000
@xakkep9000 Жыл бұрын
Fantastic unexpected feature that I don't even supposed about!!!
@lengors7327
@lengors7327 Жыл бұрын
I tipically use name mangling in my projects and I've never had any issues with it regarding bugs or unpredictable code. But I guess that's just because I wasnt a moron that despite knowing those variables shouldnt be touched outside the class still attempted to do so...
@tobb10001
@tobb10001 Жыл бұрын
So... what would happen, if I imported your Widget example, inherited from it, and my class would also be named Widget, and I also defined a `__count`? Both should mangle to `_Widget__count`. So the mangling wouldn't even meet it's purpose because we chose weird names? Weird.
@yomajo
@yomajo Жыл бұрын
Can you even do class Animal(Animal): ...?
@mCoding
@mCoding Жыл бұрын
Although it is certainly bad practice, Python will not stop you from creating two classes with the same name even if one inherits from the other. In this case, even mangled names will clobber each other. Think of it like a punishment for a bad decision. (Unless you didn't know some parent of a parent had the same name, in that case you just got blindsided).
@OrchidAlloy
@OrchidAlloy Жыл бұрын
class Widget(module.Widget): # ouch
@JackDespero
@JackDespero Жыл бұрын
If you inherit from a class and you name the daughter class like the mother class, that is completely on you, my man. Do not complain about hammer's functionality just because you keep hitting your fingers on purpose.
@kawashirov
@kawashirov Жыл бұрын
Woooa, I knew about this feature, but I didnt know how cursed is it. You can fell into this even if u don't use "private" variables.
@DMSBrian24
@DMSBrian24 11 ай бұрын
Damn I knew about this and thought it probably makes sense as a middle ground between true private and no safeguarding at all, but never realized how many rough edges it has...
@slovhol
@slovhol Жыл бұрын
This applies to "private" methods as well
@steven7936
@steven7936 Жыл бұрын
I'm new to python, coming from Typescript. I had no idea you could do name Mangling as a means to set variables to private and public. Time to whip out chat GPT to learn more .
@mCoding
@mCoding Жыл бұрын
I wonder if chatgpt knows about mangling 🤔
@steven7936
@steven7936 Жыл бұрын
@@mCoding It does :) I asked it myself. It also clarfied the _ notation as well which I didn't know was a thing.
@alexd7466
@alexd7466 Жыл бұрын
it is not meant for private/public ; these concepts do not exist in Python. double-underscore is only to avoid name conflicts in some situations where classes are inherited and happen to have the same variable-names.
Жыл бұрын
this channel is incredible. can anyone recommend me other coding channels as good as this one?
@scullyy
@scullyy Жыл бұрын
Who are these people naming stuff with double underscore if they didn't intend to name mangle!
@richardcoppin5332
@richardcoppin5332 Жыл бұрын
This pretty much echoes the embolism the more experienced devs have on StackOverflow when a less experienced dev tries to use double underscores in their variable names.
@Corncycle
@Corncycle Жыл бұрын
i understand the tradeoffs of the different conventions for "private" variables, but i don't understand this last bit at the very end of the video: "in the wild, the most common approach is to use the single underscore and to just hope subclasses don't clobber your implementation details. if you're very good about keeping your classes defined as if they were statically typed, then you could also write a linter rule to help you." what would the linter rule be, and how would it help you with this problem? my guess is that you would just have your linter forbid you from overriding variables, but i dont see why this only makes sense in a context where you've defined your classes as if they were statically typed. id appreciate if anyone could shed some light!
@mCoding
@mCoding Жыл бұрын
You've got the right idea about the linter rule. As for why it only works if your classes are statically typed, if your classes are not statically typed, how does the linter know what the variables in your class or any of its parents are? If it doesn't know what they are, it can't warn you if there is an overlap. Keep in mind the linter does not run at runtime, so it cannot simply inspect the object in question. Instead, it operates on the source code, typically by computing the AST or CST and checking based off of that.
@Corncycle
@Corncycle Жыл бұрын
@@mCoding i still don't understand the problem. lets pretend someone implements a class ClassA which initializes a variable _counter to 0 on all of its instances. if i make a class ClassB that inherits from ClassA, and i initialize _counter as an empty list on all of my ClassB instances, shouldn't this be as much of a visible problem as if i initialized _counter as 100,000 on all of my ClassB instances? regardless of whether my linter can tell what type _counter is supposed to be, it should be able to see that it is being assigned in ClassA and overridden in ClassB, yeah? i must admit i don't know much about linters, so its possible i might just not have the background to understand the problem, but i think its equally possible i might just be misunderstanding the situation to begin with lol
@dwarftoad
@dwarftoad Жыл бұрын
Seems like any use of a variable starting with double underscore other than "self.__foo" in the class definition should be an error.... (or warning.) Is there a function that returns the mangled name as a string, given a variable or property? (For setattr()/getattr(), debug messages, etc.)
@mCoding
@mCoding Жыл бұрын
I don't think they expose the official function, though you can feel free to copy the one from this video (and maybe optimize it a bit). I wrote it by going line by line though what the CPython implementation does and translating it to pure Python.
@TylerLarson
@TylerLarson Жыл бұрын
Wow. I had a use for this feature 3 days ago, and I had long forgotten about it. I just refactored my code so that I no longer had a use for this feature. Maybe I'm better off forgetting it again.
@kamurashev
@kamurashev Жыл бұрын
Thanks! I saved this video to just throw the link at the person who would ever ask me: -why do you hate python?
@m0Ray79
@m0Ray79 Жыл бұрын
It will be interesting to investigate those cases applied to Cython...
@juicetin942
@juicetin942 Жыл бұрын
I guessed it will print 0, got the reason right. It was a first for me
@puddles1364
@puddles1364 Жыл бұрын
I took a deep breath with you.
@antina314
@antina314 Жыл бұрын
How about writing a custom __getattr__/__setattr__ on your class that would check if its not modified from outside of the owner class to know there is an issue early?
@hjewkes
@hjewkes Жыл бұрын
I've always felt a little bad not using `__` on my instance variables, but apparently this feature isn't all its cracked up to be lol
@CrapE_DM
@CrapE_DM Жыл бұрын
If you TREAT them as private variables, you don't have to worry about any of the [seemingly] weird behaviors. (Other than how it mangles strange class names)
@alexs3973
@alexs3973 10 ай бұрын
My brain is now positively mangled after watching this.
@mCoding
@mCoding 10 ай бұрын
You better see a __private practitioner!
@sachinfulsunge9977
@sachinfulsunge9977 Жыл бұрын
Just a feedback, the memes are kinda disrupting my focus and maybe of some other users too, i liked your previous video types, where we had no memes in between jumpscaring, just your serious voice and some serious feature analysis, thanks.
@mme725
@mme725 Жыл бұрын
6:24 -ish, but if it always fixes one underscore name in mangling always, does that mean it can generate conflicts? Like, if you daresay have classes A, _A, and __A (double underscore) in the same project, would all 3 resolve to "_A__variable" names?
@funkenjoyer
@funkenjoyer Жыл бұрын
It would seem so but keep in mind they are "kept" under their respective class anyway
@AlyceIsFree
@AlyceIsFree Жыл бұрын
I think the thing folks forget about name mangling is that its like the cover of a switch, it doesn't stop you from using the switch, but it does give you a warning that the switch might do something bad
@AlyceIsFree
@AlyceIsFree Жыл бұрын
This also feels like some descriptor hell, because if defined outside of the class its not mangled, and the fact that the names or transformed on get/set of the variable, which would correspond the the behaviour of __get__ and __set__ of descriptors, if they were storing the name as `owner.__name__` + `self.__name__`
@dovedozen
@dovedozen Жыл бұрын
This is one of those things that drove me so bonkers when I was trying to figure out how to do [whatever] with objects in ren'py that I just. gave up and picked a way that didn't involve objects at all. I wonder if going back to it with the context in this video would help at all?? Something to try next time I feel like learning by tying my brain in a knot for sure.
@daniel_rossy_explica
@daniel_rossy_explica Жыл бұрын
I too tried once to use Ren'py. Couldn't get it to work, got frustrated and left it.
@dovedozen
@dovedozen Жыл бұрын
@@daniel_rossy_explica I'm way stubborn, so I've gotten okay at it, but it really is one of those things you HAVE to be stubborn to learn! Every time I flip over a "surely there is a straightforward way to do [whatever]" rock, a million "lol no. change the source code" bugs crawl out from under it..
@daniel_rossy_explica
@daniel_rossy_explica Жыл бұрын
@@dovedozen From what I remember there is close to no documentation either so you HAVE to look at the source code to understand it. And then they have that strange way of writing scripts for the games. I remember thinking "why don't you just use straight python to make the games instead of this complicated parser?"
@dovedozen
@dovedozen Жыл бұрын
@@daniel_rossy_explica Ooh but the parser taught me how parsers work; I'm legitimately fond of it now 😭 And the documentation has definitely gotten better, although if you're not checking the discord and forums you definitely miss out on a lot! Then again I've never really encountered anything open-source that had GOOD documentation, so maybe I just don't know how good it could be? Also, the script thing is fantastic for lazy people (ME (WROTE AN ENTIRE CUSTOM THING TO AVOID HAVING TO USE BRACKETS ROUTINELY (TOOK WEEKS but made me feel like a WIZARD))).. you would not believe how much of my ability to write VN stuff is predicated on the programming part being as brainless as possible; Writing Brain and Programming Brain are NOT friends with each other.
@dovedozen
@dovedozen Жыл бұрын
@@daniel_rossy_explica ok I say the documentation is fine and IT ISN'T BAD but I just remembered that I had to learn to print dictionaries to find out that the reason changing a thing called "name" wasn't working is because it was actually called like. "something_name" internally & it was very much not written down anywhere that this was the case so. again I may just be living in the dirt over here
@lior_haddad
@lior_haddad Жыл бұрын
8:48 I saw 424242 and immediately thought of b"BBB"... I wonder if you can use metaclasses to make this even worse. (Or maybe make it function "as you'd expect"?)
@brianb.6356
@brianb.6356 Жыл бұрын
I have known about name mangling for a while and I honestly don't think that the details are that bad, nor have I ever been confused by it.
@codingcrashkurse6429
@codingcrashkurse6429 Жыл бұрын
Same here. But I also would never use it in my code anyway
@JansthcirlU
@JansthcirlU Жыл бұрын
you must be VERY smart, wow
@CrapE_DM
@CrapE_DM Жыл бұрын
@@JansthcirlU When you use these variables as if they're actually private (you can only access them from within the class definition), none of the "weird" behavior become a problem, since you won't run into them.
@artur1998g
@artur1998g Жыл бұрын
The video started like something obvious, but actually I didn't know that "mangle" works so weird.
@AnimeGIFfy
@AnimeGIFfy Жыл бұрын
lmao this video gets more and more hilarious as it goes, i love it
@vectoralphaAI
@vectoralphaAI Жыл бұрын
So do we use __x for sudo "private" variables or should we just omit them entirely and have everything public by default like always.
@raianmr2843
@raianmr2843 Жыл бұрын
Just use a single underscore.
@doc7115
@doc7115 Жыл бұрын
It obviously prevents suckers like James to access the “private” variable so I call this feature a huge win.
@aonodensetsu
@aonodensetsu Жыл бұрын
pseudo
@thedjsente5858
@thedjsente5858 Жыл бұрын
interesting overview, but i do use double underscore myself to denote private(although this is making me rethink that). On the one hand, I think you should NOT be able to access a variable that is suppose to be private from outside the class (which double underscore accomplishes), however on the otherside here, the ability to write a different double underscore variable from outside the class feels absurd. If we aren't going to get an language based update to block this, then perhaps we need a linter rule to block being able to write to double underscore from outside class implementation.
@JackDespero
@JackDespero Жыл бұрын
Private variables do not exist in Python. You are able to access everything by design. This is all about avoiding accidental name collisions.
@BloodyMobile
@BloodyMobile Жыл бұрын
5:00 I feel thrown back to JavaScript's evaluation of values and how the weirdest shit can end up as false, true or just a string...
@ceefarbs560
@ceefarbs560 Жыл бұрын
God I stuck on this bug once for days in Pygame where layers are set in the exact same way
@fejfo6559
@fejfo6559 Жыл бұрын
What if you mix manual and automatic mangling? Does it refer to the same variable?
@groaningmole4338
@groaningmole4338 Жыл бұрын
Before I saw this video: If you want to use private variables, do it in C++, not Python. After I saw this video: If you want to use classes, do it in C++, not Python.
@rikschaaf
@rikschaaf Жыл бұрын
I don't understand why it is an issue, since private fields are meant to be PRIVATE. If you try to publicly access a field that is private, then it's no wonder that you get unexpected behavior (though, I would have preferred an illegal access exception or something, but that is just my Java brain). So yea, you gotta know what double underscore means and how to use it properly and if you really need to access a private field from a public context, then make sure you know what you're doing. In a way, prefixing the accessor of a private variable with _ in Python is like setting declared private fields to be accessible with reflection in Java. Both make private fields accessible, but they do it in a completely different way.
@badoli1074
@badoli1074 Жыл бұрын
But you shouldn't get "unexpected behaviour", because that's simply confusing. You should get an error. I shouldn't need to "guess" that i made a mistake, i should know.
@MindlessTurtle
@MindlessTurtle Жыл бұрын
@@badoli1074 - that's the IDE's job -- not Python's.
@31redorange08
@31redorange08 Жыл бұрын
I agree. Why would anyone access a "private" field from outside the class? The video creator says every Python developer will fall for this feature. This implies that every Python developer doesn't know what they are doing, at least at some point.
@shadeofsound23
@shadeofsound23 Жыл бұрын
@6:30 perfect use of focus mangling. lol
@FekuEntertainmentLtd
@FekuEntertainmentLtd Жыл бұрын
Hey James, Has anyone ever told you, you sound like "Corey Schafer" -- another renowned name in the Python community.
@digiryde
@digiryde Жыл бұрын
Great explanation!
@marionascimento450
@marionascimento450 Жыл бұрын
To be fair in other languages nefarious authors can still access private fields via reflection (java), memory transmutation (rust), etc.
@TechSY730
@TechSY730 Жыл бұрын
I think the lesson to learn here is that inheriting state in a non-controlled type hierarchy is something to try to avoid. This applies to OOP languages in general, not just Python. With Python having an extra reason for why, this lack of sane way of namespacing member variables across inheritance.
@lepidoptera9337
@lepidoptera9337 Жыл бұрын
One should NEVER put state in a class to begin with. State is a singleton. There can be only one. Classes are the antithesis of singleton objects. Data can be in classes (where it is also usually misplaced), but state... never.
@TechSY730
@TechSY730 Жыл бұрын
@@lepidoptera9337 I'm a little confused as to how you are using "state" and "data" in you comment. What I was referring to as "state" is mutable state stored within the instance, like mutable instance variables. When you say "data", do you mean like immutable (or read-only) stuff in the instance?
@lepidoptera9337
@lepidoptera9337 Жыл бұрын
@@TechSY730 State is the persistent information that controls a program's primary function. Think about protocols between two communicating programs. If the sender requests a transmission, the receiver either has to guarantee reception by design and acknowledge that the data has been received (possibly even with a check sum mechanism) or flag a transmission error. What data is being transmitted is completely irrelevant to the protocol. The relevant state variables are only controlling that the transmission was valid. In a simple protocol that might be half a dozen binary flags, even if a GByte of data is flowing per second. A user interface of a program meant to be used by humans is, in this sense, also a protocol. The user expects that certain inputs produce reliable actions inside the program and he expects feedback about the progress of those actions. Whatever variables are being used to assure that, that is "state". If I click on "Save", I expect the program to save my data, no matter what is in that data. I expect some kind of confirmation that my data has been saved, even if it is subtle. If this doesn't happen, then there will be an extremely unhappy, insecure user. I had to work with programs that did none of that. I won't name names, but it was hell. So, yeah, the meaning of "state" here is NOT the entire content of your memory. State is what is really important and what can only exist once because the "user" only exists once. And this problem isn't just "external" to a program. It is also internal to the way classes communicate with each other. And this is where the problem begins to grow to epic proportions: how do you make sure that state is immutable if anybody can call a state changing member function? Yes, your "state variable" in an object may be local, but your call graph to the member function of that class isn't. Anybody can change your state, anywhere, any time and there is nothing in the syntax of a typical OOP language to prevent that from happening. OOP therefor gives this false sense of security, when in reality the atomizing of state across multiple classes makes it nearly impossible to track the actual state changes without constant analysis of the call graph.
@jermeekable
@jermeekable Жыл бұрын
type checkers catch this right? like mypy or pyright?
@delinquenice
@delinquenice Жыл бұрын
Have a like for the anger fade. Perfect! Interesting content too
@rhtcguru
@rhtcguru Жыл бұрын
Whoa!! I'm so glad you made this video. I was literally jaw open watching this.
@seasong7655
@seasong7655 9 ай бұрын
1:48 I think this behaviour is pretty nuts. They should rather give out an error, if you access a variable that's supposed to be private. But that's just my opinion.
@SteveRowe
@SteveRowe Жыл бұрын
I wasn't aware of this, but it makes me glad that I don't start my member variables with __.
Unlocking your CPU cores in Python (multiprocessing)
12:16
mCoding
Рет қаралды 288 М.
super/MRO, Python's most misunderstood feature.
21:07
mCoding
Рет қаралды 209 М.
О, сосисочки! (Или корейская уличная еда?)
00:32
Кушать Хочу
Рет қаралды 6 МЛН
Balloon Pop Racing Is INTENSE!!!
01:00
A4
Рет қаралды 16 МЛН
where is the ball to play this?😳⚽
00:13
LOL
Рет қаралды 14 МЛН
Python's 5 Worst Features
19:44
Indently
Рет қаралды 68 М.
Positional-only and keyword-only arguments in Python
9:48
mCoding
Рет қаралды 83 М.
25 nooby Python habits you need to ditch
9:12
mCoding
Рет қаралды 1,7 МЛН
str vs bytes in Python
8:35
mCoding
Рет қаралды 76 М.
__new__ vs __init__ in Python
10:50
mCoding
Рет қаралды 203 М.
Async for loops in Python
16:36
mCoding
Рет қаралды 55 М.
Asynchronous Web Apps in Python
15:30
mCoding
Рет қаралды 26 М.
Python lists remember what you did to them
10:04
mCoding
Рет қаралды 127 М.
Python Generators
15:32
mCoding
Рет қаралды 127 М.
Do YOU Understand WHAT JUST HAPPENED!? 😂 #shorts
0:57
LankyBox World
Рет қаралды 1,6 МЛН
ТЕЛЕФОН МЕНЯЕТ ЦВЕТ😅 #upx
0:34
RanF
Рет қаралды 639 М.
Массаж головы пранк🤣
0:55
Kirya Kolesnikov
Рет қаралды 5 МЛН
Satisfying Vend 😦 Ep.5 #shorts #satisfying #vendingmachine
0:23
TYE Arcade
Рет қаралды 17 МЛН
Американцы красят асфальт?
0:27
BAZAR CLUB
Рет қаралды 188 М.