Python 3.12 Generics in a Nutshell

Рет қаралды 46,686

Күн бұрын

Generics in Python 3.12 can transform your code by allowing classes to be used with different data types, enhancing flexibility, and catching bugs early. In this video, I'll dive into generic classes and subclasses and show you how they can streamline your development process.
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
🔥 GitHub Repository: git.arjan.codes/2024/tuesday_...
💻 ArjanCodes Blog: www.arjancodes.com/blog
✍🏻 Take a quiz on this topic: www.learntail.com/quiz/zmbpcx
Try Learntail for FREE ➡️ www.learntail.com/
🎓 Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
🛒 GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
Social channels:
💬 Discord: discord.arjan.codes
🐦Twitter: / arjancodes
🌍LinkedIn: / arjancodes
🕵Facebook: / arjancodes
📱Instagram: / arjancodes
♪ Tiktok: / arjancodes
👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Dale Hagglund
- Kit Hygh
- Alexander Milden
- Bean
🎥 Video edited by Mark Bacskai: / bacskaimark
🔖 Chapters:
0:00 Intro
Chapters:
1:09 Generic classes
3:25 Subclassing Generic Classes
5:34 Conclusion
5:54 Outro
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!

Пікірлер: 131

@ArjanCodes 2 ай бұрын

👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis

@ponysmallhorse Ай бұрын

after many years working as python developer, I must say that dynamic typing was a mistake. We wen't full circle and now type annotating our code to make it workable. I wish we just added static typing to python in the end, could make python faster.

@dawidskreczko 2 ай бұрын

I love your content. Keep going!

@ArjanCodes 2 ай бұрын

Thank you so much Dawid! ❤️‍🔥

@diegol_116 2 ай бұрын

Good video! It's a shame that this syntax hasn't been there since previous versions, it's much more readable.

@rockNbrain 2 ай бұрын

Generics is great, I use them a lot in typescript 🎉

@daviddelaney363 2 ай бұрын

Looks like Arjan is the new Knuth...good stuff.

@ArjanCodes 2 ай бұрын

You give me way too much credit, haha. Knuth is next-level!

@ReRubis 2 ай бұрын

Great video as always. :)

@ArjanCodes 2 ай бұрын

Glad you liked it!

@tulliolevichivita5130 2 ай бұрын

The best case of using generics for me was my own implementation of a B+Tree. I think the next step in python must be an implementation kind of a standart library like C++ has.

@IvanIvanov-dk6sm 2 ай бұрын

After your first video about Generics i started to use it in my project

@ArjanCodes 2 ай бұрын

I'm glad to hear, Ivan! Are you enjoying working with them?

@IvanIvanov-dk6sm 2 ай бұрын

@@ArjanCodes actually, yes. But some moments are not that straightforward when classes are inherited from Generic-based class. And also when you need to create, for example, a dict attribute of a class. Which keys are types and values are generic-based object instances. While declaring If you highlight type of values inside generic-based instances, MyPy will complains about it. For example: self.my_dict: dict[type[MyObject], CustomSet[int]]. But if i remove int after CustomSet, Mypy will be satisfied. Usually i use generics to work properly with TypeVars when i need to bound main type. Also i used them to create custom OrderedSet[T] and DequeSet[T] objects

@tikendraw 2 ай бұрын

Hey, arjan. Can you make a tutorial on how to use logging in production , what are dos' and dont's when logging in work environment, what function to log. how will it go with async , a tutorial on a real world example? I have seen the logging video on your channel , it is good , but some serious knowledge and real world use case will be helpful to understand the actual use. Make a tutorial when you are free and willing . Thank you.

@ratulr 2 ай бұрын

mcoding has a in depth video on this topic

@HandcartRule46 2 ай бұрын

Super useful, thanks!

@ArjanCodes 2 ай бұрын

Glad it was helpful!

@herberttlbd 2 ай бұрын

I think we are getting to the point where statically-compiled languages with type-inferencing are starting to look more GvR Pythonic than modern Python. Anybody remember the days when Django was the villain because they were writing Python as if it were Java?

@alexufba 2 ай бұрын

Generics reminds me Templates in C++

@101Mant 2 ай бұрын

@@alexufbaor generics in Java or C# or other languages that have been around for decades.

@ravm84 2 ай бұрын

I just have an impression that implementing all those type-features will make Python even more complex than C++ with far less performance, and still it won't be type-safe. But anyway, this is interesting. Maybe there are benefits I can't see now. I started to use type hints quite long time ago, which makes the code easier to understand, especially with support with IDEs which supports that. Anyway, very cool video.

@senzmaki4890 2 ай бұрын

well type-features are opt-in, you can always just write untyped python but the libraries you use for instance are typed making your dev experience much much better

@jma42 2 ай бұрын

most of this will benefit library authors, no more T= t.TypeVar("T") bloat!

@daze8410 2 ай бұрын

I suspect it will be a requirement for the removal of the GIL in the future

@frustratedalien666 2 ай бұрын

I have worked on a large enough code base with absolutely no type hints and non-existent documentation and I hated every minute of it. I can't figure out what argument I am supposed to pass to these methods, so each time I touch these legacy repos, I make it a point to add type annotations that seem rational to me. Makes my life easier in the long run. You don't have to use it, though, if you don't like it.

@Naej7 2 ай бұрын

I don’t see how it could be more complex than C++, nor be less performant

@user-wz7is8kt7i 2 ай бұрын

Are there series about functional programming ?

@aaronsteers 2 ай бұрын

Hi, Arjan. Whenever new typing features come out for subsequent python versions, I always feel left in the dark about how/if they'll be backported with the "from __future__" magic - and whether prior versions of Python will at least tolerate the syntax, if not directly support it. If so, I can use them. If not, I have to wait a few years. I can never really enjoy videos like this unless I have those questions answered - otherwise, it's just another homework assignment for me to add to my backlog. Can you answer this for me?

@heinereniscaicedo7510 2 ай бұрын

What a Great video! new Python's Generics are so similar to golang's Generics

@guruyaya 2 ай бұрын

I used it (in the old py10 syntax) to create a Pydantic collection type, that puts the return list of values into an attribute called "data" Why? I found that a lot of json parsers don't like to get a list as a reply - they expect a json objet. This solves the need to create a custom object for every time I want to return a list of multiple object

@orlanino 2 ай бұрын

This is ok when one uses a basic type like int, float, etc. But what if I want to build, let's say a Stack[myCustomObject]? What do I have to implement in the class? I assume __eq_, at least. Probably iterator as well?

@DanielCordey 2 ай бұрын

I think I will use this feature. It's a safe approach in some situation, but I'm not going to generalize all of my codes with it;

@ArjanCodes 2 ай бұрын

That's a good way to go about it, Daniel!

@ButchCassidyAndSundanceKid 2 ай бұрын

I prefer to use SingleDispatch. However, Python still doesn't support MultipleDispatch natively (have to pip install another package), it'll be great if they could include that in the next version.

@LucasAlmeida-ef9io 2 ай бұрын

SingleDispatch is used to implement polymorphism. Think about two different classes with the method "add". SingleDispatch will be applied in the "self" parameter of this method, turning possible to behave differently depending on the type of self. So it's kinda low level thing that is used as workaround to implement function overload. Then, I think that it will not be implemented natively soon...

@glensmith491 2 ай бұрын

I think I now understand why I never understood generi type. Because they are not generic types but are generic types with value added. For instance, a generic drug is simply the basic drug sold without error checking (or at least minimal error checking), no extensions and no cool packaging.

@sinasalahshour 2 ай бұрын

this is amazing

@ArjanCodes 2 ай бұрын

Thank you!

@evgeniy_toropchin 2 ай бұрын

yeah, another python type hinting which we will use only for ide :D so cool

@kiraleskirales 2 ай бұрын

The Python community is giving low priority to supporting generic variables a.k.a. Higher Kinded Types. It is currently impossible to annotate an operation that keeps a parametric container as is, but changes the type it contains (the package "returns" offers this functionality but you need to add a plugin to the mypy checker). They (Guido Van Rossum himself!) say that it is a lot of work and it is not worth it. I was surprised to read that because even simple operations cannot be fully specified in the annotations. What is your take? Should operations like that be avoided in the first place? Will they ever support them?

@R15YZF 2 ай бұрын

Thanks for sharing! In Java what makes generics even more useful is to restrict the generic types to an interface. Ex. Numerics can be an interface which is implemented by both Int and Float. Is such a thing possible with generics in python?

@marckiezeender 2 ай бұрын

Yes. You can use typing.Protocol to define an interface. Also, many abstract classes in the python standard library also behave as interfaces. I believe that Arjan as a couple videos on Protocols.

@Naej7 2 ай бұрын

You can create a type which is equal to int|float

@shlomokallner3180 Ай бұрын

In pre-3.12 (i am currently using 3.10 and 3.8) enabled that on the 'TypeVar' construct via the 'bound' parameter. I use all the time.

@jeanhadrien 2 ай бұрын

What is the difference with just passing a type argument in the init method of the class?

@cheukmingau983 2 ай бұрын

Say you have 2 instance of the class, 'a = Stack[int]' and 'b = Stack[str]', we can know that a can only stack integer but b can't, it can only stack strings. With type annotation we can leverage static checkers like mypy to verify whether our code is correct or not. If we pass the type in the constructor, as the attribute is defined in the instance level, we cannot annotate the correct type in the class level and the static checker or even inteliisense will not work. Also, specifying the type through generics is like a 'meta-configuration' where we can specify type alias, say in Python3.12 we can specify 'type IntStack = Stack[int]' or 'IntStack: TypeAlias = Stack[int]' in previous versions. This configuration works before the initialization stage thus another level of configuration.

@jeanhadrien 2 ай бұрын

@cheukmingau983 Oh, I did not think about intellisense/type checking. Good point! Thank you 😊

@abomayeeniatorudabo8203 2 ай бұрын

I use generics all the times in python and also typescript.

@NdamuleloNemakh Ай бұрын

What is the difference between using len(seld._container) == 0 vs self._container == [] on is_empty()

@rikschaaf 2 ай бұрын

What about covariance and contravariance? So let's say I have a list[Cat] containing several cats. Can I assign that to a variable of type list[Animal] or is there some sort of way to define the type list[object extends Animal]? (or some similar syntax). If there isn't co-/contravariance modifiers, does that mean you can then add a Dog object to that list[Animal], causing actions on the original list[Cat] to break whenever Cat-specific methods are called or Cat-specific fields are accessed?

@cheukmingau983 2 ай бұрын

In Python you can call whatever attributes / methods you want, just whether or not it throws an AttributeError. For your case of adding a Dog object to that list [Animal], normally if you follow the intellisense and static type checker, you would not be able to call Cat specific methods that Animal does not. When you annotate your variable as list[Animal], you have already told the static type checker that you only consider it as Animal, thus it will only allow (at the static type checker level) Animal methods. If you don't care about the intellisense / static type checker stuff, you can always call any method you want and result in AttributeError

@NicolasChanCSY 2 ай бұрын

FYI, on top of the reply above, I would like to supplement a few things copied from PEP-0695 (proposal of this new syntax of generic type) regarding variance. The PEP writes "This PEP eliminates the need for variance to be specified for type parameters. Instead, type checkers will infer the variance of type parameters based on their usage within a class. Type parameters are inferred to be invariant, covariant, or contravariant depending on how they are used." (An overview of the algorithm of determining variance is in the PEP if you are curious.) `TypeVar()` also has a new `infer_variance` argument in Python 3.12 besides the original `covariant` and `contravariant` arguments. In the section "Explicit Variance" in PEP-0695, the authors explained why they rejected it: "We rejected this idea because variance can generally be inferred, and most modern programming languages do infer variance based on usage. Variance is an advanced topic that many developers find confusing, so we want to eliminate the need to understand this concept for most Python developers." Similar writings can be found in the Motivation section as well.

@marckiezeender 2 ай бұрын

variance is inferred based on the signatures of the methods you define in the generic class. If there are methods that return T, then T will be covariant. If there are methods (other than __init__) that accept T as a parameter, then T will be contravariant. If both are true, then T is invariant. The type checker will determine this automatically.

@rikschaaf 2 ай бұрын

@@marckiezeender but I can think of situations for parameters to be either-or. In java in a list, the addAll() method parameter would be Collection

@marckiezeender 2 ай бұрын

@@rikschaaf It still infers the variance, based on the variance of those parameterized generics. I should have written "etc.," because the cases with the param type and return type are only the simplest examples.

@user-tb8se1gq2l 2 ай бұрын

So python is now fully OOP and also fully procedural?

@ravenecho2410 2 ай бұрын

Interesting items in python i encounter pydantic - awesome for json validation Langgraph, langchain - do pretty well for api abstraction their pydantic parser is pretty good Sqlglot - pretty interesting capabilities for transpiling and inspecting abstract syntax trees NamedTuple - simple class to clean up access Dotenv - easy way to get variables Sqlite - built in DB which obeys the api pattern Sqlalchemy - an ORM which can be nice Mypy - static type checker Then i would like to point that a module with functions is exactly equivalent in access and calls for a class without any data I think finally, TDD (test driven development) will play a MASSIVE role in Artificial Intelligence and Machine Learning based features. Theres absolutely no way around it... i think😅 Great vid as always!

@twelvethis3979 2 ай бұрын

Unfortunately, I cannot use generics like this at the moment. I would love to, though. My company supports only Python 3.9 and it'll be five years - I guess - before I can use Python 3.12 😅

@Naej7 2 ай бұрын

I feel you

@shlomokallner3180 Ай бұрын

❤ i am stuck on 3.8

@julians.2597 20 күн бұрын

@@shlomokallner3180 given then 3.8 is going to be End of Life even for security patches before the end of this year, meaybe you can motivate them to at leaste make the horrifying jump to 3.9?

@berdeter 2 ай бұрын

Well I come from the Java world and adding a bit more type checking always feels better for me so this feature is important. It's always a bit wierd that type errors in python are only actually warnings for your IDE.

@ravm84 2 ай бұрын

Yes it is just a warning, and you must use an IDE which understands that or perhaps some type checking tools... I just wonder, if you employ all those type-checiking features why not use C++ or Java instead. You will get real type checking and far better performance.

@NicolasChanCSY 2 ай бұрын

@@ravm84 If I understand correctly, the point is to have gradual typing (so this can be compatible to all Python code because they don't want another Python 2-->3) and development speed (Python is still strongly typed in runtime. Python developers can still write their scripts quick and dirty without caring any typing stuff, while providing an option in case some projects want to get good code quality via type checking and better autocomplete).

@markuswagner5754 2 ай бұрын

@@NicolasChanCSYin the end the choice of programming language depends on your use case. Python is good for quick and dirty scripting. But type annotations can help, when projects get a bit bigger and it becomes less transparent, which types you have to expect. I also find type annotations quite helpful for writing python libraries, as it reduces the amount of assumptions your library user will take and you may not thought about. Nevertheless for a high performance program or highly robust application Python maybe is the wrong language.

@jamesclarke7259 2 ай бұрын

Love this, could i request more jenkins please?

@taraskuzyk8985 2 ай бұрын

I wanted to use generics but PyCharm currently offers no proper type hinting for them :( Very sad

@rafiullah-zz1lf 2 ай бұрын

Love the way u teach. But cant afford the $ for the course.

@littleconan7929 2 ай бұрын

The main issue I have with it is that it depends on modern typing that requires import. Docstring typing does not require import and thus avoids circular imports. but you cannot do everything with it.

@BambuzMitZett 2 ай бұрын

Google: if TYPE_CHECKING

@marckiezeender 2 ай бұрын

you can guard your imports with `if typing.TYPE_CHECKING` if you want to avoid circular imports. (also requires __future__.annotations until 1.13)

@littleconan7929 2 ай бұрын

@@marckiezeender Wow, I did not know this statement existed. Seems great !

@marckiezeender 2 ай бұрын

@@littleconan7929 Yeah, I use it all the time, to the point that I put ALL my annotation-only types behind the guard, just for consistency.

@GuihVicentini Ай бұрын

Why have generics in a non-typed language?

@Alticroo Ай бұрын

I really do not understand the aversion to type hinting. I cannot stand working on codebases that have mystery functions, arguments, etc... IDE intellisense alone is worth it; At the very least, this largely improve productivity and interpretability. This isn't about solving runtime bugs by hard enforcing types; it's about avoiding those situation entirely.

@natnaelberhane3141 2 ай бұрын

At this point, it's just better to use a statically typed, high level language like Java or C#

@smalltimer666 2 ай бұрын

So basically python wants to be Julia now ? I thought the only (and perhaps big) advantage of python over proper languages was the quick and dirty dynamic typing. Without that, python is just a slow C++.

@user-ml5em9eo2e 2 ай бұрын

Frankly at that point, just switch to rust

@andreacazzaniga8488 2 ай бұрын

such a limited use case that the chances of abuse are far greater than normal implementation

@maleldil1 2 ай бұрын

I've noticed in Arjan's videos about type hints that you cover the surface well, but minor details suggest that you either don't understand things properly or you don't use them in practice. For example, if you're using Python 3.12, there's no reason to import `Optional` from typing. Instead, just use `T | None`. Also, using `Literal[0]` for your `sum` method doesn't make any sense. `Literal[0]` doesn't mean that the value might be zero - `int` already means that; instead, it means that you're explicitly writing down the value `0` in your code, which you aren't. This makes no sense for `sum` since you're never actually writing the value `0` in your code. In general, `Literal` is very rarely used, and is usually only helpful in adding types to existing APIs. Also, there's no reason to explicitly annotate the local variable `total` inside `average` since its type can be inferred to be the same as `sum`'s return type, even if it's generic.

@deathineyes 2 ай бұрын

Typescript in Python?!

@Deadlious 2 ай бұрын

These generics seem a mess. They do not feel intuitive or pythonic. I guess they can bring some positives if you are maintaining some sort of public library that handles all types of inputs or if you are working on a project with an extremely large codebase, but honestly, I don't think I'll ever use them.

@kayakMike1000 2 ай бұрын

Here i thought it was Gen-ER-ics.

@user-mu5vj4bv3d 2 ай бұрын

Interesting, but I don't understand how to utilize it in real life :)

@Naej7 2 ай бұрын

Generics are only useful for low level needs, where generalization in needed. Most of the time, what you must code is specific, not generic.

@Nalewkarz 2 ай бұрын

I don't like the direction of typing changes in Python. Something wrong is happening in Python lately. I like type hints but it's going too far in my opinion. I mean generic types are OK but using construction like (int, float) instead of Union is just one big tragedy. Python core developers are making language for describing the code rather than just typing, so in the future in some cases You will have to know Python, and in other cases Type hinted Python will be the requirement like Javascript vs Typescript. In the same time there is no type checker in standard library. It's one big mess lately.

@MarioDanielCarugno 2 ай бұрын

Please don't let Python become a monster like C++ !! Looks like it is starting to accumulate features and everytime you have more ways of doing the same things I really don't understand why languages are inexorably updated with new features... is it something mandatory or what ?

@muditjha7235 2 ай бұрын

Agree 100% I wont be surprised if they make type checking a core part of the language.

@squishy-tomato 2 ай бұрын

"using construction like (int, float) instead of Union is just one big tragedy" they are semantically different: union allows mixed types in the collection. that construct doesn't.

@Naej7 2 ай бұрын

People rejecting types annotations really scare me. It’s intellectually wrong, and I really don’t want to work on their codebases, it seems like a nightmare.

@hieu8276 2 ай бұрын

After many years, it’s sad to see Python is going to same route with C++, ton of features and syntactical sugar that essentially do the same thing.

@UNgineering 2 ай бұрын

Oh no, python is becoming TypeScript with all the syntactic complexity

@maleldil1 2 ай бұрын

Generics have been available for almost 10 years, since 3.5, via TypeVar and Generic[T]. This syntax is much better than before. Anyway, Python's type system is still very far from TypeScript's complexity, as you can't do much meta-programming with it.

@nick_59 2 ай бұрын

I don't know man. If it wasn't for type hints I wouldn't ever use Python

@UNgineering 2 ай бұрын

@@nick_59 I love type hints, it's when we get into generics with abstract factories, that's when the syntax starts to get in the way of development.

@Naej7 2 ай бұрын

@@UNgineeringDeveloping abstract factories without any type is suicidal

@UNgineering 2 ай бұрын

@@Naej7 I like to live on the edge

@iankaranja7765 2 ай бұрын

Doesn't all this type obsession defeat the purpose of python the dynamic/duck typing language.

@NicolasChanCSY 2 ай бұрын

It is optional. And type hinting can provide useful context to both IDE and code users what are the inputs and outputs of the functions and classes. Instead of reading the reference documents, you can just read the function signature to have a better idea.

@marckiezeender 2 ай бұрын

Well, for one, duck typing is just the dynamic-typed version of interfaces (called Protocols in python).

@Naej7 2 ай бұрын

No, it’s complementary

@Netum6am 2 ай бұрын

But can I have code like this? class Factory[T]: def create(self) -> T: return T() If not, then there are still no generic types in Python. Only a generic annotation.

@marckiezeender 2 ай бұрын

@dataclass class Factory[T]: t: type[T] def create(self) -> T: return self.t() value = Factory(int).create() # value is known to be an int.

@Netum6am 2 ай бұрын

@@marckiezeender I am not asking how to write a factory. That was just an example. My point about generic types stands. I know there are solutions to this example. I could also write a factory functions that creates factory classes. I could write it as a decorator. I assume there are many other solutions. Nevertheless, I had cases where type checking still failed me even with these workarounds. In my usecases I don't often need a generic object like in your example. I really need a generic class. In those cases, mypy often spetacularly fails. Although that said, it seems to sometimes fail even in some trivial situations.

@marckiezeender 2 ай бұрын

@@Netum6am The code I gave you behaves exactly how you want your code to behave. Both at runtime and during type checking. Factory(A) is is an A factory, and calling the create() method returns an instance of A. Also, using an arbitrary decorator on a class is currently not supported by mypy; it only recognizes @dataclass - like decorators.

@Netum6am 2 ай бұрын

@marckiezeender Please, stop answering something nobody asked. I find it a bit rude. My original comment is a simple yes/no question. Does the presented code work in python 3.12?

@marckiezeender 2 ай бұрын

@@Netum6am I don't mean to be rude. No, the syntax that you made up is not valid python syntax. But there IS slightly different syntax that allows you to do the same thing. I know the Factory is just an example, I was building on the example you provided.