Visual Guide to Transformer Neural Networks - (Episode 2) Multi-Head & Self-Attention
Рет қаралды 167,960
Күн бұрынVisual Guide to Transformer Neural Networks (Series) - Step by Step Intuitive Explanation
Episode 0 - [OPTIONAL] The Neuroscience of "Attention"
• The Neuroscience of “A...
Episode 1 - Position Embeddings
• Visual Guide to Transf...
Episode 2 - Multi-Head & Self-Attention
• Visual Guide to Transf...
Episode 3 - Decoder’s Masked Attention
• Visual Guide to Transf...
This video series explains the math, as well as the intuition behind the Transformer Neural Networks that were first introduced by the “Attention is All You Need” paper.
--------------------------------------------------------------
References and Other Great Resources
--------------------------------------------------------------
Attention is All You Need
arxiv.org/abs/1706.03762
Jay Alammar - The Illustrated Transformer
jalammar.github.io/illustrated...
The A.I Hacker - Illustrated Guide to Transformers Neural Networks: A step by step explanation
jalammar.github.io/illustrated...
Amirhoussein Kazemnejad Blog Post - Transformer Architecture: The Positional Encoding
kazemnejad.com/blog/transform...
Yannic Kilcher KZfaq Video - Attention is All You Need
www.youtube.com/watch?v=iDulh...
@HeduAI 3 жыл бұрын
*CORRECTIONS* A big shoutout to the following awesome viewers for these 2 corrections: 1. @Henry Wang and @Holger Urbanek - At (10:28), "dk" is actually the hidden dimension of the Key matrix and not the sequence length. In the original paper (Attention is all you need), it is taken to be 512. 2. @JU PING NG - The result of concatenation at (14:58) is supposed to be 7 x 9 instead of 21 x 3 (that is to so that the concatenation of z matrices happens horizontally and not vertically). With this we can apply a nn.Linear(9, 5) to get the final 7 x 5 shape. Here are the timestamps associated with the concepts covered in this video: 0:00 - Recaps of Part 0 and 1 0:56 - Difference between Simple and Self-Attention 3:11 - Multi-Head Attention Layer - Query, Key and Value matrices 11:44 - Intuition for Multi-Head Attention Layer with Examples
@amortalbeing 2 жыл бұрын
Where's the first video?
@HeduAI 2 жыл бұрын
@@amortalbeing Episode 0 can be found here - kzfaq.info/get/bejne/ap6XdbN9nMeseo0.html
@amortalbeing 2 жыл бұрын
@@HeduAI thanks a lot really appreciate it:)
@omkiranmalepati1645 Жыл бұрын
Awesome...So dk value is 3?
@jasonwheeler2986 Жыл бұрын
@@omkiranmalepati1645 d_k = embedding dimensions // number of heads
@thegigasurgeon Жыл бұрын
Need to say this out loud, I saw Yannic Kilcher's video, read tonnes of materials on internet, went through atleast 7 playlists, and this is the first time I really understood the inner mechanism of Q, K and V vectors in transformers. You did a great job here
@HeduAI Жыл бұрын
This made my day :,)
@afsalmuhammed4239 Жыл бұрын
True
@exciton007 10 ай бұрын
Very intuitive explanation!
@EducationPersonal 9 ай бұрын
Totally agree with this comment
@VitorMach 7 ай бұрын
Yes, no other video actually explains what the actual input for these are
@nitroknocker14 3 жыл бұрын
All 3 parts have been the best presentation I've ever seen of Transformers. Your step-by-step visualizations have filled in so many gaps left by other videos and blog posts. Thank you very much for creating this series.
@HeduAI 3 жыл бұрын
This comment made my day :,) Thanks!
@bryanbaek75 2 жыл бұрын
Me, too!
@lessw2020 2 жыл бұрын
Definitely agree. These videos really crystallize a lot of knowledge, thanks for making this series!
@Charmente2014 2 жыл бұрын
ش
@devstuff2576 2 жыл бұрын
@@HeduAI absolutely awesome . You are the best.
@nurjafri 3 жыл бұрын
Damn. This is exactly what a developer coming from other backgrounds need. Simple analogies for a rapid understanding. Thanks a ton. Keep uploadinggggggggggg plss
@Xeneon341 3 жыл бұрын
Agreed, very well done. You do a very good job of explaining difficult concepts to a non-industry developer (fyi I'm an accountant) without assuming a lot of prior knowledge. I look forward to your next video on masked decoders!!!
@HeduAI 3 жыл бұрын
@@Xeneon341 Oh nice! Glad you enjoyed these videos! :)
@ML-ok9nf 9 ай бұрын
Absolutely underrated, hands down one of the best explanations I've found on the internet
@HuyLe-nn5ft 11 ай бұрын
The important detail that set you apart from the other videos and websites is that not only did you provide the model's architecture with numerous formulas but you also demonstrated them in vectors and matrixes, successfully walked us through each complicated and trivial concept. You really did a good job!
@rohanvaidya3238 3 жыл бұрын
Best explanation ever on Transformers !!!
@forresthu6204 2 жыл бұрын
Self-attention is a villain that has struck me for a long time. Your presentation has helped me to better understand this genius idea.
@malekkamoua5968 2 жыл бұрын
I've been stuck for so long trying to get the Transformer Neural Networks and this is by far the best explanation ! The examples are so fun making it easier to comprehend. Thank you so much for you effort !
@HeduAI 11 ай бұрын
Cheers!
@rohtashbeniwal9202 Жыл бұрын
this channel needs more love (the way she explains is out of the box). I can say this because I have 4 years of experience in data science, she did a lot of hard work to get so much clarity in concepts (love from India)
@HeduAI Жыл бұрын
Thank you Rohtash! You made my day! :) धन्यवाद
@ja100o Жыл бұрын
I'm currently reading a book about transformers and was scratching my head over the reason for the multi-headed attention architecture. Thank you so much for the clearest explanation yet that finally gave me this satisfying 💡-moment
@chaitanyachhibba255 3 жыл бұрын
Were you the one who wrote transformers in the fist place, because no one explained it like you did. This is undoubtfully the best info I have seen. I hope you please keep posting more videos. Thanks a lot.
@HeduAI 3 жыл бұрын
This comment made my day! :) Thank you.
@rishiraj8225 2 ай бұрын
Coming back after a year, just to revise the basic concepts. It is still the best video on YT. Thanks Hedu AI
@Clammer999 Ай бұрын
I’ve gone through dozens of videos on transformers and the multi-head attention is one of the most complex mechanisms that require not only a step-by-step explanation, but be accompanied with a step-by-step animation, which many videos tend to skip over but this video really nails it. Thanks so much!
@wireghost897 11 ай бұрын
Finally a video on transformers that actually makes sense. Not a single lecture video from any of the reputed universities managed to cover the topic with such brilliant clarity.
@alankarmisra 9 ай бұрын
3 days, 16 different videos, and your video "just made sense". You just earned a subscriber and a life-long well-wisher.
@sebastiangarciaacosta5468 3 жыл бұрын
The best explanation I've ever seen of such a powerful architecture. I'm glad of having found this Joy after searching for positional encoding details while implementing a Transformer from scratch today. Valar Morghulis!
@HeduAI 3 жыл бұрын
Valar Dohaeris my friend ;)
@andybrice2711 2 ай бұрын
This really is an excellent explanation. I had some sense that self-attention layers acted like a table of relationships between tokens, but only now do I have more sense of how the Query, Key, and Value mechanism actually works.
@MGMG-li6lt 3 жыл бұрын
Finally! You delivered me from long nights of searching for good explanations about transformers! It was awesome! I can't wait to see the part 3 and beyond!
@HeduAI 3 жыл бұрын
Thanks for this great feedback!
@HeduAI 3 жыл бұрын
“Part 3 - Decoder’s Masked Attention” is out. Thanks for the wait. Enjoy! Cheers! :D kzfaq.info/get/bejne/nbBpnsR72KubaJ8.html
@adscript4713 3 ай бұрын
As someone NOT in the field reading the Attention paper, after having watched DOZENS of videos on the topic this is the FIRST explanation that laid it out in an intuitive manner without leaving anything out. I don't know your background, but you are definitely a great teacher. Thank you.
@HeduAI 3 ай бұрын
So glad to hear this :)
@EducationPersonal 9 ай бұрын
This is one of the best Transformer videos on KZfaq. I hope KZfaq always recommends this Value (V), aka video, as a first Key (K), aka Video Title, when someone uses the Query (Q) as "Transformer"!! 😄
@HeduAI 9 ай бұрын
😄
@shubheshswain5480 3 жыл бұрын
I went through many videos from Coursera, youtube, and some online blogs but none explained so clear about the Query, key, and values. You made my day.
@HeduAI 3 жыл бұрын
Glad to hear this Shubhesh :)
@cracksomeface Жыл бұрын
I'm a grad student currently applying NLP - this is literally the best explanation of self-attention I have ever seen. Thank you so much for a great vid!
@kafaayari 2 жыл бұрын
I won't say this is the best explanation so far, but this is the only explanation. Others are just repeating the original paper.
@rayxi5334 Жыл бұрын
Better than the best Berkeley professor! Amazing!
@devchoudhary8892 Жыл бұрын
best, best best explanation on transformer, you are adding so much value to the world.
@sujithkumar5415 Жыл бұрын
This is quite literally the best attention mechanism video out there guys
@jamesshady5483 Жыл бұрын
This explanation is incredible and better than 99% of what I found on the Internet. Thank you!
@laodrofotic7713 19 күн бұрын
This video is amazing, finally someone intelligent that can actually teach things! I believe this creator is truly talented and fully understands the concept, unlike many others. She also not only understand but is very good at communicating the information in a clear and intuitive way, which I haven't seen before. I have only watch this video up to 12:33 at the time of this comment but I am truly impressed, will watch the others. Thank you Hedu for sharing this.
@wayneqwele8847 6 ай бұрын
Thank you for taking the time explain from a linear algebra perspective what actually happens. Many teachers on youtube are comfortable just leaving it at math symbols and labels. Showing what actually happens to matrice values has sharpened my intuition of what actually happens under the hood. Thank you.🙏
@darkcrafteur165 Жыл бұрын
Never posting but right now I need to thank you, I really don't believe that it exists a better way to understand self attention than watching your video. Thank you !
@giridharnr6742 Жыл бұрын
Its one of the best explainations of Transformers. Just mind blowing.
@Srednicki123 Жыл бұрын
I just repeat what everybody else said: these videos are the best! thank you for the effort
@bendarodes61 2 жыл бұрын
I've watched many video series about transformers, this is by far the best.
@madhu1987ful Жыл бұрын
Wow. Just wow !! This video needs to be in the top most position when searched for content on transformers and their explanation
@HeduAI Жыл бұрын
So glad to see this feedback! :)
@frankietank8019 10 ай бұрын
Hands down the best video on transformers I have seen! Thank you for taking your time to make this video.
@pedroviniciuspereirajunho7244 Жыл бұрын
To visualize the matrices helped me to understand better transformers. Again, thank you very much!
@ghostvillage1 Жыл бұрын
Hands down the best series I've found on the web about transformers. Thank you
@fernandonoronha5035 2 жыл бұрын
I don't have words to describe how much these videos saved me, thank you!
@bochengxiao1352 2 жыл бұрын
Thank you so much! It's the best Transformer video ever! Really hope more on other models.
@HeduAI 2 жыл бұрын
Glad to hear that! :) Do let me know if there are certain models that you would like to see covered in future videos.
@shivam6565 Жыл бұрын
Finally I understood the concept of query, key and value. Thank you.
@raunakdey3004 Жыл бұрын
Really love coming back to your videos and get a recap on multi layered attention and the transformers! Sometimes I need to make my own specialized attention layers for the dataset in question and sometimes i dunno it just helps to just listen to you talk about transformers and attention ! Really intuitive and helps me to break out of some weird loop of algorithm design I might have gotten myself stuck at. So thank you so so much :D
@persianform Жыл бұрын
The best explanation of attention models on the earth!
@skramturbo8499 2 жыл бұрын
I really like the fact that you ask questions within the video. In fact those are the same questions one has and first reading about transformers. Keep up the awesome work!
@jackziad 3 жыл бұрын
Your videos are so good at getting complex ideas across in an intuited way. You are like the 3Blue1Brown equivalent for AI. Keep it up and keep producing high-quality video content, at your own pace of course 😋
@HeduAI 3 жыл бұрын
3Blue1Brown is one of my favorite channels! Therefore, you comparing these videos to that channel is one of the best compliments ever. Thank you! :)
@rishiraj8225 Жыл бұрын
@@HeduAI yes.. this is awesome explanation comparable to 3Blue1Brown.. make more..
@user-ne2nr2yi1h 7 ай бұрын
The best video I've ever seen for explaining transformer.
@mrkshsbwiwow3734 Ай бұрын
This is the best explanation of transformers on KZfaq.
@geetanshkalra8340 2 жыл бұрын
This is by far the best video to understand Attention Networks. Awesome work !!
@Scaryder92 2 жыл бұрын
Amazing video, showing how the attention matrix is created and what values it assumes is really awesome. Thanks!
@sowmendas812 Жыл бұрын
This is literally the best explanation for self-attention I have seen anywhere! Really loved the videos!
@binhle9475 Жыл бұрын
Your attention to details and information structuring are just exceptional. The Avatar and GoT references on top were hilarious and make things perfect. You literally made a story out of complex deep learning concept(s). This is just brillant. You have such a beautiful mind (if you get the reference :D). Please consider making more videos like this, such a gift is truly precious. May the force be always with you. 🤘
@nizamphoenix 8 ай бұрын
Being a professional in this field for ~5years can say this is by far the best explanation of attention. Amused as to why this doesn't pop up on YT's recommendation for attention at the top. Probably, YT's attention needs some attention to fix its Q, K, Vs
@HeduAI 8 ай бұрын
You made my day :)
@oliverhu1025 Жыл бұрын
Probably the best explanation of transformers I’ve found online. Read the paper, watched Yannic’s video, some paper reading videos and a few others, the intuition is still missing. This connects the dots, keep up the great work!
@prashantchauhan1742 2 жыл бұрын
This is Gold. I was confused after going through the paper. And boom this cleared it..
@HeduAI 2 жыл бұрын
Cheers! :D
@Abhi-qf7np 2 жыл бұрын
You are the best😄😄, This is THE Best explanation I have ever seen on KZfaq for Transformer Model, Thank you so much for this video.
@artukikemty Жыл бұрын
Thanks for posting, by far this is the most didactic Transformer presentation I've ever seen. AMAZING!
@RafidAslam 3 ай бұрын
Thank you so much! This is by far the clearest explanation that I've ever seen on this topic
@MCMelonslice Жыл бұрын
This is the best resource for an intuitive understanding of transformers. I will without a doubt point everyone towards your video series. Thank you so much!
@maryamkhademi 2 жыл бұрын
Thank you for putting so much effort in the visualization and awesome narration of these series. These are by far the best videos to explain transformers. You should do more of these videos. You certainly have a gift!
@HeduAI 2 жыл бұрын
Thank you for watching! Yep! Back on it :) Would love to hear which topic/model/algorithm are you most wanting to see on this channel. Will try to cover it in the upcoming videos.
@freaknextdoor9040 3 жыл бұрын
Hands down, this series is the best one explaining the essence of transformers I have found online!! Thanks a lot, you are awesome!!!!
@HeduAI 3 жыл бұрын
Cheers! 🙌
@adithyakaravadi8170 2 жыл бұрын
You are so good, thank you for breaking down a seemingly scary topic for all of us.The original paper requires lot of background to understand clearly, and not all have it. I personally felt lost. Such videos help a lot!
@franzanders7762 2 жыл бұрын
I can't believe how good this is.
@SOFTWAREMASTER 11 ай бұрын
Most underrated video about transformers. Going to recommend this to everyone. Thankyou
@oludhe7 3 ай бұрын
Literally the best series on transformers. Even clearer than statquest and luis serrano who also make things very clear
@jackderrida Жыл бұрын
Holy crap, this tutorial is good! I've had GPT-4 generate me so many analogies to refresh my understanding of the same concepts you perfectly explain here.
@SuilujChannel 2 жыл бұрын
thanks for these great videos! The visualizations and extra explanations on details are perfect!
@Ariel-px7hz Жыл бұрын
Such a fantastic and detailed yet digestible explanation. As others have said in the comments, other explanations leave so many gaps. Thank you for this gem!
@dominikburkert2824 3 жыл бұрын
best transformer explanation on KZfaq!
@HeduAI 3 жыл бұрын
So glad to hear this! :D
@cambostrongbo 3 жыл бұрын
Speaking my kind of language, all made sense when you started referencing avatar
@HeduAI 3 жыл бұрын
Hahaha :D Glad to know....
@hubertkanyamahanga2782 10 ай бұрын
I am just speechless, this is unbelievable! Bravo!
@jonathanlarkin1112 3 жыл бұрын
Excellent series. Looking forward to Part 3!
@HeduAI 3 жыл бұрын
“Part 3 - Decoder’s Masked Attention” is out. Thanks for the wait. Enjoy! Cheers! :D kzfaq.info/get/bejne/nbBpnsR72KubaJ8.html
@chenlim2165 Жыл бұрын
Bravo! After watching dozens of other explainer videos, I can finally grasp the reason for multi-headed attention. Excellent video. Please make more!
@JDechnics 2 жыл бұрын
Holy shit was this a good explanation! Other blogs literally copy what the paper states (which is kinda confusing), but you explained it in such a intuitive and fun way! Thats what I called talent!!
@mrmuffyman 2 жыл бұрын
You are awesome!! I watched Yannic Kilcher's video first and was still confused by the paper, probably because there's so much detail skipped over in the paper and Kilcher's video. However, your video goes much slower and in depth so the explanations were simple to understand, and the whole picture makes sense now. Thank you!
@jasonpeloquin9950 Жыл бұрын
Hands down the best explanation of the use of Query, Key and Value matrices. Great video with an easy example to understand.
@VADemon Жыл бұрын
Excellent examples and explanation. Don't shy away from using more examples of things that you love, this love shows and will translate to better work overall. Cheers!
@adarshkone9384 Жыл бұрын
have been trying to understand this topic for a long time , glad I found this video now
@madhusharath 3 жыл бұрын
Wow! Truly wow! Your ability to explain complex stuff in layman terms + reference to well known series/anime shows how in-depth your understanding actually is!
@HeduAI 3 жыл бұрын
Your comment made my day :)
@suttonmattp Жыл бұрын
Honestly understood this far better from this 15 minute video than from the 90 minute university lecture I went to on the subject. Really excellent explanation.
@davidlazaro3143 Жыл бұрын
This video is GOLD, it should be everywere! Thank you so much for doing such an amazing job 😍😍
@cihankatar7310 Жыл бұрын
This is the best explanation of transformers architecture with a lot of basic analogy ! Thanks a lot!
@an_experienced_guy 9 ай бұрын
This is by far the best and the most clear and insightful explanation of transformers, I've tried to understand it through multiple blogs and videos and stack exchange answers, this is the first time every component became clear to me and how they all work in conjunction. Thanks a lot for this series. Amazing explanations.
@andrerocha155 8 ай бұрын
Congrats! Fantastic explanation. You really used your Transformer skills to "pay attention" to the most important issues in this matter and explained it as simply as needed to be understood even by a 10 years old child!
@jboyce007 6 ай бұрын
If only I saw your videos earlier. As everyone in the comments says, these are THE BEST videos on the subject matter found anywhere! Thank you so very much for helping us all!
@HeduAI 6 ай бұрын
Cheers! :)
@mariosconstantinou8271 Жыл бұрын
These videos are amazing, thank you so much! Best explanation so far!!
@wolfie6175 2 жыл бұрын
This is an absolute gem of a video.
@ariasardari8588 2 жыл бұрын
Your ability to convey concepts is quite impressive! Probably the best tutorial video I've ever seen. From now on, every time I open KZfaq, I first check if you have a new video It was fantastic! I greatly appreciate it.
@HeduAI 2 жыл бұрын
Thanks a lot Aria! Really means a lot :)
@adityaghosh8601 2 жыл бұрын
Blown away by your explanation . You are a great teacher.
@kennethm.4998 2 жыл бұрын
You have a gift for explanations... Best I've seen anywhere online. Superb.
@vanhell966 2 ай бұрын
Amazing work. Really appreciate you, making complex topics into simple language with the touch of anime and series. Amazing.
@marcosmartinez9241 3 жыл бұрын
These are the best serie of videos where I finally can find a good explanation about the Transformer network. Thanks a lot!!
@HeduAI 3 жыл бұрын
Cheers! 🙌
@McBobX 2 жыл бұрын
That is what I'm looking for, for 3 days now! Thanks a lot!
@minruihu Жыл бұрын
it is impressive, you explain so complicated topics in a vivid and easy way!!!
@aaryannakhat1842 3 жыл бұрын
Spectacular explanation! This channel is sooo underrated!
@gowthamkrishna6283 2 жыл бұрын
wow!! The best transformers series ever. Thanks a ton for making these
@henrylouis5143 2 жыл бұрын
Brilliant presentation, it's none other than the best I've seen. Great appreciation for your work!!! Cristal clear organization.
@HeduAI 2 жыл бұрын
Thanks Henry! Glad you liked it :)
@DeepakSadulla 2 жыл бұрын
The KZfaq example helps a lot with understanding QKVs. Really good explanation... Thanks!!
@hewas321 Жыл бұрын
No way. This video is insane!! The most accurate and excellent explanation of self-attention mechanism. Subscribed to your channel!
@markpadley890 3 жыл бұрын
Outstanding explanation and well delivered, both verbally and with the graphics. I look forward to the next in this series
@HeduAI 3 жыл бұрын
“Part 3 - Decoder’s Masked Attention” is out. Thanks for the wait. Enjoy! Cheers! :D kzfaq.info/get/bejne/nbBpnsR72KubaJ8.html
@nicholasabad8361 2 жыл бұрын
By fair the best explanation of Multi-Head Attention I've ever seen on KZfaq! Thanks!
@HeduAI 2 жыл бұрын
Glad to hear this :)
Hedu AI by Batool Haider
Рет қаралды 65 М.
Hedu AI by Batool Haider
Рет қаралды 130 М.
3Blue1Brown
Рет қаралды 2,7 МЛН
Layerwise Lectures
Рет қаралды 685 М.