Visual Guide to Transformer Neural Networks - (Episode 3) Decoder’s Masked Attention

Рет қаралды 65,179

Күн бұрын

Visual Guide to Transformer Neural Networks (Series) - Step by Step Intuitive Explanation
Episode 0 - [OPTIONAL] The Neuroscience of "Attention"
• The Neuroscience of “A...
Episode 1 - Position Embeddings
• Visual Guide to Transf...
Episode 2 - Multi-Head & Self-Attention
• Visual Guide to Transf...
Episode 3 - Decoder’s Masked Attention
• Visual Guide to Transf...
This video series explains the math, as well as the intuition behind the Transformer Neural Networks that were first introduced by the “Attention is All You Need” paper.
--------------------------------------------------------------
References and Other Great Resources
--------------------------------------------------------------
Attention is All You Need
arxiv.org/abs/1706.03762
Jay Alammar - The Illustrated Transformer
jalammar.github.io/illustrated...
The A.I Hacker - Illustrated Guide to Transformers Neural Networks: A step by step explanation
jalammar.github.io/illustrated...
Amirhoussein Kazemnejad Blog Post - Transformer Architecture: The Positional Encoding
kazemnejad.com/blog/transform...
Yannic Kilcher KZfaq Video - Attention is All You Need
www.youtube.com/watch?v=iDulh...

Пікірлер: 308

@HeduAI 3 жыл бұрын

Correction in the Video: A huge shout-out to @jackzead "What comes out of the encoder should be the key and value matrix and what comes out of the bottom part of the decoder is the query matrix." *TIMESTAMPS* Here are the timestamps associated with the concepts covered in this video: 0:00 - Recaps of Parts 1, 2 0:52 - Residual Connections 4:25 - Transformer Decoder 9:52 - Decoder's Masked Attention

@muhammadsaadmansoor7777 3 жыл бұрын

I was wondering if you would correct this. Thanks.

@HeduAI 3 жыл бұрын

@@muhammadsaadmansoor7777 I tried but was not able to find a way to add notes to a video which has been published. Apparently, KZfaq has a few tools that get enabled to the channel owners with a certain number of subscribers only. Let me know if you know how to add notes to the video.

@ivanmatas3884 3 жыл бұрын

@@HeduAI You can pin this comment at the top. Excellent video, congratulations.

@HeduAI 3 жыл бұрын

@@ivanmatas3884 Thanks :) This comment is already pinned to the top.

@torgoron9695 3 жыл бұрын

@@HeduAI I think it's not anymore pinned to the top - don't know how this works though. Maybe you can also add to the video description that there's this error or the correction. Thanks a lot for this extremely awesome content! Saved me loads of time and was fun to watch at the same time!

@LeManneCris 3 жыл бұрын

These explanations are so good that they should be included into Tensorflow's and PyTorch's documentations, seriously !

@HeduAI 3 жыл бұрын

This made my day :) Thanks!

@koustubhavachat 3 жыл бұрын

True

2 жыл бұрын

This series is a gem. Really clear explanation, much easier to get back to the paper and the math with such overview. Huge kudos!

@dy8576 Жыл бұрын

This series deserves so much more "attention"

@alsonyang230 Жыл бұрын

I have tried to learn transformer from various sources with several attempts in the past a couple of months, but I couldn't get the gist of it. I wish I have spotted your 3 episodes earlier, they were extremely helpful and easy to follow. I really appreciate the time, effort, love and joke you put into your videos, and sincerely wish you could keep making these types of videos on DL topics. I love the fact that each concept/technique is well explained with a concrete example as well as why it was done this way not the others. You are truely gifted at explaining complicated concept for people with limited DL background. Please keep up the good work

@ahp-6785 2 ай бұрын

You are the mother of StatQuest and 3Blue1Brown. Both of these guys are awesome in explaining complex ideas in simple words. But you are the best.

@ninjahunterx7497 2 ай бұрын

I don't know about StatQuest (haven't seen his ones) and 3Blue1Brown is good because of the visualization he brings with his advanced animations. But honestly, here she explained all these concepts using simple animations and had a good structure throughout the videos, each connecting well to the other. Very commendable if you ask me.

@pratikpratik8495 3 жыл бұрын

Please make series of NLP , Deep Learning. Your teaching style is amazing.

@HeduAI 3 жыл бұрын

Thank you! I am glad you enjoyed it!

@ankoor 3 жыл бұрын

You have an amazing gift of explaining complex things in an intuitive way!

@HeduAI 3 жыл бұрын

Thank you! Glad you found it useful :)

@user-rr7xc9ih3r 11 ай бұрын

it hurts that ur skills are so under-rated, the ai community needs more of you!

@royeyono6512 2 жыл бұрын

You are truly truly gifted in explaining very technical topics in an intuitive way! I often get embarrassed to go to KZfaq to learn about DL concepts considering that I have studied the topic for a couple years now, but in all honesty, the scientific community desperately needs people like you. Thank you for investing your time in making the world a better place!

@HeduAI 2 жыл бұрын

Roy, you truly made my day. Thank you so much!

@azerioauditore511 3 жыл бұрын

Her way of explaining is so crystal clear. Its very evident she is a genius

@HeduAI 3 жыл бұрын

When Azerio from the Assassin's Creeds calls you a genius, you've gotta agree :D Thanks!

@rajarshideeplearning6500 2 жыл бұрын

Literally THE best 3 video playlist 👌👌

@SaiGopalCAP8978 Жыл бұрын

You are my savior. Went through a bunch of videos yesterday, didn't understand shit.. finally found these awesome short 3 videos today.

@user-bx7ho4dz1m 2 жыл бұрын

Your three episodes are the clearest and best explanation to Transformer that I've ever come across. They help me a lot and I thank you for that. I can see that you got talent to explain things complicated in a very simplistic way. Looking forward to watching more of them.

@bendarodes61 2 жыл бұрын

I've watched many videos about transformers, this is by far the best.

@maysammansor Ай бұрын

Batool you are really a great teacher. Thanks for the content. Please provide more vid on LLMS and AI

@whitelady1063 Жыл бұрын

havn't seen such a good series of vids in a realy long time

@weihongchin8140 Жыл бұрын

this is my 3rd comments for my entire life using KZfaq! The best video for explaining transformer! Hope you will make more videos in future!

@jackziad 3 жыл бұрын

There might be a mistake in the video. What comes out of the encoder should be the key and value matrix and what comes out of the bottom part of the decoder is the query matrix. No? The query is the matrix that is looking for the right key and value pairs. The rest is very good. I always learn something new in your videos.

@HeduAI 3 жыл бұрын

Hmm... you maybe right. Let me confirm this and get back in a few days. If it is indeed a mistake, I will try my best to inform future viewers about it by either editing the video or pinning the edit comment to the top. Thanks for the critical feedback!

@nivu 3 жыл бұрын

The encoder output is used as the Query and Key input to the 2nd Multi-Headed Self Attention (also known as Encoder-Decoder Attention) of the Decoder Block. And the Value comes from the 1st Self-Attention Block of the Decoder. The explanation in the video is correct.

@jackziad 3 жыл бұрын

@@nivu Well, I have seen Jay Alammar's explanation and I have also seen 2 code implementations of Transformer and all 3 references explain and show that the encoder outputs the key and value matrices and the 1st self-attention block of the decoder outputs the query. Again this makes sense since Key and Value matrices always come in pairs (a key is a kind of an index into the value, they are supposed to be together). The query is the question that asks which key is the most relevant and so it makes sense it comes from the decoder since the decoder knows the ground truth. In case of language translation from english to german, the encoder would study the english langage, the 1st self-attention block of the decoder would study the german language, and the 2nd attention block of the decoder would study the interaction between the 2.

@jackziad 3 жыл бұрын

This is what Jay Alammar says about this point: "The “Encoder-Decoder Attention” layer works just like multiheaded self-attention, except it creates its Queries matrix from the layer below it, and takes the Keys and Values matrix from the output of the encoder stack."

@HeduAI 3 жыл бұрын

@@jackziad Thanks! Have added the corrective comment and pinned it to the top, accrediting you. Thanks again for pointing this out.

@selimcanbayrak3278 2 жыл бұрын

It is amazing how you explain these complicated models in a way that is so simple to understand. It reminds me that Einstein said "If you can’t explain it simply, you don’t understand it well enough."

@munugotiswapna8357 10 ай бұрын

Your explanation is too good..I am able to understand these transformers because of the way explained

@jb_kc__ 5 ай бұрын

Super clear explanations. Really appreciate you putting this series together! (and love the pop culture references)

@prajwalanagani5590 3 жыл бұрын

This is a great playlist and one of the best I have come across for transformers. Thanks! And looking forward to your next set of videos.

@lassadsmaoui2929 Жыл бұрын

I passed a long time searchig a good explanation untill i find these series . awesome !! keep going 😍😍😍

@ushas3349 Жыл бұрын

The most difficult topic "Transformer Network" is explained very clearly with visual guide. Hats off to the speaker. I appreciate the efforts taken to prepare this guided tour.

@HeduAI Жыл бұрын

Thanks a ton Usha! :)

@benak495 2 жыл бұрын

Thank you for all your videos about transformers. They are incredible. It's obvious you've worked hard on them, and they've been really instructive.

@gesitsinggih Жыл бұрын

Your video is worth more than 1000 page book. Your 3 videos allow me to understand and implement from scratch Transformer architecture. Any gap in explanation forces me to think for myself, deepening the understanding. Thank you.

@ruantwice 2 жыл бұрын

This video series is the best one I have found about transformers! Seriously, a job well done! Thank you for your effort!

@vahanhovhannisyan7603 3 жыл бұрын

This was such an awesome explanation! Thank you very much!

@mbunooo 3 жыл бұрын

this series is AWESOME and this channel should have more suscribers!!!

@ruslan124 Жыл бұрын

I don't know who you are but I do know that this series was the best explanation of Transformers I have seen or read and I have seen and read a lot. Thank you.

@fritstuininga8159 10 ай бұрын

These explaination videos are so good. It would be a crime not to subscribe. You earned yourself a sub today!

@norchiegaming 2 жыл бұрын

Please make more videos explaining stuff intuitively! These are the best videos I've found on the topic and helped tremendously

@jackskellingtron 3 жыл бұрын

Fantastic. Just finished the series without stopping. "Layer normalization simply means standardizing neuron activations along axis of the features" I've seen several descriptions of layer normalization, and they are usually paragraphs long. Yours is one sentence long and somehow 100x clearer. Animations are perfect as well. Yer a wizard 'edu

@zhitaochen5665 Жыл бұрын

You explain the difference between training and inference so well. This video deserves so many more views.

@aaryannakhat1842 3 жыл бұрын

Stupendous! Thank you for this intuitive explanation, I was really struggling to get my head around Transformers.

@justdo_it3699 Жыл бұрын

wow... this video give me perfect understand about transformer. Your teaching style is amazing, perfect, no question left. plz make more video. I'm very thanks for your effort

@user-ej6uj5rf2q 5 ай бұрын

Best video series ever! Thanks is not enough.

@martinsoderstrom449 2 жыл бұрын

Thank you! This is the best guide I have seen on here. :)

@baskaisimkalmamisti Жыл бұрын

Truly it is the clearest explanation of the transformer architecture. Please keep posting that kind of great explanations!

@stevehe5713 Жыл бұрын

Amazing videos! The best out of all the transformer videos ever watched. Thank you!

@AnuRag-fo2ts Жыл бұрын

Wonderfully explained. Looking forward for more

@paweosek5009 Жыл бұрын

I was trying to understand it for quite a long time, and for the first time I don’t feel like a child lost in the woods 😂 you really have a gift!

@HeduAI Жыл бұрын

This has got to be the sweetest comment :)

@pollopallo7401 2 жыл бұрын

You probably saved me for tomorrow exam, thank you so much!

@_divya_shakti 3 жыл бұрын

finally wait is over , thankyou maam

@jonasmanuel 2 жыл бұрын

This was an amazing mini series. Well done!

@yusun5722 2 жыл бұрын

The best detailed explanation of the transformer. Hope to see more of this type videos.

@lalluyoutub Жыл бұрын

Finally & finally, I understood Transformers!!! Best playlist that I found on Transformers!!! Thank you @Hedu AI. Hope to see you make more videos.

@picassojeus6114 Жыл бұрын

You are a ROCKSTAR 💚💚💚. So lucidly explained... Shows your eloquence. Glory to you.

@yassine20909 Жыл бұрын

You got the magic, of explaining very complicated material and making it so obvious, within you. Thank you for the series on the attention model, you did a Great job. 👍

@EnderKira73PC 2 жыл бұрын

Please, go on making videos. This is the best content I've found on Transformers

@alexandrbyzov3581 Жыл бұрын

This is an extraordinary good explanation! Thank you very much! Instant subscribe!

@yiweijia6922 2 жыл бұрын

Thank you! It's a really incredible tutorial. Everything is explained in detail and intuitively!! Love your teaching style and it's really worth a subscription. :)

@mydigitalwayia956 2 жыл бұрын

Nunca había visto una explicacion tan excelente como la suya. Muchas felicitaciones. Por favor publique mas videos.

@HarshitSingh-xx8ey Жыл бұрын

These videos were sooo good. You should continue making more

@alankarshukla4385 Жыл бұрын

This is JUST EPIC!! This is the best explanation out there!! I looked everywhere but always seems something is missing. Thank you so much for making this great Video.

@arjuntheprogrammer 2 жыл бұрын

One of the best explantion process I have ever seen

@rahilmehrizi6692 3 жыл бұрын

This is an awesome video, thank you very much!

@goodlifenepal 2 жыл бұрын

You have created so great learning asset. Hats off to you.

@kamalamarepalli1165 Жыл бұрын

Very very wonderful content, voice, explanation, animations...so good...thank you so much.

@brandonkim4675 Жыл бұрын

Wow, you absolutely killed it! Your explanation is mind-blowingly easy to follow for even one who does not have any background in AI, like me. I have watched a lot of videos on this topic ever since ChatGPT hit the headline, but none of them managed to make me feel like I understood anything. No video has come close to the way you broke down the process step by step. Your use of visual aids was simply fantastic and made the whole concept so much clearer. Thank you soooooo much.

@salonikothari7494 3 жыл бұрын

i have never heard things being explained so clearly and articulated with all key details as much as possible ! thank you so much .... > :)

@HeduAI 3 жыл бұрын

Really glad to hear this :)

@SathvikKullolli Жыл бұрын

Amazed by the way of explaination!!.Most intuitive and brilliant articulation of one of the hardest concepts!!A huge THANK YOU!!!

@ravirajm 2 жыл бұрын

This is the easiest to understand explanation of Transformers with amazing graphics. I would pay for this content :)

@mohitmeena541 Жыл бұрын

This is epic content on youtube. Please keep uploading!

@amparoconsuelo9451 9 ай бұрын

Now I understand the complex computations in AI coding.

@yashrajverma8412 Жыл бұрын

Best explanation ever, Please continue to upload videos about new research papers in the area. Keep up the good work!

@hacspidartist6645 Жыл бұрын

tysm ~~ this series really is the masterpiece

@bonadio60 2 жыл бұрын

Fantastic explanation!! Thank you

@ahmadsawalhah9620 3 жыл бұрын

Really I wonder how could you do that, simple , clear , straightforward, beautiful ..... thank you very much,,,, Keep going

@martinhlosta8249 3 жыл бұрын

This was the best explanation of Transformers and Self-Attention I have come across so far! Thanks a lot for this series of videos!

@HeduAI 3 жыл бұрын

You are very welcome! :)

@HiggsPatch 8 ай бұрын

Nicely done. Very clear and succinct. Loved the narration, and the acting parts from Game of Thrones. Huge thank you for making me understand Transformers!!🙏

@shadabalam2122 8 ай бұрын

The best transformer explanation 👏👏

@jyuan251 2 жыл бұрын

These explanations are amazing!

@esn832 Жыл бұрын

You should get a Nobel Prize for this video series! Thank you very much!

@HeduAI Жыл бұрын

Hahah! :D I gladly accept the Nobel Prize from you :)

@aayushjariwala6256 Жыл бұрын

Some video includes concept only and other includes maths. But your way to teach is just incredible. You haven't just included maths and concepts but also examples to show each detail without skipping in very less time.

@HeduAI Жыл бұрын

Your comment made my day :)

@csescholar8890 Жыл бұрын

mind blowing explanation i have seen ever

@Yogesh-rg1if 10 ай бұрын

Thank you very much. Great explanation. 👌

@noahkasmanoff6366 3 жыл бұрын

Very clear and helpful! Thank you!

@sharjeel_mazhar 3 ай бұрын

You have ny utmost respect, ma'am!

@csescholar8890 Жыл бұрын

its mind blowing explanation i have seen ever

@arulyashwanth5314 2 жыл бұрын

Amazing explanation🔥🔥 Thank you

@ollipringle 3 жыл бұрын

Best explanation on transformers I've found

@Zixtys 5 ай бұрын

What a phenomenal video.

@scharlesworth93 Жыл бұрын

This is well done, subscribed.

@saitrinathdubba 2 жыл бұрын

This is brilliant!! Thank you very much :)

@dr.bhavanabhamare7452 3 жыл бұрын

Really u have amazing skill to explain complex things in an easy way

@abilashnair1745 2 жыл бұрын

Best teacher in the world

@dewa_1912 Жыл бұрын

Thank you so much. May god bless you.

@markpadley890 3 жыл бұрын

Notwithstanding jackzead's observation, another very good video. My understanding of this very important NLP architecture is way, way better as a direct result of this entertaining video series - time for celebration indeed - well done!!

@HeduAI 3 жыл бұрын

It appears the hard work put into creating this video paid off! :)

@saifuddinnabilmohammedzain9608 Жыл бұрын

More videos please, if this channel continues at the same level, it will become one of the most important references in the world

@HeduAI Жыл бұрын

Wow, this brightened my day :)

@mehmetakifozkanoglu2259 3 жыл бұрын

The best explanation that I have ever watched, thank you so much

@HeduAI 3 жыл бұрын

You are very welcome! :)

@chaitanyachhibba255 3 жыл бұрын

these are the best videos available, I hope millions watch this. I have been through many videos in the past week and I wish I would have seen this sooner. Thanks lot, Please keep on posting more videos..

@HeduAI 3 жыл бұрын

So glad to hear this!

@saruaralam2723 3 жыл бұрын

Your all videos are best I came across (explaining with matrices, example, etc). Kindly make more contents.

@ankurdas1477 Жыл бұрын

Great Series!

@isaaczheng2769 Жыл бұрын

This serie is so good! PLEASE make more AI tutorials

@SandeepSinghPlus 3 жыл бұрын

I just coded whole Transformer toy implementation with help of these video. Kudos to you!

@HeduAI 3 жыл бұрын

Sounds great!!

@rajv4509 Жыл бұрын

If I said thank you a million times that would not be enough!!! You have simplified a complex topic in such a way that anyone can understand it. Keep up the good work! The world of AI needs more people like you :)

@HeduAI Жыл бұрын

Thank you so much for your million thank you's :) Should be enough fuel to keep me inspired for the next several videos :)