Developing a deep learning project (case study on transformer)

Рет қаралды 6,536

Күн бұрын

❤️ Become The AI Epiphany Patreon ❤️ ► / theaiepiphany
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
In this video, I talk about how I work on my deep learning projects on the example of the transformer which I've recently open-sourced.
You'll learn about:
✔️ Snippets of my personal story
✔️ How I think about approaching a new project
✔️ Problems I encountered developing the transformer
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ My transformer implementation: github.com/gordicaleksa/pytor...
✅ The Annotated Transformer blog: nlp.seas.harvard.edu/2018/04/0...
✅ Original paper: arxiv.org/abs/1706.03762
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
0:00 Open-sourcing original transformer and why
3:00 Creating great projects takes time
5:40 My time management
9:00 My story and embracing failures
12:05 Overview of the transformer project
14:03 Data and task definition
15:57 Training loop
20:10 Problems I encountered
21:44 Beam search fun
23:10 BucketIterator fun
25:35 Optimizing things to speed up the loop
26:25 Translating from English to German and vice versa
30:00 Hardware requirements
30:47 Wrapping things up
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany ► / theaiepiphany
One-time donation:
www.paypal.com/paypalme/theai...
Much love! ❤️
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► / aleksagordic
Twitter ► / gordic_aleksa
Instagram ► / aiepiphany
Facebook ► / aiepiphany
👨‍👩‍👧‍👦 JOIN OUR DISCORD COMMUNITY:
Discord ► / discord
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com/
💻 FOLLOW ME ON GITHUB FOR COOL PROJECTS:
GitHub ► github.com/gordicaleksa
📚 FOLLOW ME ON MEDIUM:
Medium ► / gordicaleksa
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#transformer #attention #deeplearning

Пікірлер: 41

@TheAIEpiphany 3 жыл бұрын

If you liked the first part of this video (which I've never done so far) and would like me to make more similar content please let me know in the comments what interests you in particular! Much love and keep learning at your own pace.

@soumyadrip 3 жыл бұрын

Would like to know the journey behind finally getting a job in a FAANG company specially in AI/ML domain and what are different rounds they conducted for hiring.

@pratikpatil1838 2 жыл бұрын

Hi sir,Can you make one video on text to SQL implementation using transformer or bert

@sergisanchezbonilla5436 7 ай бұрын

i just loved hearing your personal story. It encourages me to start learning through my carreer, since i like a lot the kind of things you do but i still feel like I'm a rookie.

@xxdxma6700 2 жыл бұрын

Thank you so much for sharing your story, it’s truly truly helpful. It’s always rlly easy to get panic and get lost when you see people on the internet making complicated stuff like super easy. I feel like a lot of the times it’s the mentality that decides how much you succeed instead of the absolute intelligence or productivity at that moment of time.

@simbaninja1235 3 жыл бұрын

Great video! I loved hearing about how you manage your time. Thank you for open-sourcing :)

@TheAIEpiphany 3 жыл бұрын

Thank you Simon! It wasn't as detailed as it could be but it does paint a rough picture. Glad you found it interesting

@tahmidhossain007 3 жыл бұрын

Great Work Aleksa, you are spot on. Baking good stuff takes time.

@TheAIEpiphany 3 жыл бұрын

Oh definitely! Thanks Tahmid!

@DevilOnTheDrums Жыл бұрын

This video gave me real motivation. Thank you.

@nikolavasic1947 3 жыл бұрын

Kralj si brate stvarno

@TheAIEpiphany 3 жыл бұрын

Kako se odgovara na to 😅 hvala? Haha salim se, hvala na podrsci!

@sasalukovic4717 3 жыл бұрын

Legendo, svaka cast

@TheAIEpiphany 3 жыл бұрын

Hvala!

@fastandbaki2163 3 жыл бұрын

Hvala na motovaciji!

@TheAIEpiphany 3 жыл бұрын

Juhu, drago mi je da ne deluje kontraproduktivno na ljude kao sto neki prazni motivacioni govori mogu da izazovu. 😅🧠

@bello3137 Жыл бұрын

A very great content aleksa, thank you so much for the information..

@leonardoberti917 Жыл бұрын

great explanation

@user-co6pu8zv3v 2 жыл бұрын

Great video! Thank you!

@damjandjoric4756 3 жыл бұрын

svaka cast, samo nastavi tako!

@TheAIEpiphany 3 жыл бұрын

Hvala puno!

@tusharrohilla7154 2 жыл бұрын

Its really helpful

@Frankthegravelrider Жыл бұрын

Great video

@quanhua92 3 жыл бұрын

subscribed! your channel's deserved for views and subscribers. Keep working!

@TheAIEpiphany 3 жыл бұрын

Thank you for the kind words! Oh I'm not stopping any time soon!

@TheAIEpiphany 3 жыл бұрын

Btw. I obviously also have bad days when it's harder to get focused and get things done. Flirting with burnouts since 94' basically haha. What the remedy is is to just take an evening off and do something that'll relax your brain. But not everything. Going outside helps a lot. Less information ingestion that's important. Also I forgot to mention that every second day I do some workout in the evening that helps a lot.

@UknownCompiler 3 жыл бұрын

your videos are interesting and in-depth, thank you for sharing the valuable infos i need your help please, currently i'm working on a translation transformer from english to arabic, spacy does not support arabic language, so i had to do tokenizing with nltk and vocab and indexing from scratch no torchtext help, i did everything except batching so i decided to input one sentence by one along one dimension to a nn.transformer model, does the built-in nn.transformer of pytorch take only batches or does it take one by one samples? if it doesn't take seq by seq then should i write transformer from scratch and adjust the dimensions? thank you so much

@o_felipe_reis 3 жыл бұрын

Hi there! Great video! Thanks for sharing and regards from Brazil 🇧🇷🇧🇷🇧🇷

@TheAIEpiphany 3 жыл бұрын

Obrigado amigo! Tudo de bom pra cê!

@o_felipe_reis 3 жыл бұрын

@@TheAIEpiphany valeuuuuuu

@TheAIEpiphany 3 жыл бұрын

@@o_felipe_reis kkkkk, eu morava no Brazil no Ouro Preto, fiz uma pratica la!

@o_felipe_reis 3 жыл бұрын

@@TheAIEpiphany ah que legal! ótima cidade! De onde você é? e quanto tempo passou aqui?

@TheAIEpiphany 3 жыл бұрын

@@o_felipe_reis Sim e o pessoal é incrivel, sinto muito saudades do Brazil. Sou da Servia! Estava so 1.5 meses la porque recebi uma oferta de emprego do Microsoft. Mas morava numa republica (Badalação) e por isso conheci muito mas do Brazil.