Рет қаралды 6,160
#mamba #transformers #llm #mambaai
Select 1080p. The Transformer language model started to transform the AI industry, but it has one main problem that can make it go extinct even before it blasts off in 2024!
Watch this easy-to-follow and full-of-fun graphics video about the model architectures and performance differences of the Transformer and Mamba language models. I will compare the functionalities of the main AI and machine learning models, and show the necessary improvements in the Mamba AI model compared to its Recurrent Neural Networks predecessor such as Long Short Term Memory or LSTM, and Gated networks.
Stick around for more videos on LLM, Natural Language Processing (NLP), Generative AI, fun coding and machine learning projects, and follow Analytics Camp on Twitter (X): / analyticscamp
www.youtube.co...
Don’t forget to subscribe and watch these related videos:
Transformer Language Models Simplified in JUST 3 MINUTES!
• Transformer Language M...
Mamba Language Model Simplified In JUST 5 MINUTES!
• Mamba Language Model S...
This Is How EXACTLY Language Models Work In AI-- NO Background Needed:
• This is how EXACTLY La...
Backpropagation Simplified in JUST 2 MINUTES! --Neural Networks
• The Concept of Backpro...
www.youtube.co...
Video Timeline:
00:00 Intro
00:27 Transformer language model
00:52 Mamba language model
01:21 Main task of any large language model
01:49 Transformer LLM encoder and decoder units
02:19 Attention Is All You Need
02:45 Mamba model architecture
03:08 Challenges of sequence modelling
03:20 Recurrent-based models
03:48 The main problem of the Transformer model
04:09 Attention Mechanism's quadratic memory complexity
04:21 Sequence scaling
04:48 Differences between Mamba's Selective Mechanism and Transformers' Attention Mechanism
05:25 Selective State Space Models
05:47 Performance degradation
05:57 Linear scaling in sequence length
06:14 Linear activation function
06:33 Hardware-aware algorithm and GPU with High Bandwidth
06:36 Parallel Scan
06:48 Training data
07:09 Synthetic Tasks
07:21 Selective Copying
07:37 Induction Heads
07:58 Performance summary: Transformers VS Mamba
08:16 Longformer and Linformer
08:20 Reformer model
08:22 Transformer-XL
08:24 Sparse Transformers
08:25 Big Bird model
Related terms and concepts:
Tags:
#mamba
#mamba ai
#SSM
#SelectiveSSM
#TransformerLM
#llm
#languagemodel
#transformers
#ai
#generativeai
#neuralnetworks
#machinelearning
#chatgpt
#generativeai
#lstm
#rnn