I'm the author :) As self-taught ML practitioner, when I started studying deep learning, Yannic was my favorite teacher. I've learned so much from you. I'm so honored that Yannic featured my paper today. After watching your review, I made some changes to the paper and re-uploaded it to arXiv. I'd like to add some supplementary explanations to Yannic's comments: 1. Feedback loop vs. Recurrent As Yannic mentioned, it's essentially incorporating recurrent features into the Transformer architecture. RNNs were the first to implement feedback loop in a sequence-by-sequence manner, while the paper implements feedback using attention. The paper emphasize feedback as RNN means more like Markov process or autoregressive concept. The paper mentions that LSTMs and GRUs successfully implemented feedback loops within the RNN framework. 2. Stop Gradient and Gradient Checkpointing Sorry for bad english. I rewrote the part. The idea is that previous research suggested using stop gradient to improve computational performance. However, with gradient checkpointing, all computations need to be redone anyway. So, whether or not you use stop gradient, there's no noticeable impact on speed or memory usage. This is why I recommend eliminating the use of stop gradient. 3. 18:00 simpler algorithm Thank you for Yannic's compliment on Appendix C "Don't". In C.3, I addressed the simpler algorithm that Yannic suggested. Actually, this was the very first method I tried. Unfortunately, this algorithm failed to remember PassKey for long sequences, and the algorithm became a bit more convoluted, as Yannic mentioned.

At this point, Yannic is in desperate need of somebody, someone to not reinvent and instead create a new invention. Here, a back pat for you for your contribution to sharing academic papers with us.

I think in the next two to three years, the moment will finally come, when someone reinvents attention and calls it something like "self-aware in-context persistent memory", sprinkling it with some neuroscience mumbo-jumbo gibberish.

All paths leads to Schmidhuber

Thank you Yannic för making these papers understandable, and breaking them down with your insight and understanding. Very interesting. 🙏

RNN: "You cannot live with your own failure. Where did that lead you? Right back to me."

So I think you make a pretty convincing argument that this is a repackaged form of recurrent neural network. And yes you're right that a long time ago people were using these as neurally inspired architectures that weren't anywhere as successful as transformers. Now what I'm wondering is if they failed because they didn't have the transformer architecture underneath them which was more similar to organized long-term memory and learning. Maybe recurrent neural networks are almost useless by themselves, but built on top of transformers they provide the powerful equivalent of what we in Neuroscience call working memory and that the combined architecture, combined in this way can take things to the next level. I know that performance metrics can be gamed and can be very misleading but ultimately it doesn't matter if we're doing something similar to what we did in the past, if this particular arrangement leads to significant performance gains in certain kinds of tasks it still might be valuable even if it's recycled. A lot of technological progress occurs with the repurposing of old inventions in a new context. I apologize if this is a very off base comment I'm just really getting to learn this stuff and there's a lot of holes in my background because I'm coming at this from more of a neuroscience perspective. I can say that interrupting the cortico thalamic loop produces humans that might know a lot of stuff and even give appropriate responses, but leads to knowledgeable people that are just reactive entities and can't get anything done and lose track of where they are all the time. Those kinds of problems seem present in the current entities like chat GPT 4.0 and Claude opus which is what I have the most experience with.

So if the paper framed itself as, “because of neuroscience reasons, the way to improve transformers should be to combine them with an RNN (specifically, in a way like this)”, specifically describing it as a form of, or variation on, RNNs, would that have alleviated most of your issues with it?

I want models that can modify or add weights on the fly. I want it to have better long term memory without having to go back and read over the rags and hope it got what I want.

Your style is very entertaining. I laughed at the section : feature is not a bug. ~34 minutes into the video.

I feel like being heard by you! Thanks you!

guau great paper revision you enlight so well the key mechanism of the net, great job in scintific dissemination

It's interesting how for working memory they describe layer wise interaction this way (at least according to these authors). In flexible hippocampally dependent memory we see multiple recurrence across many system layers as well as layer-wise specialization in things like Pattern Separation (DG/CA3), Pattern Completion (CA3), and Relational Binding (CA1). We know the hippocampus is critical in humans for creative flexible reasoning. I feel like it's the brain region and memory system we're missing, not PFC mediated working memory...

The biological neural point of single/small group neurons in the loop diagram at 2:17 reminded me of how we remember the past and imagine the future, as well as the sci-fi movie ‘Inception’ (2010).

Funny that I thought of the same idea right after watching your last review on infini attention lol

Isn’t reprompting with a model’s own output just this, but at a higher level? (it’s effectively concatenating a state to a transformer, that came from a transformer…)

Hey Dr Kilcher, you should put your reputation where your snark is!😮😂 show us how you would architect your recurrent neural network to act as short-term memory. Snark aside, I really would like to see how that might work. Would you just prepend an RNN to a transformer?

Transformers use masking during training and are trained to predict the next token given previous tokens. I wonder if the memory mechanism/RNN node should also be trained to do the opposite: mask off the prefix and given the current tokens and the previous memory, predict the previous tokens.

The problem with the simple algorithm you proposed is when the generation reaches the block (or context) length in the middle of a sentence, then it is not so straightforward to generate the next token without the previous ones, while maintaining the sentence structure

Thanks for the explanation. Total aside ... I downloaded llama-3 and set the context window to 8k and it worked fine. I boosted it up to 9k and got pure gibberish. That was the first time using too big of a context size had that happen. The closest I've had before when I had an error for using too big of a context size my program simply crashed as I ran out of memory. Happily fine tuned(?) models have come out with longer context lengths. But I found it interesting. I obviously didn't understand what's going on underneath the hood to fully understand why, exactly, that happened... But videos like this give me a better foundation to make guesses.

This framework reminds me a [cls] token added at the end of each block. As shown in the following paper, adding a few of [cls] tokens, called registers, at the beginning improves performance. VISION TRANSFORMERS NEED REGISTERS, ICLR 2024.

What you are drwawing at 17:00, isn't that exactly what "Vision Transformers Need Registers" (DINOv2) proposed?

Well, I do think all those new papers have one thing in common: create "a hierarchy of memory" with increasing granularity and compute from less recent to most recent data. What's missing, I think, is not just going with hierarcy from linear to quadratic, but from linear to quadratic to *cubic*.

Well, I can instantly say they are onto something, but not sure if correct way. I've been experimenting with ANNs for 15 years. For the last 5 I've been working on a structure where the signal never dies.. I want the "loop", the amalgamation of the last states pulses through the net, meaning different things to different areas, preserving the "state" and working memory until the next input comes, so that the input is affected by the current "mood" of the net. Still strugling with this approach, but it is what is needed for real AI. In us, the signal never dies, it needs to be there to propegate more signal. I was blown away when people solved my difficult task in a simple ways with LLM, "just have many agents interacting with eachother". Yeah, together they then acchive what I want my single net to do, the signal never dying, but I want it in a single net.

I think I actually have thought about a similar idea of a handful of output tokens being paired with input tokens; and it sounds like that's what's happening here... And you say this is basically an RNN but for this kind of transformer system? Alright, fair enough. I often also propose something to pair with it to give the most versatility; have a sort of data-table akin to a notebook it can write to, some of its outputs are akin to coordinates on this table, some are "here's what to write to it" as well as one that simply determines whether or not to write or not, and another set of coordinates for what to read for the next loop of the neural network. Having this kind of "Notepad" as well as a means to make a short term memory by doing a direct loop from output straight to input, could allow it to better remember both long term and short term information at the same time. I imagine its probably simple enough for all this to be implemented in a single system, especially in a system where this "RNN" functionality as you've described it is already implemented.

can you do a video telling how you analyze papers , how to find the limitations of pares , something like that ? critical analysis in computer science papers , if that possible .?

Proposal at 17:00 seems natural, has it been done? Its just as parallelizable as a regular attention mechanism, isn't it?

It's closer to Block-Recurrent Transformers than transformersxl. TransformersXL reused output of previous layer rather than its own. Which makes sense for anything than Alberta where each layer is the same. Also the only reason xl stop backprop is publishing year. They simply had no resources to propogate that much. Also BRT authors were like "oh,yeah,we have vram now, so we let backprop through more CTX window" they called it "Slide" model. OK,after watching all video,its closer to RMT except they route memory to the same layer. I don't understand why they didn't went with RMT route and just computed them normally(appending memory) without second attn

This field bounces back and forth between pure engineering and cognitive sciences: first they were inspired by parallel-distributed processing and neuron-like units like the brain, then it was all about optimisation and infrastructure (cognitive-science was irrelevant), now they've realised they'll have to go back and get more inspiration for cognitive/biological sciences to achieve AGI. IMO feedback loops were inevitable from the start. How much impact will this paper really have?

Maybe the limited dimensions is a form of regularization?

To some degree I kind of get why you harp on this point a lot that this is just reinventing recurrent neural networks. And its really quite strange that nobody actually talks about these basically just being rnn architectures. But on the other hand these are at least novel in that they manage to combine the magic of transformer attention with the magic of rnns in a way that supposedly actually works well. Does it really matter if the "new" component they are introducing has been invented before separately? The only real benefit of seeing this connection is that you might be able to transfer some of the rnn experience over to these new hybrid architectures. Or is it just the annoyance that it seems like they haven't properly studied the older literature on ML and are kind of these transformer kiddies that claim having invented stuff based on ignorance? I am not trying to start a fight here! I am just curious where this frustration comes from.

What's up FAM?

I think it is outrageous when journals don't require the submission of the code for the acceptance of the algorithm. What if as part of the price for the publication, the journal offers cloud computational resources for anyone who wants to test the algorithm?

Why does it bother you so much that these papers are basically reprising RNNs? I mean yes, that's what they're doing, but they're doing it in different ways (there's countless variations on the RNN itself after all), so what's the problem? RNNs were always an intuitively good idea anyway, held back by vanishing gradient/info loss over thousands of tokens. All these papers, by "recurring" over big chunks rather than token by token, basically solve this. I think it's really exciting.

the shaved head makes the LLM's work better the power is building

Okay, we get it. Transformer need some sort of latent representation to be able to think coherently about huge chunks of data. Okay, expectable. But why are there four works, which imply it, but call it by different names?

Thank you Mr Yannik for your explanation of TransformerFAM: Feedback attention is working memory.

i thought attention was all i needed lol

Please start reviewing papers without backpropagation

just ask chat-gpt to write summary of the conversation after each answer and call it a day lol

woah

It's 23:24 where I live, please let me sleep XD

I get his frustration, but really IMO the era ot 'totally new' approaches is likely dead. We are now in the refinement stage, with low-level techniques being minor variants on each other and the innovations being tweaks on existing paradigms. this may change and there may be totally novel inventions down the pass, but i'd be unsurprised if nothing new comes for years to come. what's important is the performance. If the performance is there, it is well worth publishing and reviewing.

They're on the right track, except for the fact that it's all predicated on backprop/gradientdescent/automaticdifferentiation and thus totally incapable of online learning! It can only work with what it has been trained on.

Integral and differential terms?

this is roast + tutorial ... a Roastorial !!

My suspicion why so many people are reinventing RNNs is the lack of proper academic education and peer review process. People are just learning about transformers and the basic math and believe they have seen it all. Just a hunch, but nowadays everyone can upload stuff on arxiv without any quality control.

Lets put RNN back

I think there should be more work on identifying redundancy in the field, probably using AI itself

we need to focus on kan

No cap on a stack fr

Unrolling w backprop is a mistake - more local learning, more sums of geometric series.

If it isn’t a big deal, why are you reviewing it?

Another episode of Yannic explaining how a paper is just RNN 😆

I'm 7:48 minutes in and you've convinced me not to waste my time by continuing watching. I get your point, but if you want to increase viewer count you need to sell the paper better. 🙃

i am a strange cortical thalamic loop

Hello, I didn't finish the video yet, but I already wanna add that it's unfair to complain about people using different terms for neural architectures when they use a known concept at a different place in the architecture. Try to compare chip design. If you design a minimal chip that is turing complete all other things you'd design around it just seem distortions of it - yet a processor isn't just a minimal turing machine but has a lot of machines in it that are optimized in some way, like ALUs or FPUs etc. which you could use as a fundamental bulding block to implement a turing machine itself. My point is that what seems the same from a theoretical standpoint has absolutely different consequences in effect and makes it necessary to give it another name from an engineering point of view.

Could you please do a video on the Larimar Paper by IBM. Also has a bit of brain lingo in it. Think it could be used to get important dense context across inference(s) in a way that is almost semi-stateful instead of just of 'knowledge updates'? 🧐 Because if the next inference could be informed by the memory unit of important context, that might be helpful for Long Term Planning and more stateful, coherent responses, better stitching the inferences together so important context survives inf. ~Like makes me wonder if they could have further RLHFd it to use the memory unit to use context stored and further RLHF to refines it's use appropriately. ie learning to maintain important information relative to tasks and current goals across inf as needed. :x arxiv pdf/2403.11901 arxiv abs/2403.11901

Great show you truth and brutal honesty are incredibly refreshing your your noting of origins justifies Ori once again you're giving credit for work and participation is proper one of your best shows for the year even though the discoveries may not seem to be the greatest I thank you very much for bringing illumination and putting it on the table sincerely Michael Crosby CRYPTEWORLD

He don’t care

I like the attempt to suggest some biological inspiration, "working memory" but could it is just not that. would be nice to see something truly biological inspired.

wonder when they finally reinvent ART 🙄 (no not artistic pictures)