This reminds me of when I was playing with play-doh when I was young. You start out with many different colors, and somehow always end up with a big brown ball.

"The more it eats its own output, the less variety the output has"... sounds exactly like the KZfaq recommendations algorithm.

I remember as a kid I used to record my voice and play it back on my speakers, which I then proceeded to record once more. By repeating it, I could hear how it slowly degraded until it was nothing more than a weird, synth-like sound.

My biggest problem with AI is that it needs to get the information from somewhere, and sometimes these sources can be slightly dodgy. I did an experiment where I asked ChatGPT about some very narrow subjects: The Danish organplayer Peter Erling, the trio Klyderne, and the artist Jørgen Fonemy. These are subjects that I have some knowledge about and actually have written Wikipedia articles about. I could see that most of the answer that I got from ChatGPT was based on the exact Wikipedia articles that I wrote! I have tried to write the truth in those articles, but if I didn't care if things were correct - or worse; if I deliberately wanted to mislead people, then AI would base the answers on wrong data, if there wasn't multiple sources available. The problem I see with AI is that we trust it too much. Already now there are people who believe that it is an omniscient trustworthy source of all answers and that it will always be more correct than human knowledge or just knowledge that we have googled or looked up in an oldfashioned book.

3rd possibility: AI learns how to gaslight us, and we forget how many legs elephants have.

This could lead to a .. Nightmare on LLM Street....

That plastic analogy is probably the most succinct depiction of AI generated content contaminating the environment and why I always thought that human intervention in the use of computers is always necessary. We can fake human thinking to a degree, but getting the full complexity is still a pipe dream.

I love that this underscores how complex human intelligence really is.

Apparently in the original Matrix movies storyline, the reason why the machines needed to keep those troublesome humans around was not as an energy source (“batteries”) but as a source of creativity. But the writers thought that this idea was too complex so they substituted the battery idea instead.

As someone who used to play with photocopiers as a kid... A copy of a copy of a copy is always much worse and weirder than you might think. Small flaws amplify until you all you get is a smudged blur.

I recognized this as a possible problem when I learned that they were training AI by allowing them to converse with people on Reddit. AI developers can now apparently pay a fee to be allowed to plug their AI into Reddit and have it learn by having conversations there. It occurred to me that, "wait a minute, wont then the AI's end up conversing with each other and training each other??? Won't this cause problems?"

Sabine, as a professional in the application of machine learning in medicine I would like to thank you for making this video! It’s understandable and it reaches a lot of people! There is the AI hype (which people should not believe because it comes from executives and rookies) and there is the machine learning reality that veterans understand. This technology will be useful in automating some drudgery and common simple tasks…. It’s dogshit at doing anything truly valuable. What’s most worrying is the very real threat, without laws, that this nonsense will create such a firehouse of bullshit that we can’t get through our email, find what we need on the web, tell the difference between fact and fiction, and generally think for ourselves!

A friend in the UK is a graphic designer; he says that over the past few months, more and more clients have been saying 'NO!' to AI-generated artwork - "it's too samey". They'd rather pay more for something original. Trouble is, AI has pushed down the rates; so while designers and artists are noting an uptick in requests for proposals, the money is much worse.

This is exactly what I was telling people the other day. Our greatest danger with AI isn't that it'll take over but that at the moment we begin relying on it most, the more it will collapse because it's going to end up cannibalizing itself.

This is already true for me when writing. In the first few rounds it appears as if the AI is very creative but soon after some repetitions it becomes clear that the AI keeps repeating itself over and over again. This is one of the reasons I am not concerned too much. It appears that the human still has to input A LOT of guidance to make sure it doesn't repeat itself and actually gives you more interesting "mixes" instead of repetition.

Glad you started this conversation. There is also the theft component of generative AI. A KZfaqr like yourself will get automatically copyright struck for using 4 seconds of a clip in a 20 minute original video. Yet these generative AI companies can use entire social media platforms with content painstakingly created by individuals across decades to create their data sets. This is peak hypocrisy in which, as per usual, corporate "big money" is protected while the individual is left with no means of defending their content. Generative AI is 100% theft in my opinion.

It's true, when you've worked enough with ChatGPT you can immediately recognize a ChatGPT text. It just always has a certain vibe that makes it distinguishable from human text.

My experience with chat GPT is that you can ask it 2 or 3 questions, get it to contradict itself, and when you point out the contradiction, it starts to ask if you are angry, and/or says IT'S taking a break from YOU to let you relax...😊😮😂

to me the scariest thing is how quickly people advocated against themselves the moment they realized the potential with AI

Short, to the point, informative. Thank you, Sabine.

Garbage In/Garbage Out. I've been saying the same thing about both AI and Analytics for the past decade and a half. People only want to look at processes, algorithms, ease of use, speediness, raw power, TCO, design and pretty UI with both AI and Analytics. You rarely hear people talk about things like bias, data integrity and context. Those three things only come into conversation when AI and Analytics produce horribly incorrect results.

You reminded me of Pandora, the music recommendator that provided you with music according to the 👍 and 👎 that you gave to the songs proposed. No matter if you started with Black Sabbath, Chopin or Yunchen Lahmo. Eventually, after a couple dozen songs, you always ended up in a Coldplay loop.

It's kinda like Google search engine. It started out sucking, then there was a time it was pretty good to find stuff. Now it sucks again....

A friend of mine who has been a musician since the late 1950's explained tp me a few years back how there is very little originality in music these days as much is dependent on preset rhythms , chords , etc much due to "electronic " devices . It can only get worse with so called AI

You make a good point. Already I can usually pick the ‘style’ of AI generated images. They have a certain ‘style’ because they are in a sense too perfect, too smooth, too balanced. It is not something one could define in some cases, but the human brain is good at recognising patterns.

There's a third option: there may be soon a deliberate attempt to poison the content to make it unreadable for AI. There are already tools out there that scramble images just enough to make them confusing for AI to use as a training set.

This actually is not that suprising, when you think about it: these AIs are basically using huge amounts of data to approximate averages of various things, and with more iterations they extract more and more core features until they just have the same set of features they are using all the time. It's like taking data scores and continually averaging them until you are left with one value.

Thank you for clearly articulating what many have been trying to point out.

As an artist, this is a known issue in HUMANS Thats why the art solution is to look at the "old masters" as mentors before learning how to draw from more modern artists

1) AI is trained on data from the Internet. 2) AI outputs data to the Internet. 3) Goto 1 ... haven't anybody acquainted themselves with the topic of "inbreeding" ?

That’s really interesting.. because comparing it to ‘bad cinema’, most bad cinema is bad in the same way, if that makes sense. Overused tropes, predictable storylines, cliche characterisation. Is there someone that can expand on this thought?

I really like this take. Thank you for all the information.

Adding random variation probably isn't as easy as it may sound because the randomness still has to follow certain rules. For example, no one is going to believe that elephant with two trunks.

My experiance with ChatGPT shows it to be a regurgitator, the test questions were in an area of X-ray physics that I know well and it spewed out all the usual stuff with no insight, no deep understanding, no creativity, nothing that would indicate any form of curiosity.

Doctor Frankenstein used snippets from a whole bunch of people to make his monster. I was told the experiment didn't turn out well for him either. 😂

I was using an AI image generator to come up with rug designs. The first few were great but it soon deteriorated to regurgitated images similar to what it already produced.

This is exactly what I've been thinking since all of this exploded into popular awareness. It's like a giant ouroboros eating it's own tail. I'm glad to see that people are talking about this. Editing to add: Will you critics please lighten up? I'm not anti-AI. I'm just agreeing with Sabine that this is a potential problem that should be studied. All new technologies have potential problems that need to be studied and understood. Pointing this out does not make me some kind of neo-luddite.

So basically, if you keep feeding the output back into the input, you could get a feedback loop.

Makes me think of the 3 body problem issue/chaos theory. The more variables, the faster chaos erupts.

Than you for covering this! I've been saying this since I saw the 1st interest in AI art

I think a big part of the convergence is that people often are attracted to certain tropes and conventions when it comes to what they like, so AI produced images are actively being 'pruned and purified' by our preference of our existing cultural paradigms. What I think is really interesting is the feedback, where people's tastes of which conventions they like are in turn influenced by AI art.

As an AI scientist, we've been talking about this for years. Once the AI starts eating its own tail it will quickly optimize to a singularity of stupidity in its own echo chamber. The only way for AI to continue to work is to automatically label all AI output and ignore it for training. Or to manually post label it by humans. Humans are necessary for AI success in any case. It would be interesting for you to discuss both the labeling servant culture and its injustices as well as the impossibility of AGI if AI depends on human labeling.

So basically ai need a human brain. Take note matrix.

Great video. This problem may be a teething problem, though. After the computers started beating the top human Chess players, Go players like myself felt smug. We said things like "Chess is about crunching through possibilities. Go requires real intelligence." For years we annoyed chess players with this sort of talk, citing the inability of computers to beat the top human Go players as proof. This lasted about 18 years until AlphaGo came along in 2016. Important point: AlphaGo does not play like humans, except faster and more accurately. It came up with some genuinely novel moves. So, I would not take any problems that AI is now experiencing as an accurate predictor of what AI will be like in another two decades or so.

Wow... this is like ultra-high-speed "Groupthink"

Every email in the future will start with: "i hope this email finds you well" 😂

Very good video! It makes total sense!

Indeed, thank you. Chris

The one thing people should know about machine learning is: a machine learning trained model will only be good as its training data. It's just learning (in theory) the pattern behind the data leading to a host of problems. The main issue is that it doesn't actually reason about the data. Let's say I train a model where I have several examples where I have pi as 3.14 and then one where it's 4. The model doesn't reason "you know.... this one example seems to be wrong" but rather it updates the model to make it slightly more likely it will give the wrong answer. So how do you prevent models training on information generated by another machine learning model? The current approach is to stick to information before generative AI become dominate but most of that information (for better or worse) is probably considered or part of the training dataset. The main problem is that there's a popular opinion in machine learning (and sadly AI) that, as an AI researcher, I have had to deal with. This opinion is the key to all AI problems is that we just need to use larger models, with more training data, and train it in the "correct" way. "Look how far LLMs have come. Just imagine how much better they will be in a couple years". But you run into the 90-10 principle: 10% of the effort for 90% of the results and vice versa. It's why self-driving cars are taking a long time: there is nearly an infinite extremely rare cases that the car needs to make the right decision in. As such, it should be expected for the current LLMs to plateau performance wise unless new smarter methods are found.

2) More randomnes in AI output might do away with the problem of repetitive AI output, but it might increase the mistakes. Instead of elephants with big heads or two heads, we might get elephants with two big heads.

In my industry gives vague answers when you ask about laws. It keeps saying that things vary. They don’t vary, there’s just more than two answers. That’s not the same as varying. Some customers go under option A and others under option B. You can’t just tell them it varies

After the autotune-as-an-effect started, with Cher singing "Believe", you have multiple generations of singers who mimic the sound of autotune artifacts quite naturally and spontaneously. Also, young people having plastic surgery to mimic the look of phone camera filters. We are also trainable, it turns out. I think that is an important element to consider when predicting the future of this stuff.

Look how people are more and more hating CGI in movies to the point that some movies refuse to do any. If you have ever read anything written by AI you know it has the ability to make the most exciting subjects boring.

Consider what generative AI actually is. It's designed to produce data which most closely resembles its training data. So distinctive and idiosyncratic ideas are actively selected against.

Real creativity requires a mind, which is a spiritual/metaphysical component beyond the physical body (brain). Until we learn to understand the metaphysical aspect of creation and humanity, we won't be able to build machines that are truly creative.

I ♥ all your videos thanks for sharing your knowledge.

Great presentation, Sabine. I have always maintained that AI is like students cribbing exam answers. One student just has to copy one thing wrong, once. From then on it is a done disaster. When scientists hypothesised robots making copies of themselves - they never saw this far into the mess.

The popular image generation models prior to Stable diffusion were GANs (generative adversarial networks). The way they worked was to have two different networks - one trained to generate images, and the other trained to classify images as real or fake. This forced the generator to learn to avoid the most identifiable characteristics and to generate a diverse set of images. Stable diffusion was more effective and scaleable for higher resolution images, keeping the whole image globally coherent. But it's likely that reviving some adversarial techniques could help with the diversity issue.

It seems like it's only a matter of scale and fixing those issues with current models.

Worst case scenario, it'll be like Walmart moving in to a community, destroying the mainstreet businesses, everyone takes Wal-jobs, and then the Walmart closes because the local economy is crap simply leaving a ghost town with people who don't have the capital, business acumen or consumer base to reboot mainstreet.

Surprised we arent already forcing watermarks on ai content. Actually blown away. Like giving a kid a staw house and fireworks and not expecting a fire😊

I kind of wonder if this problem actually started with the widespread use of the internet. We went from communicating with books, which had to meet a certain standard before the expense of publishing could be justified, to zero-cost sharing of opinions on the internet, to having machines lap up these opinions and feed them back to us. Each of the above steps involving less rigour than that which precedes it.

I’m not surprised by the finding and I dare say that it is obvious. I dare say that because such a thought has crossed my mind and I have often tried to convince others who are willing to listen beyond the hype; AI is “artificial” but not “intelligent”. There are different ways to assess and critique AI - philosophically, linguistically, psychologically, biologically, etc, which many thoughtful experts, beyond the industry, have challenged the claims of AI, in particular AGI - but, for me, it comes down to creativity and novelty, the latter AI lacks completely and the former AI can only mimic. If you want to be impressed, see a human child.

This is probably a reflection of the filtering used to keep AI presentable to the public. Filtering limits the bandwidth of the input data and successive runs with a filter would narrow the scope even further. It's like running noise through a feedback loop that also has a filter on it. Eventually you may get a sinewave tone with enough passes through the filter. There's also something about the limiting rule sets that act a lot like quantization with generative sound design when used with random input values. If you change the rules to the point of quantizing down to one note, that one note is all you're going to get regardless of the input values and how random they are. Weird analogy, but I swear there's some very basic AI behavior that can be observed with something like generative music - and certain rules like picking scales or rhythm patterns will make a melody fit into a genre just by letting a computer do its thing. Sure images or language are more complicated, but it seems that similar nuances in emergent behavior are there.

A third potential is that we decide to move on from brute forcing LLMs to work and get more efficient or different learning models. A human does not need to look at a billion images to learn how to draw. Even if we don't have AI that are capable of what we can do, it does demonstrate that there are better ways to design AI. Right now it's kinda brute forcing and incredibly inefficient

AI can learn from itself when there is an objective outcome to measure. For example Chess, Go and Poker AI engines can improve by playing against themselves (though they also benefit from historical game records and playing against humans). Where there is no objective measure, such as art or creative writing, it's difficult to see how AIs can improve without human input.

Unfortunately it will only be a matter of time and they will either get better with the parameters that introduce random variables that simulate creativity or they will achieve singularity which they are moving extremely fast towards. I believe we just have to keep creating original works of art. Ai can never replace that. Draw and paint more originals and let Ai exist for the people that don’t want the joy and fulfillment that true creativity brings them.

The problem is oldest than your video suggests. Automatic translation and even grammar correctors have been distorting human language (and reducing language variance) for decades, and we have incorporated their language quirks in our day to day language.

I think that if we program in randomness, they'll introduce wacky and impossible and obvious problem elements. The problem is in explaining how to adjust and add randomness to a program which doesn't understand the original state and how it has simplified and made things uniform. It doesn't understand what it does, so how can it introduce some oomph. How can it know when it's introduced too much? There are way to many parameters which can be tweaked.

Very interesting findings there and suggest that while initial models create biases, refined models may also create average bias.

I've been drawing attention to iterative decay of information in these systems for some time. Eventually we will just end up with a 'grey goo' situation similar to that suggested for uncontrolled nanobot replication.

Nice cover...Sabine. thanks

This issue has occurred to me for quite a while. I have worked with Big Data extensively and had brief real world experience with AI development. AI’s reuse of AI-generated data seems highly likely to result in a “creativity asymptote.”

I’ve been saying this to friends for about a year now. I’m glad I’ve finally run into an expert identifying and addressing this potential problem.

Great stuff! I've highlighted this spiraling feedback loop on my blog and in one of my videos, and I'm happy to see similar thoughts here. Since the launch of ChatGPT, content creators, including science and medical communicators, have feared job loss due to generative AI. I've been more optimistic, saying that communicators with original ideas will thrive in a constantly paler and monotonous information landscape. Thanks!

Never heard this before ! That's really interesting. I'm sure we'll hear about it, that's sounding like a nightmare. Thanks

Please, would you put under the video the references of the sources you use? A study from Japan, another from France - please, give us the links! Thank you.

its already happening on youtube. the same videos with different thumbnails usually one or two words are changed

Thanks! you are one of the best things about the internet!

Thank you for the video! I am not an expert on AI just used to work with big data. It has never occured to me that AI collapse due to low quality of input might be an issue, but it actually makes sense. I would like to point out that adding more randomness might now be even viable solution to this problem since all the existing random number generator are in fact pseudo-random (the random result can be predicted if you know the algorithm) Guess we will see how it will play out in the future

One potentially positive side-effect of this "averaging effect" of AI output - if it continues - is that creative people who want to distinguish their output from the common generative stuff will be forced to be more individualistic and idiosyncratic to be distinct and valuable. Of course if generative output is then trained on their later output this becomes an "arms race".

Exactly! I've been wondering about the potential for this problem. Thanks for pointing it out.

AI as it currently stands will become a generalization of people. If it doesn’t become a unique individual, or become better at imitating us, we’ll just learn to recognize them as we grow up and live with them.

AI is geared to come up with a solution using the smallest amount of computations possible. It means that ignoring diversity and details is part of its basic make-up. It can't capture interesting quirks and details and spurt them out once in a while, in some outputs but not others. It also means that if it found one way to get to an acceptable solution, it will only try to get there quicker the next time around. If you've experienced getting stuck in a rut with ChatGPT, that's why.

This reminds me of something happened in bycicle industry, sales were declining so "creatively" they "invented" 29" wheel and stopped producing 26" .

I wonder if the generated elephants look so similar because usually generated images try to match the sample sizes (512x512, 1024x1024) which only leaves so much room for good compositions and wonder if in future with larger models we might see this change a bit more.

I hope it boots universities a bit to start bringing individuality back into academia. The amount of students I heard that said "whatever I'll just chatgpt it and work over the bumps" is staggering.

I called this a few months ago, good to see I'm not the only one who can see where this is heading.

1:10 Why so surprised? It's well known that in echo chambers all differentiating opinion/perception gets eliminated. And when AI is following the input data given, it will convergence to a consensus in order to establish its rules. This is also why "learning the rules" works, the randomness is just to be less sensitive to small input variations.

As a 80-90s kid, I just wanted to say your thumbnail looks like artwork for a gameboy game with that left stripe.

This is why we have "refiner" model layer in generative pipelines.

Very neat. I intuitively assumed it would go the divergent route... but convergent?? That's a surprise. 😮

I absolutely love reading comments & knowledge shared by people. Seems like a computer or program doesn't really know the world, discernment. Still pretty interesting & useful.

The core element of the transformer algorithm (attention) is weighted average. The learning algorithms (loss function) revolves around minimizing average squared error. When you average over a lot of stuff, and all you care about is to have as small deviation from the average as possible, you shouldn't be surprised that you won't produce outliers, just average (one might say, mediocre) outputs.

I think we'll start to filter datasets to exclude 'low variety' data to sort of solve the problem. It'll be an arms race of NNs that generate believable outputs and NNs that are trained to detect the differences.

I think Frank Herbert might have already told us the eventual solution in Dune.

I learned about these problems a LOOOONG time ago! I was raised on the acronym "GIGO". . . then again, my father was a computer programmer with Monsanto back in the 1950s and could program computers in MACHINE language. Eventually, he was able to learn the brand new "high level" languages of FORTRAN and COBOL. I've since turned my attention to Sociology and can see where mutually constructed social realities have crept into compute programs -- and those social constructs are especially strong in AIs.

I love listening to your channel, you explain the most complex subjects in a clear easy and simple way for easy comprehension Thank you and keep up the great channel Love it

Didn't even think of AI Art cannibalism. That is such an interesting thought-experiment. Thanks for the video and I'l be back for more!

I have said this from the start, people forget that AI is just fancy web scrappers. Why on earth would content creators continue building websites if people are going to by-pass search engines and go straight to AI, they will not get visitors so no advertising revenue. So since no information is going to be published online where is AI going to scrap their data from? Will become stale in 5-10 years. If I was a big recipe site, fishing site, car site I would be suing AI makers, no one gave them permission to scrap their data, is their even a tag we can put in that forbids AI from accessing our sites? We can block Googlebot and other search engines from accessing out data if we want, AI?