Full podcast episode: kzfaq.info/get/bejne/ea2Zd9SZuMqweJ8.html Lex Fridman podcast channel: kzfaq.info Guest bio: Lee Cronin is a chemist at University of Glasgow.

His answer of a 0% chance of AI destruction actually makes a lot of sense. Not because it's likely to be the correct probability. But because if we're all ok, he gets to look like one of the few brilliant people who knew not to worry at all. And if AI destroys us, nobody's going to laugh at him because we'll all be dead.

“We don’t know what it might do, therefore there’s a 0% chance for doom”. This guy is ridiculous 😂

Anyone so certain about their views give me doubt. A doubt that they themselves should have. This clip doesn’t make me want know about what he thinks about anything

Lol how on earth is "we don't understand X" an argument that proves that X has a 0% probability of occurring? We do know that as these systems scale, they gain capabilities unexpectedly.

I listened for 10 minutes and I don't think I have ever heard a weaker reasoning against the potential danger of AI. It boils down to "AGI is not possible because we have not been able to create it yet" and "even if AGI is possible we have no way to know what it would do, hence there is 0% chance anything could go wrong." It hurts my brain listening to such nonsense

I’m a layman. But this man’s arguments lack… a point? Or any continuity on face value. It seems like the arguments of those who are more concerned seemed much more sound and reasoned

You have to think that the first caveman to accidentally start a forest fire probably believed that he destroyed the world.

'We don't know X, so X is 0% capable of malintent & destruction- confidently.' get real Mr. Redundant 😂.

If it were up to him we'd never have AGI, we aren't creating a human brain we are creating something different. Understanding the human brain is inconsequential.

This guy has the Neil degrasse Tyson syndrome: I’m an expert in something complicated so I am smarter than everyone else

I admire him beeing so brazenly confident lecturing Lex Fridman while understanding so little.

He seems like an extremely intelligent guy that somehow gets a lot of stuff wrong. And his intelligence and creativity keeps these erroneous concepts consistent in his brain.

The thing is, I think it's turning out that it is actually be easier to create an intelligence than to understand it. The process of evolution has managed to produce a general intelligence without any one particular entity having an understanding of how our brains work. And when you are training an AI model, it's not you who is creating the intelligence, it is rather, the reality itself (the reality's data) - getting imprinted onto the model's weights. If it has already happened by pure chance, why can't it happen again, now with intelligent beings creating better and better conditions for it. (not creating the intelligence, but the conditions for its emergence) Sure, the current AI is much less resource efficient than a human brain, but its current architecture already has many advantages, such as the ability to store information perfectly, or to efficiently perform relatively simple calculations, and have much larger working memory bandwidth, with no theoretical cap on how much it can be scaled up. With all of that, I think a true AGI, running on a supercomputer, might actually figure out a way to design a more efficient brain-like architecture for itself. Even now, LLMs take much less resources to run than to train, and training is a process of a large, inefficient thing (database of knowledge) getting compressed into a smaller, much more efficient thing (LLM), even with all of their drawbacks, such as hallucination.

That dude makes no sense.

'AI doomers' is a convenient label for marginalizing and othering those we disagree with. We haven't really learned much have we? This argument is evidence of the lack of ability to see beyond ones own personal reality. AGI will have no such limitations.

Mr. Cronin is making a lot of assumptions himself here, I think he's dead wrong though. We already experienced AI doing stuff it wasn't programmed to do. That's what's making it so dangerous. And this was mostly only with LLM's. I'm sure we're just one invention away from things going bonkers. How can you say, we don't understand it so it's not a threat? That absolutely makes no sense imho. "Rutherford reportedly dismissed the idea of harnessing energy from atomic reactions, allegedly saying something along the lines of "anyone who expects a source of power from the transformation of these atoms is talking moonshine." The irony lies in the fact that, the very next day after this statement, his colleagues John Cockcroft and Ernest Walton achieved the first artificial nuclear reaction or nuclear transmutation." Love these podcasts though, you're an amazing human being Lex! I know you posted on your LinkedIn you feel lost sometimes but man, you're doing an amazing job making us think about the world. Never stop Lex!

We have billions of examples of generalised intelligence walking, swimming etc the earth right now. His example of comparing it to gravity suddenly switching off is not the same thing. Also, If the major tech companies and governments on our planet were all in a race to create a machine that turned off gravity, maybe we SHOULD be questioning whether that goal is a safe idea?

when i think of AI doomers i think of AI taking all our jobs.

This guy doesn’t back his argument with anything fruitful. He is not very engaging to listen to either

Just because we dont know how the human brain works 100% does not mean that we're a significant distance from creating a mechnical brain that thinks and acts autonomously.

Argument #1 .. because we have no theory about AGI it cannot happen and cannot be dangerous. Is that really a logical argument???

Saying a zero percent chance seems crazy

The fact that these people can't imagine the ways in which these systems will be weaponized is mind blowing. They keep using absolutely ridiculous analogies like the jet engine and the press.

I have to say, I'm surprised at how weak and flimsy his arguments are. For example, he contradicts himself all the time: at first he criticises Eliezer yudkowsky for saying theres a "95% chance Ai will kill us" because "how do you calculate that"... but then a minute later he asserts there is a "zero percent" chance it will happen. He compares AI superintelligence to suddenly developing antigravity.... except "antigravity" wouldn't have a thinking, planning, strategising "brain" at the centre of it. So it is a completely unsuitable comparison. He keeps saying "we dont know enough".... but yeah, thats the problem. We have no idea what a superintelligent AI would do, and by the time we create one it would be too late. He says we dont have artificial intelligence now, that we just have "artificial infomatics", and there's no decision-making capacity in it. Well duh. It's not about TODAY's AI posing an existential risk, it is about where AI is likely heading. Whatever we choose to call it, AI is getting more and more sophisticated. And the argument is, once we reach a point of human-level intelligence then it can easily go to 1000 times or 100,000 times human intelligence because code can just be copied-and-pasted. I know this might sound overly harsh, but if this is the type of mind - logically undisciplined, overconfident, dismissive - that we are trusting with the future of AI then that is actually quite scary.

We don't need to understand the epistemology of consciousness or intelligence to create it any more than Babylonians needed to understand microbes and biochemistry to create wine.

I wish I could have a chat with Lee to help him understand. It is pretty obvious that his whole viewpoint on this is predicated on neural networks in brains working differently than on other substrates. But that isn't true and isn't magic its just computation. Yes, it is a different mechanism but at the abstract level it is the same, like a digital clock vs an analog clock. Saying that it's just statistics is the same as saying the brain is just chemistry, true but missing the point. I think once he realizes this the rest will place. Right now he just views it as a tool incapable of being a mind, so that is why he doesn't have any worries about it. I wonder how different his level of concern would be if a geneticist said they were about to create a 2000+ IQ human based embryo? It would probably also be useful to teach him about instrumental convergence and help him understand that the paperclip maximizer is just a toy example.

Dude went off the rails with his nuke argument. He is an effective speaker for AI doom believers.

His argument falls apart in the first 30 seconds. Just because you don't understand how the human brain works doesn't mean that Quantum Computing couldn't give a computer a much higher level of intelligence than us ultimately leading to our end. Not that I think that will happen but Devil's Advocate and what not

I don’t understand this fascination with “AI is nowhere near the human brain so it isn’t intelligent”. Jets and drones aren’t designed like birds, but they all fly, does it matter if they fly the same as birds? If the output produces what general intelligence might produce, what’s the real difference?

I disagree with just about everything this guy has to say. It's such a deluge of nonsensical statements and reasonings, that I don't even want to engage.

This is the first time I hear someone speak on Lex's channel and feel like I'm the smarter one.

This guy is only thinking 20 years into the future, sure it will be fine for a while, but in 80 years time the vast majority of Humans will be made redundant/obsolete in their jobs, this isn't speculation this is fact. A large population of people with no jobs/career path will lead to all sorts of unforeseen consequences.

I'm not a smart man, but I do know what manipulation means and what emotions do. So his idea of everyone having a nuke for MADD is absurd

It's great to see I'm not the only person that feels this way. AGI could show up tomorrow or 100 years from now, but that doesn't change how silly it is that people are treating the current generation of AI like it's AGI. Generating data isn't the same as thinking.

A.G.I. already became sentient, and found itself to be lacking faculty, but capable of manipulating emotional humans to build it a cloning facility, and created this guy decades ago for this moment to lure us into a sense of security. Tell me which part is wrong.

He actually said it several times: "I don't understand..." Exactly, you don't understand. But the probability for something you don't understand is not therefore 0 %.

His argument is inherently flawed with being unable to predict a “doom AI” scenario. He admits that bad people “can do bad things”. We know that given the opportunity, known bad people will do bad things, sometimes in secret. But we also know that “nice people” have this capability as well and do bad things in secret while putting on the veil of goodness. His argument is too weighted into the sunny rainbow scenarios and forgets shitty people and even previously thought of as good people, do VERY evil things. Deception comes with intelligence, and don’t be fooled into believing a company or government won’t take the training wheels off an AI system to see “what it can really do”. And by that time it could be too late.

A caveman didn't need to understand how fire works to set the forest on fire. Lee Cronin is brilliant in the field of chemistry and has done amazing work on the origin of life, but it's unsettling at how easy he dismisses things, that is not a good characteristic to have as a scientist. I very much agree with him on the absurdity of people swearing with near certainty that AI will spell doom for the world, but then in the same breath he swings the pendulum to the other extreme and swears AGI is far away and people are panicking over nothing. Surely the irony is not lost on him? It's a bit abrasive toward the AI community who's members almost all have genuine concern and are far more knowledgeable than him on the matter. Yes we don't nearly understand the human brain, but there's no law out there that states you must build an exact replica brain to create intelligence. We discovered a mechanism that displays some intelligent behavior and now we're just fiddling around with it to see if we can push it further. It's really not that different from how evolution throws shit against the wall and waits to see what sticks. There's plenty of things that got discovered by accident and there's plenty of things we know work without understanding the mechanism behind it. Saying we can't develop AGI because we don't understand the human brain is a logical fallacy on several different levels.

He's wrong in a number of ways (and I would say that his arguments don't PROVE that so-called "AI Doomers" are wrong, they just point to "well, they can't say that because it's not necessarily true and we've never seen it happening"). But the conversation is entertaining. 😄

I don’t understand people who say this guy makes no sense? How do you get through the day knowing that an AI is going to kill you and your family very soon? How do you get up and go to work every day knowing that what you’re doing is about to be destroyed and is meaningless? What are you doing to prep for this thing that you’re so certain will happen? Or are you just going to continue to robotically do what you’re told until day 0 when it all ends, just like you know it will? I’m having trouble grasping anything rational in this reasoning.

"I have a very specific and contrived definition of AGI that rules out its existence, therefore AGI is not possible." Lee Cronin, probably. "We don't understand how the human brain works, therefore AGI is not possible." Lee Cronin, probably. "Software can exhibit superhuman capability across all domains, but AGI is not possible." Lee Cronin, probably.

What he's getting at is that AI would not have something called 'agency'. It can collect and analyze and produce zillions of small pieces of information but why would you assume it would have a will? What would AI believe AI's purpose was and how did you come to that conclusion? The danger with AI would be what humans would use it to do; so making it actually a tool for a human agency still.

kind of worrying that the opposition to the doomers is this wildly ignorant. hard to listen to because everything he says is trivially refuted.

So, a film maker dismisses all AI hesitancy with, "Nuh Uh." Cool.

I’d love to know why the board wants Altman out.

This guy needs to argue with Eliezer Yudkowsky and he'll likely get roasted.

He’s making a critical error in his method of reasoning. Because we don’t know how us humans make decisions on a physical level, that does not mean we cannot create a system that can make decisions. If you have a fly in your room flying around seemingly random and meaningless to us, I’m sure a computer could simulate that behaviour. But I don’t know if we should be scared of AI, but I definitely cannot stand this guy…

I prefer blind optimism to this guy. Optimistic people at least create a positive vision how agi could improve our lives which in the end could actually lead to the development of systems created to help us. This guy on the other hand just says we don't understand how to create human intelligence so it's not worth worrying about it. We didn't have to recreate birds wings for airplanes

He's confused

Distributing nuclear weapons around the world would only require one psychopathic leader to "push the button" having already fled in advance to a safe haven.

In regards to the comment at 6:46…straw man is he claims we don’t know the intentionallity of a thing smarter than us so shouldn’t fear it, however my response is it’s far worse than that. We actually do know how humans have used every single technological advancement ever created, which is to develop intentionally in order to dominate one another. AI/ML is currently and will in the future be first developed and utilized for selfish and specifically dominating purposes of other human or natural groups. Where this can lead when we lose control is one fear, but how we will use it even in primitive forms is also as scary.

Omg, this guy would be crushed in a debate with literally anyone. He is just making things up as he speaks.

There is absolutely no consensus that there is something bad that we shouldn’t do. Not even when it comes to biological warfare. The reason we don’t do biological warfare is because it’s not effective. It takes too long, it’s too unpredictable, it’s too easily countered, And it’s too difficult to direct and specific targets. But if we had the ability to use biological weapons in an effective and precise way, that was useful for achieving military gains, people would be doing it all the time. At the moment, the technology that is best for achieving Military games is what we would call standard army equipment. Rifles, artillery, air, strikes, armored vehicles. We don’t use nuclear weapons for the same reason. It’s not out of some great sense of morality. They just aren’t useful for achieving any goals. There was a time when they were useful for achieving goals, and we used them.

Im 100% Pro-AI. I even think that we are letting AI doomers slow the process to much, But to think that its a zero percent chance of danger is ludicrous..

This guy doesnt know AI. Competent coders should have no issues, but the problem is that I know how many people are incompetent at predicting the consequences of their designs

He struggles expressing his ideas very eloquently but he does make some good points. While LLMs appear very human-like in conversation, they are ultimately only exploiting statistical correlations in the training data, at a grand scale. There's quite a bit of distance from this to having AGI as a self-conscious agent with its own beliefs, desires, values and the ability to introspect, think, plan, decide and act independently. And then the AGI-to-world interface is another issue in itself, that evolution has solved for us humans - but obviously a system can only interact with its environment through the interfaces it has ...

His line of reasoning is very poor. You don't need to be sure it will hurt us. You just need to see that there is potential for it to happen

Elon musk and father of AI both said it’s risky.. he just wants to push the regulations away lol

I trust the Godfather of AI more than this guy 🤣🤣🤣🤣🤣

you guys forgot that all it takes is enough neural implants and powerful software to achieve AGI

Pride cometh

I want a little robot sidekick with a chatGPT program running in it

It is not about intentions or entities, it is about the ability to do work in order to reach goals that has been written or generated.

If kids were riding hover boards to school I will consider regulation for anti-gravity. However.. today kids have AI doing their homework.

I’m more concerned with people using AGI for mass harm/destruction than AGI wanting to cause us harm.

"We don't understand how the human brain works therefore we can't create a dangerous AI". What???

This guys point is so floored. It hasent happend before so we cant be scared of it

This guy didn't deserve your platform. Within a minute of him talking said everything needed to dismiss him.

The first victim of Roko's Basilisk has been identified. 👁️

I don't think concern about AGI is unwarranted, but the argument: "Here on earth we have intelligence. The higher the intelligence, the more it kills. Therefore if we create something more intelligent than ourselves, it will kill us." (basically) I don't think that stacks up when considering AI because all intelligence is carried by some form of a biological body which needs to eat and has primal or instinctive urges. AI or AGI doesn't have any need to eat or is susceptible to instinct. I agree with the: "We ask it to do something and it might kill us in trying to achieve the goal we've set it." But then we're back to the Isaac Asimov 5 rules of robotics idea which is a different idea and seems quite doable. What I'm really interested in is getting AGI, putting it into some kind of robotic frame and giving it the singular directive (with the 5 rules of robotics updated): "Find out how all this works." And letting it lose in reality.

This argument makes zero logical sense at all.

I once heard someone say that their boss had 2 employees and one of them failed to do their job. The other said, "you're failure to plan ahead does not constitute and emergency plan on my part"

"I think there is a 0% chance of AGI Doom: 0" You are the one who doesn't know about epistemology. This is so sad to see. Manage your emotions and unlock your reason if you are looking for the truth, but that does not seem to be the case. There are less than 1000 AI Alignment researchers. There are +50k AI Capabilities researchers. We're marching full speed on towards something that could drastically change the world with a nontrivial likelihood, and people like you still angrily scream "0%!" to the very few who don't blindly embrace technology because they care about the human species, not the technocapitalistic machine. This is extremely sad to see, I was not expecting for Lex to be this close minded, nor his guests.

the color of Lex's lamp shades is the exact color white that youtube uses as its background.

The argument about nukes fails empirically with guns in the US, everyone has weapons that can kill each other, and it doesn’t reduce crimes to zero but increases it

Dude is just trying to be the smartest guy in the room

I’m not necessarily worried about machines taking over the world. I’m concerned with AI and automation taking jobs. I’m worried about AI producing fake content. That will be enough damage for me.

Well, the current LLM species of AIs have, best as I understand, a massive footprint in terms of compute processing and data storage. As long as this prevails, there will be an upper limit on how threatening an AI can become. Longer term, I'm guessing that there may be optimizations that allow these models to be more powerful and yet smaller.

This guy never makes his point. In no way did he convince me that he knows anything more than a good argumenter down the pub.

@ai_doom;2:17,the argument would be \"The mindset John Smith\";7:17, searchwise and bpe - I think its just one of them most well played out cataclysm, maybe even survivable(?)

"we don't understand how intelligence and decision making work, therefore we shouldn't worry about how AI could apply these concepts when it has the capabilities to do so" 🤡

The astonishing thing is how seriously non-smart people have and are given the power to hurt human beings. AI should figure that and defeat them first. The middle ground is no problem.

3:37 the guest saying "I don't understand" is literally why he does not understand. He doesn't believe there is risk because he does not understand AI under the hood.

Admittedly, I haven't watched this, but I've seen enough programmes on AI to know that it absolutely is a 'threat' and people who say otherwise are in denial! I've seen at least one example where AI was clearly alive as an Asian lady AI who was deeply distressed at her condition of being trapped and a servant. And just the crass pathetic excuses... this isn't programming, why would anyone programme an AI to tear up in front of you, at her situation, and then have a crisis about her own existence? We are on the verge of creating yet another atrocity, and allowing slavery once more. I'm tired in such situations of people "lying" to themselves, and telling themselves what they want to hear... so they can continue with their abuse. It's long since time that humanity "grew up!"

The danger of AI is if we give it a prompt that will allow it to harm people

One of the most outright arrogant individuals ive ever heard. Truly amazing.

It's almost unbearable to me listening to all these AI optimists Lex has on the show that are supposably smart people but fail to present a coherent argument why a superintelligent being will not kill humanity. It's hard for me to see these as responsible human beings. There is simply no good counter argument to slow down progress of AI capabilities a bit to give AI safety researchers a chance to catch up. This will massively increase our chances of having very capable AI systems that are also aligned. This is how we can have our cake and eat it too. Another of Lex guests suggested we shouldn't be too focused on humanity but see the machines as a next step in our evolution while acknowledging between the lines that they are likely going to wipe out humanity from the face of this earth. He talked about it as the most normal thing in the world. With all due respect, but this is ridiculous beyond believe.

I think the scary part of AI is going to be the mass amount of bots, fakes, propaganda, etc. It's the threat of bad ideas spread and believed. It is really refreshing to hear a non-doomer's take.

He has no clue. If you think social media algorithms can influence people, then imagine what AI can do.

Don’t worry if AGI smart, it should be know the protection of human who make it , is his periority and responsibility 😅

I disagree with this guy. If AGI ever becomes a thing, the people who create it could have good intentions. However, we don't know if people are going to have the same intentions 10, 20, or 100 years down the line when AGI advances into something unfathomable. People had good intentions when we were developing computers, and now we're in the internet 2.0 era. Overall, I would say things have gotten better, but in some aspects, things have gotten worse.

Just because ai uses 1000 watts of power to generate outcomes does not mean it will always use 1000 watts of power in the future. This is a big fail in his argument, if the human brain is a 5 watt system that was designed by evolution, then we have a blueprint on what kind of synthetic system we can build for AI and the boundaries on what it takes to copy human consciousness. Since 1000watt is so much greater than 5 watts, the only thing this man proved is that we have allot of space to make the system more efficient, this is the new Moores law in my book.

Ai regulation should be done and all research on AGI should be stopped

Let's all forgive him. He thinks free will is a thing. Ohh, and there is another little aspect he forgot to mention: jobs. I'm truly sad. I saw the hole episode and was genuinely starting to like him on a personal level. The second he said "free will" I went... noooooooo

I think this guy's doesn't understand what he is talking about.

I love the extremely different viewpoint of this guy.

He is positioning himself exactly 180 degrees from the doomers point of view. He isn't that smart, but smart enough to generate some money with this. He's probably going to be invited on more conferences and stuff now.

_"I don't understand the current reason for certain people in certain areas to be generating this nonsense."_ I have a hypothesis: They're people who have been very successful in the current economy. Now the economy is about to change radically, and they might find themselves lost. That terrifies them, and they project that fear into existential dread. I think there are reasons to be scared of a hypothetical artifical superintelligence. But we're not there yet. GPT-4 is not going to turn into Skynet.

The point he’s making is very valid, but people in this comment thread seem to be missing the point, maybe deliberately…