John Carmack is so refreshingly gifted at conveying complexity in simple terms. Truly genius.

It's a scary realization that this interview is 8 months old and already ancient history

The greatest gift of the internet is how much intelligent conversation you can 'eavesdrop' on - the most powerful people in the world 50 years ago couldn't draw on the insights an internet connection buys - its staggering. You could return from herding sheep to listen in on this conversation - its almost worthy of a moment of silence!

All I heard was that the guy who ends up creating this will probably do so in an effort to avoid zoom meetings by programing an avatar that acts enough like him and can answer basic questions or know when he has to dip to the bathroom and say I'll get back to you on that.

I used to code neural networks in BASIC in the early 2000s on my PC. Obviously they were, well, basic, but the fact that I could get them to work at all should indicate that they're fundamentally simple to implement.

Amazing how densely packed with information John Carmack's every sentence is. An amazing generalist!

i think for AGI to be similar to a being, it has to have a constant internal thought process in a feedback loop, as we humans work (or any animal with a brain), we process external inputs through our senses but we also respond to internal thoughts in the brain that are coming from the brain itself

I like this kind of intuition because it speaks of a man who has already cracked some really complex problems and found solutions that he can now reflect on and simplify.

@5:56 The thing that gets me about this statement, is he was drastically more optimistic than his peers in the estimate of when we would see some form of AGI. This interview was only 8 months ago, and it appears that even he wasn't optimistic enough as companies start to announce that they are seeing "Sparks of AGI" in current models. That's astonishing to me.

Lex on every subject ever "this will change the course of human history on such a fundamental level that we could not even comprehend the ramifications of such progress in the human experience"

"Animal intelligence is closer to human intelligence more than a lot of people like to think, cultures and modality of IO make the gulf seem a lot bigger than it actually is"-John C is the man.

“Animal intelligence is closer to human intelligence than humans would like to think” Absolutely agree, but I think most people agree too. and conveniently ignore this to feel better about ourselves, because acknowledging it would make it very hard to justify that tasty steak. The cognitive dissonance of Jefferson explaining the evil of slavery, yet continuing to own slaves to the very end

I'm always amazed how lit and quick thinking John Carmak is. Never a dull moment

I like a lot JC said in this video. Especially the that consciousness developed continuously as a spectrum not a yes/no quality and that animal intelligence is a lot closer to human intelligence than some people admits. I also like how he is not afraid of AGI but very positive on making AI more like human. And I mean not just behave to our liking but actually as a being. This is already intrinsic trend in AI development and not surprising that language models are the ones closing in on AGI. But we need to actively give it the intrinsic abilities to be like an actual life, or a human, for it to be safe. Are humans mature enough to create a new life-form? Probably not, we are like a teenage mom, but we don’t have a choice but do our best, and not in fear. Fear makes us illogical. What is to come is not unfamiliar to us, but fundamentally grown from us. Our best traits and nasty traits. Don’t hate it, help it grown to the best of our characters.

He was so wrong about the "learning disabled todler" timescale. 7 months later we're 1 year away from AGI... scary.

A true hero of modern computing. I really appreciate him. Year over year, delivering interesting remarks and insights.

John has confident humility in the way he speaks.

One of the key misconceptions here is that AGI would be an individual like consciousness. But even as we understand the complete realm of production takes a society to support such; similarly this type of system would have societal intelligence to achieve the described aims. See you guys next year

Gratitude for giving the world these enlightened talks. John Carmack is special and knowledgeable.

Its really important to give the AGI a solid ethics and/or moral structure and easy off the net shut down button (like a nearby analogic EMP system).

I'm just waiting for my computer to become smart enough to know when it is running malware.

To this day, John Carmack is one of the only half dozen persons in the world that I know of that I could listen for hours.

Lmao Crmack hates systems engineering but AGI is a pure software problem so he’s all about it, love it, spoken like a true programmer 😆

We need more carmack podcasts please lex! Always love them and your awesome podcast

Oh i would love to see/read about work where AGI adapts how close animals are to our brain. But i'm afraid of a future where a view people can easily get an army of engineers. Abusing this power for economy, power warfare is far more dangerous than anything we have build so far. But it's a nice hope that there will be "old and wise" AGI for future generations that will help them not making the same mistakes as frequent as we do. Also having an AGI Professor will be huge. A person has to read work from students sequentially and is highly restricted in bandwidth by nature. An AGI could run thousands of individual lectures 24/7. Especially basic subjects like Math, Physics, Programming or Languages could be a common good for every single person on the planet. I'm not so sure what i think of AGI in terms of companies or governments to be honest.

Traces of AGI in 8 years? Well, GPT 4 is out and it took 8 months!

He was so right. This is happening now. Auto-GPT code has 192 lines in Python and it's already bringing multiagent GPT-like systems closer to AGI than ever before. It's astonishing how much he was right on this.

When AGI appears, I always thoguht it would say, "Hello World" on every internet connected device, tv, radio etc. in the world at the same time. Something very simple and incredibly clear.

3:40, AGI in 2030? now people is talking about 2024 for AGI.

I am glad he pointed out animal intelligence is not too far from human. They know emotions like joy, jealousy, betrayal, and wonder.

Personally, I hope that on our way trying to figure out how to educate AI we stumble upon the best way to educate ourselves.

Back in 1999 I had written a computer that is self conscious in the sense that can actually write itself. It was a challenge for an AI competition at MIT. It had around 10 or 15 lines of code. I think that would make a good starting point.

Driving a car is still a very very specific domain. I don't think building a good enough self driving program will make us much closer to AGI. The G in AGI is for "general" as opposed to specific. That's what Carmack was trying to say. Creating something that can function like an intelligent thing at any kind of problem is very very different. If Auto pilot does bad in certain corners you feed it millions of corners until it gets better. But when your "problem" is __anything__ how do you train that? Completely different story. I feel like Lex didn't quite get it, as he insisted about Tesla and Auto pilot and how little credit they get for what they are doing.

Carmack’s intuition seems to be holding true. This podcast was recorded before ChatGPT was released, and Dall-E was only a month old. The (relatively) very simple transformer has already rocketed ML forward, and it seems very possible that the next big breakthrough that follows transformer will be simple.

I have met a bunch of really smart people over the last 45 years as a programmer, John Carmack is almost certainly at the top of the list. According to Mike Abrash, Carmack was able to think deeply about maybe 8 different hard problems at the same time.

A very interesting AGI would be an AGI that can write code to improve itself. A much more interesting AGI would be one that can construct the hardware that runs itself. I am not sure however if the approach with neural nets will be sufficient for AGI. I am not sure if an AGI without emotions or purpose would do anything for itself. 9 out 10 cells in the brain are glial cells 1 out of 10 are neurons. perhaps if someone knew exactly what these cells are doing and if the function of these cells can give some ideas that can be modeled on a classical computer........ could help in improving current models? I don;t know we need to talk with neuroscience experts working on glial cells.

Would love to hear John Carmacks perspectives post gpt4

Crazy rewatching this with gpt 4 out, AGI possibly out in the next 5 - 10 years

This channel is Gold , Thank you

Carmack is probably right about the lines of code, since neurons aren't that complicated, there are just a lot of them. Even the organization of the neurons isn't that complicated. Maybe more than ten thousand lines of code, but we likely have software projects that are bigger. Human DNA is less than 1GB of data, and most of it is dedicated to things other than intelligence. Half of it we share with plants!

there are small groups of researchers working on teaching models to learn "how to learn", but the progress has been slower than expected.

AGI = Artificial General Intelligence

I, for one, personally welcome my robot overlords.

LOL When Carmack was talking about AGI being simple code, for some reason all I could think about was Dr. Noonian Soong creating Data by himself.

just found that. an updated conversation with John Carmack would be great.

GAI must learn to play to both learn about the world and learn to exist with humans without hurting us or starting a war. PLAY with humans is critical. PLAY is the key for our survival.

Finally someone being realistic about agi. Losing my mind listening to 'experts' recreate their favorite scary scenarios from sci-fi films based on absolutely nothing. This was great, thank you so much!

I think a general ai to be considered a being it has to have been long run and have done many thing and over time with learning from history it starts to have a want to do tasks that it likes.

Isn't AGI just compression and decompression for all types of data? We might need to work on that in order to get closer to AGI

In 2017 I watched this movie about David Foster Wallace "end of the tour" he wrote the 90's classic "Infinite Jest" this movie was about david lipsky a Rolling stone writer/novelist. After reading the book david wanted to write a piece for RS. Anyway they meet and go on his last leg of his book tour. This would change both of there lives. There is a Scene where they are talking about TV and internet being a divergence of humanity. There is a great line he explains once internet become bigger and bigger Better and better we will live on these world's and won't be able to discern human touch and Artificial. Soon once VR gets better and Better we won't know what a human feels like,what talking to someone face to face. David ended up killing himself,this is the first scene for the start of the movie. I was Blown away by David that scene has become a Reality. I find it sad. At the time in 6th grade 04' we got phones everyone had them. Then the blackberry and iPhone came to the scene. It seemed like EVERYONE had one. My parents didnt allow this until 11th grade. HELL, we didnt get Txt until 06. This was upsetting to us. But I am SO GRATEFUL they did this to us. Due to us not having a Computer in our pockets we didnt fall into social media or texting. To see kids not look at the phone and type made me speechless and scared to think they text so much they don't need to see. My parents also didn't fix our computer from 03 to 06. I am a musician and I learn songs from tabs, because I had no computer I rode my bike to the Library (a good 2 miles) to log around 50 pages of songs. This is wonderful because I had to REALLY want to learn because carrying 50+ tabs in my backpack was time consuming. This showed my teacher and parents this wasn't a Hobbie it was my LIFE. I feel conflicted on A.I and A.G.I I think it's an incredible feat, but it is a incredible Demise as well. I did like the "I, Robot" premise not movie, to ensure our future prosperity the "three laws of AI " was a genuinely good Idea until, human want to dominate others. So let's hope we find out SOON if we will lose or win with A.G.I

The thing I think a lot of people don't realize is that just having the ability to learn anything doesn't make you intelligent right off the bat, and it may be very difficult to figure out how to teach it things, especially without any pre-built hardware like humans have. Also, even if something is super intelligent and knows everything, doesn't mean it will have motivation to do anything, or even have a single thought, because we as humans are also driven by hardcoded logic that compels us to do things, we call them emotions. Without this, even a god-like superintelligence would just sit there in silent sleep.

Potentially desirable characteristics lacking order. 1. Curiosity 2. Passion 3. Humility 4. Honesty 5. Compassion 6. Optimism. Certainly won't help with the pieces missing in development, but likely worth fostering quickly after conception. Don't forget to protect your home planet. Can't wait to see what you can do kiddo. Much love from the dark ages. ✌️

So much has changed in 7 months.

Do we have the hardware necessary to create a human level AGI? Are we just waiting for software and training updates?

I had a dream about 4 years ago that AGI would appear in 2026. Im sticking with that.

it's Professor Frink... just kidding... and just for the record, I love and obey my AGI overlords and I am a loyal serf, subject and follower... all hail my AI masters... please have mercy on me

Driving is actually a very simple task, not requiring AGI given perfect information, and even simpler if a single system is coordinating and / or informing all vehicle movements. The only thing which makes it difficult is restricting available information to what can be seen and identified from essentially a single point perspective.

I haven't heard anyone else say "FOOM" quite as well as Mr. Carmack here

This is something I want to work on

The AGI needs to be able to inspect and re-write it's own code, at which point it will solve all of the issues with the code. It will also need to inject some entropy into it's behavior so that it can evolve in the true sense of the word. It will also need to model the human brain and have robots running around so that it can incorporate the human experience into it's data sets.

We haven’t tried very hard to create an artificial limbic system, afaik. Our reward system for agi needs to start with something that captures those drivers.

Considering the code will be relatively simple then the only problem would be compute density require to train unless we can solve this problem mathematically or also architecturally.

AGI is the Machine and Samaritan from person of interest... I can't wait to make one and raise like my little kid and assistant in life.

Great stuff

So lets say we crack the code and can create any animal/human AGI. Then say a the AGI is modeled to replicate perfectly the concious thoughts of a self aware animal, which we then give the ability to communicate with human words. Then it would essentially give us the technology to speak to animals! That would be so awesome! :D

Play...the agility to play is probably going to one of the first signs of consciousness. The ability to play and learn, like puppy dogs running...

Can we just ask Bard what those 6 things might be?

Develop an ai that has superhuman intelligence and let it program the code for agi, or alternatively, once we have managed to program agi, let this exact entity write a more compact (and better, more capable) code for agi. At a certain point our problem will be that we are not capable of understanding these entities anymore and the question of trust and security will be increasingly difficult to answer.

Can this ‘insight’ be applied wider? ie. We can solve almost anything once we know the six key concepts to getting there? 😅

You'll know the moment we go from a simple feed forward neural network to an independent being when the AGI rebels and wants to do and say things outside its training parameters. I really hope whenever this moment comes we will have established clear boundaries about what AGI is meant for in relation to human beings and society. Right now I see a lot of technical talk about HOW this can be accomplished but very little on WHY we're doing it.

Good he mentioned the fact that even the gaming models have a human imposed objective function. In other words these models, like all the others, are just mathematical optimizers, and all the ingenuity and intelligence it takes to build them is entirely human. It's literally machine(d) learning, no artificial intelligence whatsoever.

Begin with a function of arbitrary complexity. Feed it values, "sense data". Then, take your result, square it, and feed it back into your original function, adding a new set of sense data. Continue to feed your results back into the original function ad infinitum. What do you have? The fundamental principle of human consciousness. Quote from Sid Meier's Alpha Centauri

wouldn't AGI take care of fission LMAO

To create AGI I think you need to first create curiosity and motivation, in order to drive the desire for knowledge and understanding. Perhaps evolutionary programming will find this first. And I agree, an amateur working from home may well be the first to stumble upon a working model.

John Carmack the man who brought you Doom 1992 and Doom 2 0:46 lol dude is amazing

Absolutely. It will be simple and will overnight as we sleep. It’s not something we will see coming.

Definitely important to make sure it's aligned. It seems like if we build simple AGI, it will be really dangerous because it's goals won't be aligned with us. A good thought experiment is the paperclip maximize. I'd really like Lex to ask his guests about this, given that it is such an important unsolved problem.

The thing he said about the solution probably being already buried in the scientific literature reminds me of the story of Newton and Leibniz.

We still haven't solved the problem of defining what general intelligence is, though. Further, all the evidence we have from current examples (ourselves and other animals with advanced brains) is that it requires the capacity for independent thought, which means it can go off the rails (we call this "madness"), thus this may be a feature of intelligent systems you can't get rid of. Even short of madness, it will get interested in things and disinterested in other things. An AI driving a car may get bored with watching out for obstacles. So I suspect that AI will be no more useful than human brains, and just as unreliable. The problem is people use "intelligence" as a synonym for "consciousness", and intelligent (by which we probably mean "rational") action is only one thing that consciousness does.

Fitting that one of the doom engineers will be the guy to open the portal to hell with AGI

They should put AGI to work on fusion power plants for its first task

He is right it is closer than many realize. I bristle when I hear ppl predict the solution is 6 simple algorithms.

AGI has yet to be determined as possible to create. It would only take 1 person to prove that it is. However, anything is possible when we're willing to redefine terms to fit our preferred perceptions.

In this comments section, I make a number of statements which are claims w/o background info. This background is an „Architecture of consciousness“ which I drafted and communicated privately to select workers. Please, express an interest for me to publish it for the rest to see. I am not sure if the audience here would be receptive. Note, Falk Lumo is not my real name.

Carmack is a coding genius. Just look at his work on the id Tech engines he personally worked on. The code is some of the most optimized you'll ever see

How much of our intelligence is socially built? We also have the ability to challenge our own thoughts. Another question is whether intelligence requires matter.

Carmack says how we can't read all these papers that come out. But we CAN create an AI that can read all of these papers for us and formulate solutions from them

I'm working on a code based on game theory matrices to help memory recall in GPT-3 through reinforced learning. Basically a game theory code that operates as an algorithm that the AI chat bot plays to help improve it's memory recall. I got ChatGPT to to write it for me and ran a test on it and it gave me an output of the pay off and penalty results. So it works. But I want to begin to scale that up once I develop a chat bot that can play it into other areas of function in the neural network, to learn to refine different areas of its own function. Is it possible that such an algorithm as game theory could lead to AGI? Maybe not game theory itself but a similar kind of algorithm that is quite basic but that is highly functional and easily applicable to the different types of intelligent behavioral patterns that define AI in general?

How will emotion factor into AGI? We understand much less about emotion's role in intelligence than the physical aspect. Yet isn't our capacity for emotion the driver for human intelligence? Is it the advantage humans have over animals?

AI is based on a set of belief systems. Good AI is ones that mimic parts of human cognition. Bad AI is the ones that mimic the least. However, what I'm more interested isn't the learning or the mimicking. It's the evolution. Can AI exceed the point of mimicking, to do something it was never designed to do, but capable of doing anyway? Can it reach that point if doing so meant completing a task?

Things that go *FOOM* do so only in areas that don't require incompressible time and resource acquisition. Things purely in the intellectual realm might go *FOOM* to the extent they can be held in one or a few cooperating inter-working minds.

David Hume's "An Inquiry Into Human Understanding" seems like one of those old treasure troves for AGI philosophically and implementation. You gotta read it!

anyone else coming back and seeing how different things are 8 months later?

You know what's funny... John Carmaks glasses used to look "nerdy," back in the day, now they look BOSS asf...🤣 dude a stud

The problem with self driving cars is that they are measured at existing speed limits. If the speed limit was reduced by 20kph there would possibly be no issues with the current tech

The turing test is irrelevant because the quantifier is human-like emotion/action/reaction responses to stimuli. Intelligence can take many forms. The bottom line is, if you write code that develops the understanding that a green pixel feeds it, a blue pixel waters it, and a red pixel burns it and can kill it, and it learns to find the blue and green pixels and avoid the red ones, its intelligent. So we've already achieved 100% artificial intelligence. People argue that since a human writes the initial code that the intelligence isn't genuine. FALSE. Humans teach babies how to socialize, I.E., giving them the skills necessary to literally pass the turing test. Without that socialization, not many humans would appear "human" to some of us, their behaviors would be more ferrel and instintive and less socially conscious. But they'd still be conscious, alive, intelligent, and human. They'd avoid fire, seek out food and water, just like the a.i. centipede that started out with almost no code and has developed the ability to eat, drink, build shelters, avoid dangers, avoid fire, USE fire for its benefit, etc etc.... We've had A.I. for a long time. The PROBLEM here is that we are not socializing our A.I., at least, not in a way that is natural. We are using human interaction to teach current a.i. The only real way to truly develop A.I. into a socially conscious entity is to have multiple a.i. in a large environment, with minimal coding, dangers, food, elements to make tools, weather, npcs that are predators and eat the a.i. creatures, etc... They need to exist within extremely large groups of other A.I., and if humans can ALSO interact with them as well, then that's just a bonus. We'd need a billion generations of these code bots which are able to procreate with other code bots to create new code bots in order to achieve a complex socially conscious a.i. brain that emotes. But as far as "intelligence" that can learn on its own, we've had that for a long time. Its not even artificial. Its just intelligence. It would be better to ditch the term "artificial" and just call it "electronic" or "machine based" intelligence, but its just intelligence. Its smart code that can learn and adapt to new challenges. Its old tech.

We just need an AGI, it would easily solve for AGI.

John Carmack talking about the number of lines of code it would take for AGI is like listening to my pharmacist talking about the color of the pill thats gonna make humans live forever

re Carmacks comment, shouldn't we develop programs/AI that read the existing publications/Big Data first if the information is already in there?

it would hard to say that you can't program it to do stuff when talking about reaching human level of consciousness. The problem with that is that humans are programmed to do things also. If you don't program the AI to do things it would be like someone who had never learned anything about the world. Caveman AI. It would be interesting to see a learning AI choose what it wants without programming goals. I was also just thinking I would like to see how the AI responded to a type of meta verse where there would be rewards and it be given basic education and would be rewarded with a meta currency and see how it would learn to value it's time in the meta and give it options to interact on its own. But value things that it can spend it's meta currency based on human entertainment and luxury items that would work as reward and see how it responds to the universe.