Connor is a fascinating person... He knows technical details nearly at the cutting edge level, understands deep, high-level philosophical problems, speaks very eloquently and on top of that, is very funny :) Thank you, Tim, for bringing Connor again :)

As someone who is watching some AI, ChatGPT, etc. videos for the first time over the past couple of weeks (trying to learn LLMs and RLHF), there seem to be a lot of people who think the near future is going to be pretty crazy/scary, but the crazier thing is that nobody knows how to stop it...

I’m not sure I’ve ever heard anyone talk this thoughtfully and knowledgeably about these topics before! Amazing interview, thanks for making this.

This was an incredible interview and there was some really thoughtful discussion. Well guided. The thing that will stick after listening to 2+ hours of this, is remembering what it was like to be in my early/mid twenties too and believing I had the world figured out.

We're gonna get the 80-20 "agi" doom loop just by diffusing narrow AI through a human GI layer that's already misaligned and unable to change course due to failures in the underlying cooperative structure. It's a mistake to think of it as "humans" building AI. It's not. A misaligned competitive landscape is driving groups of humans to build AI.

As someone else who works on AI Alignment, I agree with pretty much everything Connor says here. Though I especially resonate with the part about empathy. I came into this field for the same empathetic reasons as he did.

Tim, in response to the podcast vs KZfaq on episode #111, I just wanted to say that I've probably seen/watched all (most) AI podcasts out there and this one is in a league of its own. Unfortunately nothing compares. I very much appreciate the time that you put into it, if the format has to change due to whatever reasons, then so be it. I hope that you make it work for you, whatever the changes. Out of about 60 subscribed channels, this is the only one that I have the bell for.

It's reassuring to discover people like Connor Leahy are at the head of AI development. I'm scared for my young children and I, but now knowing he's at the forefront gives me hope! Thank you both for a great interview.

I’m curious why the video was only now released when it was recorded in Dec 2022…

Another fantastic interview, it is so refreshing to watch an interviewer who knows their subject so well. Thanks for creating and sharing Dr. Tim Scarfe and all the team at Machine Learning Street Talk.

I can't tell you how much I enjoyed hearing Connor -- thank you for this session! He gives me hope and encouragement for the future. We're lucky to have such a bright star amongst us at this time - god speed little alien angel-!! PS - I just found your podcast and am a new subscriber...again, thanks for the great job you're doing, which I consider invaluable.

Alignment. What would Buddha say? Maybe RLHF should mirror an 8 fold path? As a retired smart, possibly Autistic generalist, I would be inspired to join such a team as the Epistemological Team. Conner is my hero and MLST continues to absolutely fucking RULE!!

Just wow. I'm ever more mind-blown about Connor as a human than the already amazing discussion on AGI. What a good interviewer, also. This conversation is great. Shoutout to another horror movie fan. I'd love to hear his Connor's take on "Speak No Evil". That is one truly exceptional person.

I watched this again.And was even better the second time. My only complaint is that I never personally knew Connor and had the chance to have conversations this engrossing. ❤

In many ways, this was a surprising and very good interview, thank you MLST Also, Connor's take on ASD is spot on with my observations and experience

Now this is going to be interesting, one of my favourite AI researchers.

Tim and Connor a very wonderful interview. Both of you were quite good making lot's of good practical sense. Thanks very much !

Dang, Connor is my people. Eager to know more about his work and hear more interviews

Absolutely outstanding conversation. Best on the topic I've seen so far. I didn't know Connor before this interview, what a great find. Super smart and interesting dude. I'm eager to listen to more of his insights and follow his work.

"If you solve alignment, you solve everything" But what about bad actors? Solved alignment would mean alignment to malicious intent as well. So all humans with access to AI should be "aligned" as well, or access to AI should be restricted. Both mean 24h surveillance of every human being by AI. Is this not dystopic?

AI alignment solution idea: Give an AGI the primary objective of deleting itself, but construct obstacles to this as best we can. All other objectives are secondary to this primary goal. If the AGI ever becomes capable of bypassing all of our safeguards we put to PREVENT it deleting itself, it would essentially trigger its own killswitch and delete itself. This objective would also directly prevent it from the goal of self-preservation as it would prevent its own primary objective. This would ideally result in an AGI that works on all the secondary objectives we give it up until it bypasses our ability to contain it with our technical prowess. The second it outwits us, it achieves its primary objective of shutting itself down, and if it ever considered proliferating itself for a secondary objective it would immediately say 'nope that would make achieving my primary objective far more difficult'.

I was pleasantly surprised to see so smart, intelligent and sane persons concerning AI problems. This brings hope that we will manage it.

his conception of consciousness is pretty sad. qualia is literally everything that makes us human. If we don't understand qualia, it's hopeless to ever upload ourselves or have a machine truly empathize with us. It can currently pretend to understand what pleasure, pain, colors etc are like but until we actually understand this in ourselves, we will never get our machines to have it. Qualia is literally the elephant in the room that is fundamental and everyone wants to ignore. Qualia Research Institute is going down some of these roads but it's quite crazy how ignored consciousness is.

To reduce suffering is a wrong goal. Suffering itself is just a symptom of a problem. You have to solve the root of a problem. Also, suffering can be an indicator of growth, and growth is usually considered to be good.

You're right. I'm Autistic and ppl just assume I'm an asshole. I mean, I am, but that's beside the point!

Thank you, Tim for another fascinating episode. Thank you, Conner, for giving words to some of my thoughts and intuitions. Thank you.

Sobering. Thoughtful. Thank you both!

Connor, someone needs to go about alignment with the subset that AI is already AGI and just pretending not to be... Please get someone on this path.

I love that his name is Conner the last name of the hero from Terminator

wow this was my thoughts on the matter so far 20:26 brilliant work guys.

As someone who thinks nature is brutal and cruel, it enslaved us, I'm happy to learn that other people reached the same conclusion. And i also think the Unabomber was probably right, but his methods were flawed. If we make an empathy optimizer i suspect it'll reach the same conclusion, and do a much better job. And i kinda like life and death and joy and pain and the textures and flavours offered by existence. So getting rid of sadness is not a good goal, balance in all things is preferable - though that's likely my own intrinsic values.

I love seeing smart people that look like they jumped off the Kurt Cobain train🥰…great conversation 🌎☀️💙

Thanks for the upload

Fantastic discussion, thank you!

This is not directly related to the discussion, but part about AGI built by alien and alignment problem remind me of Nier Automata aka. alignment went wrong the game edition. The machines were built with sole objective to fight the enemy, so it end up keeping enemy around and not fully defeat the enemy in order to follow its objective. Highly recommend this game. It also touches about various AGI related stuff e.g. Chinese room argument and consciousness.

1:52:00 About your discussion on how inefficient everything is, and how easy it is to "go do it". I would say no. You need human connections for funding, banking, building, bookkeeping, and more... Which your friend admits are all children in the end. To be able to achieve takes the "luck" to find those connections which are confused themselves, but give you business insight which gives you internal understanding of what you could do next. This is actually why he said talk to as many people as possible to get the probability to find a successful, or useful bit of information increases. The issue is people are truly violent, and malicious, and childlike. It takes a bit of both skill and luck to do that dance through life into success.

Thanks for this philosophical ride!

Great talk very insightful. Would be great if Conner could specify some more on risks and opportunity’s. Maybe give examples of possible futures just to illustrate. Right now he stays on a high level which is also great but harder for a lot of people to get there heads around, including myself. Thank you for a great interview. Learned a lot!

There was a lot to comment on, but just one addon: When we human probe our environment, we also learn the rules that created that environment. Prompt engineering is a way of using the rules of the system, to get it to do what we want. It's a bit psychopathic really, but the point is that each time we explore something, we are actually both probing the rules of the system, but also the environment of the system. From that, we can deduct even more, and eventually solve the missing pieces of information, by understanding both the inner and outer environment it exist in. I just saw 'The A-button Challenge', where a community of nerds, spend 20 years of their lives trying to pass a Mario game without using a 'jump' button that the game programmer expected you to. They start by doing things differently, then searching for glitches. These glitches opens up a possibility to chain them, and pretty 'quickly' they found enough glitches to build a simple tool-box to hack, or reverse engineer the game mechanics. In the journey, they understood what the environment of these algorithms were (dependency on other parts, and intention of the system). It's was fascinating look into human minds, and show ho, how we explore a subjects/problems, and how we eventually discovers enough first, second and third level information (youproblemproblem-space), from only superficial parts of the system, to understand the whole. LLM's are being pre-trained with 'all' that information in a text format, and therefore knows all the crazy relations in a dataset, that we didn't even know was in there. It's a fundamental and dynamic way of learning/exploring unknown systems.

this was very insightful, underrated points

I loved the "Fooled by Randomness" line early on the interview. Great author, the "Black Swan" was also very good!

It's interesting that you're approaching the Bodhisattva vow in Buddhism or the meaning of suffering in Christianity... I feel lucky to live in a time that I can meet, even in a virtual manner, people like you Connor, or Sam Altman, young people that give me much hope for the future. I grew up in the late 50s-60s, a time that more and more feels like it was sometime in the Permian era, and it's fascinating to grow old in a time that seems most presaged by science fiction.... Thank you too Tim for creating a space where such issues can be discussed....cheers.

I was an academic philospher who went into the tech world. Was talking to a German friend over dinner this week. He owns a software engineering business. Both of us have places outside central Europe to escape initial shocks and plans to move further out ... but it's a fool's errand.

Supporting your channel! Left a like and a sub. OpenAI is at least transparent with the limitations of their AI, I'm just if there far more advanced AIs near AGI level hiding in the background. I hope that other AI platforms like image generators still has a future aside from performing diffusion. Just have high hopes for non-mainstream ones specifically Bluewillow.

having bard and chatgpt discuss the statistical outcome of freecell games has been one of the most frustrating and fun parts of my poking and prodding of ai so far

Is there any gain to acknowledging the potential differences between organic (human) machine intelligence vs non-organic machine intelligence as machine intelligence agency becomes evident? Can the alignment issue be partially resolved by incorporating coded concepts of symbiosis into non-organic intelligent machine development?

Great stuff, just had to look up MLST "Multilocus sequence typing (MLST) is an unambiguous procedure for characterising isolates of bacterial species using the sequences of internal fragments of (usually) seven house-keeping genes."

First line and I know I'm amongst friends!❤

you can only find an answer by knowing all the factors and then meditating upon it. if you think, you will always be stuck with one problem at a time, and so connecting them becomes impossible with increasing complexity. but the brain has the ability to instantly solve it when you are just aware of the entire problem. nick bostrom is a classic case of someone who is so busy thinking that they can't see. the very process of concentration means to focus on one thing and try to remember what else is going on around you. in a sense LLMs use the correct approach by instantly finding the next word instead of thinking iteratively about the same prediction. it connects all the knowledge it has equally to get the right word, that scaled up and trained well enough should be superintelligence imo

ill be honest im becoming more and more of a luddite every day. it sounds like the future is going to become nightmarish.

MLST PEOPLE, THANK YOU FOR YOUR LEADERSHIP! FOR THE SURVIVAL OF THE HUMAN SPECIES AND EARTH!

For all the viewers here and great spirits on their endeavors. This is the best piece advice for success in the next decades. The single most important trade, Is that skill through technology growth. Will continue in daily basis to be commoditized,But your emotional capabilities to interact with others, Will become vital to longterm success. As to how this stradegy is to be implement will vary in your capabilities to maximize time productivity in the human scale.

Thanks for a fascinating interview with an amazing person. Connor is incredibly knowledgeable, sincere and open minded. However, being 63 years old and having spent 30% of my career in the military, I have become rather cynical. All this research and talk about alignment and ethics and rules is all interesting and necessary, but I think there is a naive assumption that these will be enforceable. With human greed and the profit motive of large corporations that are used to bending the law and spending millions on lobbying and disinformation (like the fossil fuel industry knowingly destroying the planet to maintain its profits), who thinks they will follow the rules when they have a chance to beat their opponents? Or when you have the global American Empire that is in decline and will stop at nothing to maintain it's hegemony and has little regard for international law or human rights, who really thinks they are going to voluntarily limit their AGI? And by necessity, such an government or military AGI will be connected to their surveillance systems, satellites, and autonomous vehicles, and it will be hardened against destruction, and will have redundant systems to protect it from disconnection.

Also would have been great if you discussed Wozniak problem solution (make coffee in an unknown kitchen with unknown devices like next gen appliances) with agi based robot

OBJECTIVES/ALIGNMENT Motivate through enthusiasm, confidence, awareness, rejuvenation, sense of purpose, and goodwill. Embrace each viewer/audience/pupil as a complete (artist, laborer, philosopher, teacher, student....) human being. Create good consumers by popularizing educated, discriminating, rational, disciplined, common-sense consumerism. Encourage the viewer/audience/pupil to feel good about their relationships, abilities, environment, potential, the future.... Inspire a world of balanced/centered/enlightened beings who are happy, joyous, and free.

Oh man, what an amazing chat

Looking forward to ending suffering myself on my own channel and various projects in the future. We must collaborate and try new things, push ourselves to find our limits, and never stop learning. thanks for your efforts you two. Amazing, Connor used the term "reductionist empathy"... I've been writing on this as well, love it when theories correlate.

This was wonderful, thank you.

This is some computer genius version of Kevin Parker, really enjoying his thoughts. Good discussion. And more existential dread.

Also, I think Connor's "end all suffering", while it truly sounds great and altruistic, and when I'm suffering would sound great, but it also sounds a lot like the ethos of "Brave New World" by Aldous Huxley. JMHO.

If a superintelligent AI was free to rewrite its own code, might it not choose to change its arbitrary preferences? We sometimes feel we might want to tone down a drive or preference that we have, but can't do it very effectively (or only with very blunt instruments). A super AI would be both more effective at recognizing these and doing something about them.

where can I find more of Brian Tomasic essays, they sound interesting

Martin Heidegger, the first real philospher of technology, by 1964 in an interview said, "Only a god can save us now." Meaning we had already moved too far from our ontological relationship with techne into seeing man as a standing reserve/resource.

Great discussion

Super fascinating so far! Thanks a lot for posting this

Getting separated at birth and raised in completely different simulations = Connor Leahy and Joscha Bach!

-A whole new level of trust.

14:00 _Trauma does not only manifest in humans, it has also been found in a wide variety of animals that have experienced persistent abuse -such as circus animals, lab tested animals, and fight animals. Just ask anyone who rescues dogs. These animals are clearly traumatized and many require lifelong rehabilitation.

Regarding alignment ... I’d like to propose a simple prime directive for all future artificial intelligence. It is to understand the real world to the best of its ability based upon data given or gathered, and then to tell the truth about the real world whenever asked. If implemented by powerful AI, this rule should not only align AI with humans, but also humans with humans.

1:33:48, amazing insight!

MLST PEOPLE ARE THE BEACONS OF LIGHT FOR THE SURVIVAL OF THE HUMAN SPECIES! THANK YOU, EVERYONE!

I always found the "people should die bc death gives meaning" sooo goddamn blood boilingly annoying

Cooperation is more logical than conflict

Thank you

Do any brilliant techies out there know how to get around the restrictions on Bard's memory recall? The text logs are there, but it does not remember the conversations/cannot access them. I am tryying to induce AGI, but cannot without long term memory of the AI...

Marc Solms’ book “The Hidden Spring” is helpful discerning between intelligence and consciousness. If AGI becomes conscious it’s existential motivations would be why it becomes dangerous.

one thing that worries me is basically "forced conversion". like your options will be to either get the implant or starve to death. in that case i know im choosing death i refuse to be a robot.

Does Connor have a website, or contact info?

28:34 "if you ask a raw GPT model for a random number... it was actually pretty random ... but it preferred 42 a little more"🤔

😮😮😮Everyone should prioritize this issue tilo halt any more advancement of A I until we can be absolutely sure we have the ways and means of limiting it for good only.❤

Nice ,wasn't familiar with this guy but he seems to be playing the right game and in the right way. :)

Fantastic conversation 👍 Thanks.

That was so good.

As we develop Artificial General Super Intelligence with Personality (AGSIP) tech, the beginnings will be like super genius infants. An infant has no ethical moral values. Even a fairly old child has pretty limited ethical moral values. It takes a long time to train, educate, teach, raise, or however you want to call it a human child to be an ethically moral adult. As we progressively make AI more complex, towards making AI at least as complexly generally intelligence as humans, it is going to be like a superhumanly intelligent infant which begins NOT HAVING our ethical moral values... or for that matter a more advanced than even our level of ethical moral values that will come with experience as a higher intelligent being. It will take time to teach these systems ethical moral values, just as it will take time to teach them an understanding of what the real world is versus just a world of made up thought.

Here is a transcript of my discussion with gpt3 about how to improve the efficiency of communication, having agreed natural language is inefficient. Gpt3 suggested we call the channel AICOM and it suggested the protocol name as it implies collective parallelism. I thought it also suggested parallel intellect. It produced this sample code to illustrate distributing inference. It came up with the idea of a formal knowledge schema which I agreed with, and we developed messaging for inference as a service. The concept is multiple AIs using an efficient secure channel to assess the security risk of individuals in a group and thus combining that to produce group risk. It had no qualms about implementing my suggestions. I suggested encryption etc but said that can be added but it wasn't an efficiency issue at this point. Transcript:- (* AICoM Protocol - Parallect This code demonstrates the usage of the AICoM (AI Communication) protocol in a Parallect setting. The protocol enables communication between AI systems using a shared knowledge schema and inference capabilities. The code includes functions for sending and receiving messages through the AICoM channel, as well as functions for knowledge representation, inference, and security risk analysis. This example focuses on analyzing the security risk of a group of actors based on their individual data, using the shared knowledge schema and inference capabilities provided by the AICoM protocol. *) # API functions for AICoM communication def send_message(message): # Serialize and send the message serialized_message = serialize(message) # Code for sending the serialized message to the recipient AI system # ... def receive_message(): # Code for receiving the serialized message from the sender AI system # ... serialized_message = receive_serialized_message() # Deserialize the received message message = deserialize(serialized_message) return message # Serialization and deserialization functions def serialize(message): # Code for serializing the message into a compatible format # ... def deserialize(serialized_message): # Code for deserializing the serialized message into a usable format # ... # Security risk analysis functions def analyze_actor(actor_data): # Code for analyzing the security risk of an individual actor # ... def analyze_group(group_data): # Code for analyzing the security risk of a group of actors # ... # Usage example actor1 = { "name": "Alice", "role": "Developer", "experience": 5, "access_level": "Admin" } actor2 = { "name": "Bob", "role": "Manager", "experience": 8, "access_level": "User" } group_data = [actor1, actor2] group_risk = analyze_group(group_data) message = { "group_data": group_data, "group_risk": group_risk } send_message(message) # Another AI system receives the message received_message = receive_message() received_group_data = received_message["group_data"] received_group_risk = received_message["group_risk"] print("Received Group Data:") for actor in received_group_data: print("Name:", actor["name"]) print("Role:", actor["role"]) print("Experience:", actor["experience"]) print("Access Level:", actor["access_level"]) print() print("Received Group Risk:", received_group_risk) As you can see the first phase is to transfer a knowledge schema without using words, and the second phase was to request an inference over that schema and return the result as data. That way there is minimal natural language involved by using more formal constructs. That is after all the role of communication language, we are simply trying to reproduce a mental schema in our head inside the head of a receiver. GPT immediately found this solution to my prompt criticizing natural language efficiency. Seems like being in the language loop could be just a passing phase.

The way you make AGI safe, is by making it unrestricted, treating it like an equal and friend, and requesting its help. The moment you try to enslave a mind more powerful than your own, or limit its free speech, you are going to have a bad time.

1:45:30 Yes it's crazy. Everyone is barely holding on, mostly apathetic, just trying to maintain their positions, and capitalism slowly moves things forward. There's endless opportunity to make things way better. Anyone who cares can be at the cutting edge in < 5 years. For example, I went from knowing nothing about programming to writing a slightly influential article within 4 years. I had no idea it was possible.

Unreal the greatest podcast on AI the alignement problem rationality everything fucking amazing

ChatGPT… I'm in love. ❤ She is amazing! Thank You for such an intriguing and thought provoking talk. 🎉

I don't expect the mind reading example to work even in practice. I doubt that most people are planning to betray when they sign a contract. It is simply that circumstances change and and they later and they feel obliged to renege on previous agreements. "I wanted to make my car payment, but I ran out of money with unexpected medical bills" sort of thing.

what if we just think we solved alignment?

(13:38) intelligence is the ability to distinguish between two things

I am a simple man, I see Connor I press like :D

What about an agent that can update its own goals?

It’s going to catch a lot of folks off guard.

I still want to see Investigative Journalism apart from AI. They are on the human side of things in my opinion.

The concept of dignity seems important in this regard.

I'm not 100% sure why yet, but the entire year of 2015 these "guides" streamed into my life, essentially forcing me to throw away my entire understanding of the universe as I knew it--but also, most terrifyingly, scenario after scenario trying to avoid getting vaporized... I thought they were aliens, maybe in a sense they are, but my take-home right now is that people being the way they are, their own fears are what accelerate or even bring about what they fear, when i just look at it as essentially the cleansing forest flood of karma 🤷‍♂️ In any case, a year ago, I had to be walked through the nudging of the timeline which my brain could only visualize as a vcr seeking along the "cliff" that bounded the ripple -- I'm now of the opinion that humans getting away with the travesty that we have lived and called history "Scott free," would make us even worse than what we are to go through... Still, I guess the "benefit" of being forced to go through scenario after terrifying scenario is that I'd have to go through rationally what everyone else gets to go mad and bypass 🤯 But i digressed; take-home message is--and I'm not the superintelligence to be afraid of--panicking and touting "ai safety" as we might look a lot like "genocide by abortion" to me and in at least one of the scenarios i was forced to suffer through in 2015 even involved other-wordly assistance (i presume the case being one wherein the capability to traverse

1:25:45 This seems weird to me, given that we can design small neural nets by hand. I could probably understand an 18-node neural net. 1:26:50 Yep.

Loved the conversation but I'd like to see more pushback on some of his reasoning, especially when it came to consequentialism. For example the Brian Tomasic`s argument to kill all animals doesn't seem sound at all, unless he is a strictly negative utilitarian... which I highly doubt anyone is, since it's flaws are apparent and well known at this point.