We're going to run out of marine animals real quick. They should have started with plankton

Hope anthropic names their next model harpoon

_- "Our mission is to ensure that artificial general intelligence benefits all of humanity"_ - Literally partners with Rupert Murdoch

the Anthropic bot calling itself deplorable and recommending it get deleted from the Internet is super interesting. It makes superalignment seem at least plausible

anthropic once again showing why they are the leaders in ai safety

Is it only me who feels that the Antrophic paper is a watershed? I mean, as someone who has studied his neurobiology is evident that something akin to even this very limited "SAE" model for the human brain would be deemed as a HUGE breakthrough in neuroscience. Obviously, this is only a proof of concept, and if it can actually be refined and perfected the implications (positive and negative) are almost self-evident.

5:45 “Gemini 1.5 pro doesn’t have the rizz of gpt-4o” isn’t something I thought I would hear you say😂

"Deeply deceptive Ai that hated itself" ✌️😔 real bot

The Anthropic research paper findings are some of the craziest discoveries I've ever seen in the AI domain.

Did anyone notice that there's currently a Golden Gate Claude version available for a limited time? If you go to Claude and look in the upper right, there's a Golden Gate Bridge icon. If you click on it, you can talk to the altered state version referenced in Claude's tinkering with the model research paper. It's crazy!

Yes, a bed-time story

Computerphile recently had an episode talking about a paper discussing the shapes of the curve of the efficacy of future training and that being logarithmic instead of exponential due to lack of data. I’d love to hear your take on that paper.

Your section on Anthropic's mapping of their model is the most interesting thing you've talked about on this channel -- and that's saying a lot. Such insights and control open up more possibilities in my opinion than just scaling compute and data.

15:02 “That’s a pretty abstract concept, right? Making an error in code.” I dunno-I don’t think it’s any more abstract than, say, the concept of, say, a mistake in grammar, which these language bots are pretty good at detecting. 17:17 “It suggested an internal conflict of sorts.” 18:18 “It sheds light on the concepts the model uses to construct an internal representation of its AI character.” I tend to find statements like these a little jarring, especially in connection with these language models. I wouldn’t say that’s an “internal conflict”-which suggests some psychological drama roiling under the surface. It’s simply two different verbal outputs that are possible, given the training data-you ramp up “the feature related to hatred and slurs to 20x its maximum activation value,” get hatred and slurs (no surprise there), and then get the verbal output that might follow what was just said. (A person who has just had a hate-filled outburst might follow it with a similar contrite, self-punishing response.) If there is any “self-hatred” going on with these AI models, I’d be _really_ surprised. And these models don’t _have_ “internal representations” of their AI character or anything else. (That’s an, to me, unfortunate carry-over from cognitive psychology.) Maybe some people would consider the word embeddings to be “representations” but I wouldn’t consider a list of features, no matter how extensive, of, say, a cat, to be a “representation” of that cat. It has, at best, weights and biases, which give rise to some verbal output when asked to describe its AI character. To me, it’s just muddy wording on the part of the people creating these models, which gets in the way of analysis.

Ad this whale comparison: Americans will measure with anything but the metric system

Anthropics work to understand the models will give them a significant advantage

0:00 (!) 5:56 Math, thought, contemplation 9:51 AI impact on photography art and industry 12:47 on undrerstanding Anthropic LLM inner workings. #monosemanticity 18:24 on AI deceptiveness 22:30 on the voice similarity to Scarlett’s from the movie “Her”

That Claude response makes me think we should hold off on giving these things full agency inside a robotic body until we have a much better grasp on what's actually going on in their minds. The last thing we want is a robot hearing some words it doesn't like, then deciding that whoever said them should be eliminated.

5:58 did you just say "rizz"

thanks god we have you to explain that anthropic paper. was waiting for this one more than anything 😅

You are unparalled. Your intellect and insights are a blessing to navigate the difficulties of the AI landscape.

Thanks Philip. I'm exponentially in awe these days.

... Accuracy is everything in real world situations... we're not there yet and the risk to go all in with a broadly available mixed mode LLM is still too great for many professionals to accept...ask me a year from now and perhaps this gap will have been bridged. I keep thinking that at some point any MMLLm will generate output of a quality that to utilize any of the current crop of "built on AI" apps will seem nonsensical...great vid as usual.

Congratulations to anyone who had "First AI that hates its self and wants to die" on their 2024 bingo card

AI researchers know very little about what "real people" do in their jobs, and this paper just reinforces that view.

Anthropic's Mono semantics paper is indeed very fascinating.

props for the links with fun/relevant captions, first time I've seen a non boring link description. Keep it coming!

Americans are counting in marine wildlife now, god help us 🙏🙏🙏

@05:24 "Otherwise this video would be way too long" No such thing for you mate! ❤ This was a very cool video, covering things I wouldn't have read otherwise🤝🖖❤

The interpretability paper is the most mindblowing/promising/hopeful thing I've heard in a while. This is exactly the direction I was hoping we would make progress in (in my vague non-expert-y way). I rememeber having read about a single neuron controlling the opening and closing of quotation marks and thinking: If we can find the neuron/neuron complex for "being deceptive" (reliably in every model), we have one problem down. Edit: This is a big feather in Anthropic's cap, when it comes to claiming they are responsible. Actual concrete results showcasing that they put resources in safety-relevant research. Big kudos!

Anthropic for the goddamn win, man!

"whale sized model" -- i thought he was saying that the compute cluster used to train the model was this "relative" size... not the model itself

the thing about monosemantics was insane. The fact it can detect incorrect coding is incredibly interesting and promising. Watching us creating digital neurons mimicking our own neurons is so so awesome so much in this video as well that's endlessly interesting. I could spent an hour writing up all the things that make this video interesting

If compute has to grow exponentially in order to achieve linear improvement in model power, isn't that diminishing returns by definition?

Thank you for staying on top of all this information from so many sources and sharing that time and work Phillip, take care of yourself, peace

Nice one! Great timing.

the bitter lesson still remains

Disappointing to learn GPT-4o's voice & vision is delayed by months. Maybe OAI isn't as far ahead of the other AIs as is widely thought. We may well see an open source solution widely available by the time 4o's is released.

So glad to hear benchmarks might get fixed

Exponential growth is so hard for the human mind to grasp. I love it, makes the future exciting!

Quite the opposite of what we’ve seen previously, OpenAI apologises and Google ships! I think the writing is on the wall at this point that the skeptics of this technology will continually be proven wrong. Everything we build now should be in anticipation of constantly improving underlying model capabilities (something I mentioned in my speech yesterday at our industry conference!) Thanks for the update Philip!

I wonder if we're missing the forest for the trees with the whole MMLU relevancy issue. Isn't the bigger question do we need to entirely rethink what it means to test for "intelligence"? I feel like these current benchmarks are akin to testing a calculator on its ability to do sums and declaring it a genius.

People don't know what's coming. A multimodal system + robotics has the capacity to disrupt any physical job. I can’t believe people are not talking about this! This is crazy 🤯🤯

To be clear, buiilding exponentially larger models and then getting 2x increase in accuracy is not exponential increase in accuracy. If anything, that's sublinear growth. An exponential growth in accuracy would to built the same size model but performing 2x better, with a clear path to then perform 4x better and so on. So far, the scaling down of large models to slightly smaller models is a one trick pony, you cannot repeat that to get the compounding effect, as far as I can tell.

@aiaxplanied by far the best videos on AI updates. You keep on being my weekly must watch Phil ✊🙏 thanks a lot for these overviews

The crazy thing that happened is audio-text- to straight audio in.. 🤯

Gotcha! Looking forward to your next video.

The internal conflict when they ramped up Claude 3's racist node, shocked me. I cannot believe this. And frankly, it gives me a lot of hope?

This is absolutely insane. I’ve spent the last 16 months working on an AI startup based fundamentally on that conceptualization technique that Anthropic found. Going to drop a video on this soon this is wild and extremely validating to see!

Excellent analyses. Thank You!

I'm very glad to see some progress into solve the black box problem =)

Another amazing video! Thanks Philip!

Just had a thought about how to visualize Anthropic's study I thought was fun. You know how the BFG gathers though fragments of imagination, then mixes them together to form dreams? Those fragments are the nodes, and the dreams are the patterns.

There's lots of KZfaq content that I have to fast forward through to get to the part that I was interested in at the end. I've started to regard that content as a kind of click bait. I never fast forward through these vids here. It's all gold.

Thanks! Excellent content, as always. 🙏🏼

Dude of course they are going to say there is no diminishing returns. They need the AI hype to keep the profits coming. Im very skeptical.

The Anthropic deepdive was particularly interesting.

Another great video. Keep it up. You are the best resource for keeping up to date on the details of AI. What would you recommend for someone wanting to get into the AI space, eventually? Someone with very little IT skills and low math skills. (I know it's a loooooooong road, YEARS LONG). I am starting with compTIA, security+, network+. However, I want to steer towards AI.

so in a nutshell 1)more computational power will lead to deeper more fundamental corellations discovered by LLM's 2) anthropic found that LLM's work by finding patterns which when reapeated across data lead to universal corellations like basic universal underlying thruth's right?

The only channel I have with notifications turned on

Great update - thank you :). During the Google conference, I was surprised that they did not present a new model. I use Gemini 1.5 for translation and brainstorming, but the hallucination level is too high to use it for other work-related tasks. At the moment, I trust GPT-4 and Claude 3 Opus more.

Great video! So interesting.

Thank you in advance. Just know your AI news overview is the best.

Fascinating update, thank you!

how sad that the discovery of AGI might turn out to be "more compute" and these larger and larger computation are done by wealthy corporations....

Best channel on AI. SO INTERESTING EVERY STORY! KEEP IT UP!

"We are nowhere near diminishing returns when scaling compute". Of course he is going to say that to INVESTORS after flushing down billions of their money in scaling compute

While the capability of LLMs may continue to scale exponentially with compute, the question is when will the availability of power generation, transmission or distribution hit the wall?

Great video as usual

17:32 "Hat is it like to be you?" ~~~ Love it. This is the kind of thing I'm most interested in.

Great video! The anthropic paper results were fascinating

This was such a fantastic report. This is so fascinating!

Great video - here is some algorithmic boost Comment

I’m following you because you actually gave me the news I was wanting. Ty

Excellent video. The importance of anthropic work on interpretability cannot be underestimated, as it attempts not only to describe, but also to manipulate the network. CloseAI should be ashamed of itself for not focusing on interpretability!

As a trainer, I can say it's not simple. There're hierarchies: prompters (who also review the responses); reviewers (of the original prompt and and the responses and reviews), reviewers of reviews (all above, but adding alignment), and a final arbiter manages all the previous stages to feedback to the original prompter. Each stage has a large amount of rubric associated. And ironically, they also use AI to determine the efficacy at each stage. After all this, then a given prompt gets fed to the to AI. Call it AI, LLM, AGI, EGG, wheveter. It is a black box to 99% of people that work on it. Training AI is not trivial.

reminding you that your community is highly appreciated!! 🙏

Amazing video!

Thanks!

I was waiting for this one, anthropic report is big, too big for me

Thanks for the great summaries of these papers that some of us don't have the time/patience/focus/intellect to read and comprehend.

So because of Gary Marcus we have a confirmation from OpenAI employee that GPT-5 will be here in November.

i remind you. understanding inner workings - helps ramping up capabilities. so safety is what you actually "do" with that info. and i don't think they are prioritizing safety RN )

Marine life as a metric? Seriously? That’s idiotic and completely meaningless for comparison purposes. Which is probably why they did it.

I'm so grateful for these videos. No idea how else I would be able to keep up with that is happening.

Great, less hype and drama and back to papers 💪🏼

Great content. Thanks!

They’re gonna run out of data to train the models

I'm in prepress myself which is similar to the photography field in going through multiple files for specific details. This is something I'm actively looking to implement. We have just started using an algorithm based process through programs such as "switch" and "pitstop" while I wouldn't feel comfortable relying on LLMs for going through this, I would feel more comfortable setting up a more rigid system using llm assistance to develop the process. But please please please look at this use case more in the future. My job literally depends on it XD

Love the channel and your enthusiasm when talking about these topics... nobody I know is even interested, let alone enthusiastic about any of this! lol

The huge improvement is when you have one account working across all of your devices and enabling you to pass information between them. This app should be able to interact with all of your apps so you can start to tell it "I want you to do ... " ... or talk to you while you're performing a task and telling you of another way. It should then start to record efficient ways of doing things, while keeping personal data private. Your ai could be in contact with what is the best way of doing things. This use could be growing the model. The next model can use old models to train them.

Ironically getting to whale level means to the limit

Had an idea, Doing statistical analysis of the output activations, similar to what anthropic was doing, but have a second model that injects activation into the other model, probing the model, and use it with permutations of the "ideas" of the model, giving it a sort of overseer that learns from the model and finds those gem "hallucinations" , didn't read the paper but maybe this is exactly what they are doing, if not, perhaps its helpful? on the other maybe its not such a good idea, and might spell disaster. ::: Writing this after watching the rest of the video, guess thats exactly what they are doing? can you clarify?

11:24. Professional in image-making industry here. What kind of photographer does really work like this? so absurd to value the quality of a photo from its metadata. Would be much better to have a capable AI that checks quality based on real image standards, even if "subjective", not metadata. So pointless to claim these advantages in a paper.

I keep coming back to this channel expecting to see you reach 1 million subscribers. C'mon people! This channel is amazing.

I wonder how good we'll get small models

The LLM benchmarks are just useless at this point...

17:09 They dialed up the hatred and slurs feature so it’ll also dial up self hatred.

Good one..!

the elongated inference time to get more out of models makes double sense with the small ones because if a 7B Lamma 3 or Phi 3 can become a GPT4-4o. that is way more exciting than even GPT-5. you can probably get that if you homogenise it to be a 5.0 with even more time and layering... its now about speed and size if that is true!