NEXT VIDEO: "How AI Learned to Feel" kzfaq.info/get/bejne/etWmaZt72K-nZJs.html SUBSCRIBE: www.youtube.com/@ArtOfTheProblem?sub_confirmation=1 WATCH AI series: kzfaq.info/sun/PLbg3ZX2pWlgKV8K6bFJr5dhM7oOClExUJ Timestamps: 00:32 hofstader's thoughts on chatGPT 01:00 recap of supervised learning 01:55 first paper on sequential learning 02:55 first use of state units (RNN) 04:33 first observation of word boundary detection 05:30 first observation of word clustering 10:10 sentiment neuron (Ilya & Hinton) 12:30 transformer explaination 15:50 GPT-1 17:00 GPT-2 17:55 GPT-3 18:20 In-context learning 19:40 ChatGPT 21:10 tool use 23:25 philosophical question: what is thought?
@michaelpoblete14159 ай бұрын
Andrew Ng and Geof Hinton already clashed on twitter, quite confrontational tone, Andrew seems to be taking the side of Lecunn.
@masudahmad8537 ай бұрын
@@michaelpoblete1415
@masudahmad8537 ай бұрын
nice skills
@user-fj9hf4bu9f7 ай бұрын
please reconsider the background music and noise levels, it really takes away from an otherwise interesting video.
@John-D.3 ай бұрын
HAL, Reduce Music Volume 🤖
@clamhammer24635 ай бұрын
As an AI researcher and developer, this is the first video that I did not leave thinking that the author was just saying words without second thought. There is so much misinformation in this space stemming from the fear of the ramifications of AI that are a result of the negative feedback loop of those same people. Well done.
@ArtOfTheProblem5 ай бұрын
I really do appreciate this comment, i work so hard on these and sometimes i think i'm crazy - the garbge funnel of ai coverage is brutal
@RyluRocky5 ай бұрын
😂 yes other AI channels just throw the word “compute” around 300 times.
@ArtOfTheProblem5 ай бұрын
YOU'LL BE SHOCKED AT WHAT......@@RyluRocky
@Joe-jv5mm4 ай бұрын
You are now to Chat GPT 16
@Blueprint4Murder4 ай бұрын
@@ArtOfTheProblem Are you too going to kiss now? Card catalog 2.0. Do you guys remember those old newspaper cell machines? Where you could search for a news headline then look up the cell for the news paper pages related to that search? That is what AI reminds me of. A more advanced card catalog. While I am sure it will be great in that function and even in mimicry of humans I don't believe it is thinking. Most humans are so narcissistic that they believe animals do not think and yet those same dullards are convinced ai does because it is a popular narrative.
@belibem9 ай бұрын
There only a handful of youtube channels that can make such concepts accessible to everyone with a curious mind (e.g. 3blue1brown, veritasium, Sabine). Art of the Problem is one of them. Brit you are a legend. Thank you for giving us this series.
@ArtOfTheProblem9 ай бұрын
Thanks for sharing, I really appreciate it.
@rikvermeer13259 ай бұрын
I'd like to add "Artem Kirsanov" (neuroscience, neurons, manifolds) and "Anton Petrov" (science, physics) to the list.
@rikvermeer13259 ай бұрын
@@ArtOfTheProblem subscribed for more :)
@ArtOfTheProblem9 ай бұрын
yay welcome to the party! sorry my output is slow though@@rikvermeer1325
@T4bboo9 ай бұрын
@@ArtOfTheProblem Man absolutely great video, I am new to your channel and instantly subscribed... only drawback for me personally was background music... those annoying tones sounding like from 80-90 games made me download your video use AI to distinguish between narration and background music to get rid of it without losing narration and listen adjusted version offline.
@PotatoCider7 ай бұрын
please never stop teaching on the internet. i can tell you have an intense passion in learning. im in love with how you explain concepts and their connections. feynman would be proud.
@ArtOfTheProblem7 ай бұрын
this makes me so happy thanks for saying that. I'm sitting here surrounded in papers working on next video
@lakazatong7 ай бұрын
@@ArtOfTheProblemthat. could not have said it better. it really was eye opening in this hot field. that begs the question, are there alternative models for reasoning? the fundamentals today are, as you explained in the video, prediction and attention learning, that is, deducing the next action/letter/whatever given the previous states of the world/space/whatever, or to be exact, the previous mental states of that, and learning how to filter out and fragment the data into a meaningful way to, to be honest, only aid the prediction my question is, other than prediction, what could it be? I'm very fond of seemingly simple and Impactful philosophical questions, and this one really hits because then if the answer turns out to be "not really" then... our brains are just wired to be good just at that honestly that would not be too groundbreaking when you start digging in how the brain works together with the body to make up for a human I'm really just writing my thoughts here that also brings the good old question "are we just monkeys with bigger brains?" and by the looks of it yes, but also the inner workings of the brain-body connection may just be the key difference truly an interesting topic while I'm at it, I see little to no video/interest in one specific approach we could tackle machine intelligence and that is, taking our brain, and building it with silicon and wires building the hardware to be the brain itself would you be interested in such topic? unfolding your findings and sharing them as well as providing a state of the art? that would be awesome
@hello-rq8kf4 ай бұрын
yubi yubi
@ArtOfTheProblem26 күн бұрын
next video is finally done! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@PotatoCider26 күн бұрын
@@ArtOfTheProblem yaay
@Just4Growers5 ай бұрын
I have watched countless AI videos but nobody explained it like you. Many thanks.
@ArtOfTheProblem5 ай бұрын
woo! thanks, stay tuned
@ArtOfTheProblem26 күн бұрын
next video is finally done! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@gonzothegreat13179 ай бұрын
Perfect. 12 years ago or so I learned about RSA from you. I'm glad to see you are still producing quality videos! Thumbs up!
@ArtOfTheProblem9 ай бұрын
so cool to hear from OGs
@QW3RTYUU9 ай бұрын
That's about my story too. I would lay in bed with the iPod Touch and watch your cryptography videos, which in turn got me into owning bitcoin in 2011. I was so young and couldn't predict what would happend (my internal LLM was not trained enough yet) and I gave it all to some guy online as a donation for their FTP which had some hot stuff in it. The end is not the goal, it's the journey that's interesting. Your video got me emotional and felt right. I'll share around. Thanks for producing it!
@ArtOfTheProblem9 ай бұрын
ha, love the connections. I know the pain also@@QW3RTYUU
@myndwork9 ай бұрын
@@QW3RTYUUholy shit same here, 2011 is when I found this channel while trying to learn about bitcoin. This channel is a treasure.
@btm19 ай бұрын
same
@jay_sensz9 ай бұрын
Great video, but please fix your audio mix. The background music is way too loud. There likely is an option for audio ducking in your video editing program that will automatically lower the background music volume when foreground audio is playing.
@sams64sf7 ай бұрын
Yes, please. If the primary goal is to spread knowledge, I think the background music is taking away from that. I really want to watch this, but the music is extremely distracting.
@DanielGirardBolduc7 ай бұрын
It actually help me get into a thinking mood as the audio help me stay longer into my head when I question myself about specific topic mentionned in the vidéo. Might not be the case for everybody tho. I have ADHD and i'm auditive, if someone else with the same characteristic feel the same, let me know :)
@neohed7 ай бұрын
Hah, was about to post the same. The music is hideous and distracting!
@britcruise26296 ай бұрын
nailed it@@SimoneDesignsGaming
@georgewright20106 ай бұрын
I don’t understand why music is played. I gave up on the video after a few minutes.
@SecretEyeSpot8 ай бұрын
this channel changed my life about 10 years ago when i found the language of coins. By serendipity, I've found this channel again while im formally studying computer science which this channel inspired me to do. Thank you
@ArtOfTheProblem8 ай бұрын
So so so cool. love this comment :))))
@ArtOfTheProblem8 ай бұрын
i have to ask what's the cs dept response to AI? when I did CS in 2008 there wasn't a class or even a deep learning textbook
@ArtOfTheProblem26 күн бұрын
next video is finally done! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@jamesreilly76849 ай бұрын
Quoting one of the hero's of tech (Hofstadter ) makes this very well executed video that much more compelling. T9 (predictive testing) is the first really useful llm ish (not a word wheel but a predictive wheel from numbers) tech that end users experienced. The notion this simple premise is how it all works is nothing short of astonishing.
@ArtOfTheProblem9 ай бұрын
I know what a time to be alive
@ArtOfTheProblem26 күн бұрын
next video is finally done: How AI Learned to Feel! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@situranjankar9 ай бұрын
One thing i like about your videos is that you are very good at explaining things and the way of presentation is such that it is intuitive to any age group . Although there are thousands of videos and articles are available related to the same topics but this one thing makes your videos unique and best. Thanks Brit. Please keep making such videos in future.
@ArtOfTheProblem9 ай бұрын
thank you for sharing, I tried to do something new here and I hope it was worth it
@ArtOfTheProblem26 күн бұрын
next video is finally done: How AI Learned to Feel! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@Nick-Quick9 ай бұрын
TIMESTAMPS: 00:05 Neural networks learned to talk, leading to more general-purpose systems. 02:30 Recurrent neural networks (RNNs) use state units to create a state of mind that depends on the past and can affect the future. 04:52 Neural networks can learn word boundaries and cluster words based on meaning. 07:10 Language models saw limited progress until 2011 when a larger network showed the potential for higher performance. 09:34 Neural networks can learn language and complex concepts with minimal human intervention. 11:43 Neural networks struggled to handle long-range dependencies in text sequences. 14:02 Neural networks use distance in concept space to find similarities and adjust their meaning. 16:20 Neural networks with self-attention and fully connected layers can generate coherent and contextually relevant text. 18:27 In-context learning allows changing the behavior of the network without changing the network weights. 20:38 Language models like ChatGPT are more than just chatbots, they serve as the kernel process of an emerging operating system. 22:41 Training networks on prediction empowered by self-attention leads to a more general system that can be retasked on any narrow problem. 24:43 Deep learning community is divided due to differing opinions on the nature of AI's linguistic abilities and thought process.
@ArtOfTheProblem9 ай бұрын
thank yoU!
@les_crow9 ай бұрын
This is the most perfect video on the entire internet. I will use it to elucidate people on the topic. If I was rich I'd pay you big for this. Thank you sir.
@ArtOfTheProblem9 ай бұрын
thank you thank you, what a compliment. this comment made my day. If you want to support a tiny bit you can here: www.patreon.com/artoftheproblem but just thankful for this comment :)
@shadow_rune61788 ай бұрын
Explaining this in a way most people can understand is paramount, you have taken time out of a very large number of peoples day, and you have used it well.
@ArtOfTheProblem8 ай бұрын
thanks would love to know what you'd like to see next
8 ай бұрын
@@ArtOfTheProblem It seems to me that further exploration of this topic may run into an epistemological barrier, so what would you say about explainging to us how you got this knowledge? Maybe linguistics, philosophy or congnitive science? Btw, could you clarify what you mean the statement 25:15 (We either look at something that looks like though or It is though)? It kind of reminds me of Daniel Dennett's "Real Patterns". ps.: Loved this video sooo much
@ArtOfTheProblem8 ай бұрын
really?? so cool, please shoot me an email when you are done your thesis. I'm going to think about a good follow up@
9 ай бұрын
Wow, this is handsdown the best video I've even seen about this subject. Thank you! It is on my favorites now!
@ArtOfTheProblem9 ай бұрын
Woo! So so happy to hear this after 2 years of work on it.
@ArtOfTheProblem26 күн бұрын
next video is finally done! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@caspa79 ай бұрын
The Tie fighter sound in the last scene.. a stroke of genius. Very clever!
@ArtOfTheProblem9 ай бұрын
ha nice catch! glad some people like the sounds i was having a blast
@classicmartini6 ай бұрын
@@ArtOfTheProblem aka stampeding elephants ;-) Very cool. Oh. G'day from Sydney, Australia.
@sguzzygang2 ай бұрын
Just wanted to say I love you guys. I’ve been watching you guys for almost a decade, when you guys were the only computer science channel on KZfaq. You guys deserve WAY more attention than you’re getting
@ArtOfTheProblem2 ай бұрын
wow an OG! thank you so much. I was happy to see that this video did better than all before it. working on a follow up now focused on RL. thanks for the outreach
@sguzzygang2 ай бұрын
@@ArtOfTheProblemno problem! I love your work. I’m excited for your future videos!❤
@ArtOfTheProblem23 күн бұрын
just posted new video on RL kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@CutStudio49 ай бұрын
I'm the one solving the rubiks cube!!!!
@ArtOfTheProblem9 ай бұрын
Best dang cube solvin' hands in the west
@eastwood4519 ай бұрын
There is no Rubik's cube.
@CutStudio49 ай бұрын
@@eastwood451 its in the first scene, the one solving the rubiks cube is me
@KodakYarr9 ай бұрын
@@CutStudio4 I think it's a Matrix joke, but I don't really understand it
@eastwood4519 ай бұрын
@@CutStudio4 It was a Matrix joke 😄 Seemed fitting in a video about AI... God job on the very real cube!
@TropicalCoder8 ай бұрын
Very nice! ...well, except for the loud background music anyhow. Google kept pushing this video into my list over several days while I was very busy following something else. Finally I popped it up and saw why. Google knew that I would like it. Kind'a creepy, really how it knew it was exactly the kind of thing for me at this moment in time. It's like, it is following my studies on this subject and knew this should be the next chapter. Your video reiterates over everything I have learned so far and adds perspective.
@ArtOfTheProblem8 ай бұрын
thanks for this feedback, I'm so happy to hear it was relevant to you. interestingly enough this is the first time in 12 years google is recommending my video strongly...and it's amazing the difference. I used to get 1k views per day, now it's 20k usually people hate my music, but not all, so I try to find a balance
@TropicalCoder8 ай бұрын
@@ArtOfTheProblem I liked your music. It fit perfectly into the narrative. Just when I was hanging on to every word 'cause it was so interesting, the music became intrusively loud at some points and destroyed my focus.
@ArtOfTheProblem8 ай бұрын
i appreciate this, I know...i think the best is some music in and out at times...but not during moments of high cognitive load@@TropicalCoder
@TropicalCoder8 ай бұрын
@@ArtOfTheProblem Exactly!
@Naitry9 ай бұрын
A lot of people in my classes (as a cs major) look at me like a crazy person when I start talking about latent spaces, geometric concepts in AI, and Douglas Hofstader's views on cognition. This video makes me feel understood :)
@ArtOfTheProblem8 ай бұрын
you are one of us :)
@HungLe-ju6dv8 ай бұрын
we too look at them like they are crazy bruh😂
@KainniaK8 ай бұрын
At least Douglw Hofstader is able to play with something like ChatGPT4 and be like "Looks like I missed something". I don't see John Searle ever do that! I also believed that what chatgpt 4 does today was always going to be impossible on binary hardware, that just manipulates 1 and 0.s But I was clearly wrong!
@stephenowesney51737 ай бұрын
Excuse me if I'm missing something, because I haven't really dug deep into Hofstader's work; how is this proof that he has missed something? I think the basis for which he describes a lot of phenomena, the strange loop, is very much intact all throughout the concepts explained in the video!@@KainniaK
@Dr.SexySpice7 ай бұрын
Was this undergraduate or graduate? If undergrad then you expect far too much out of your classmates 💀, you are truly one in a thousand
@CatsAreRubbish5 ай бұрын
I just discovered this video and this channel. Wow! Informative, concise, not trying to push an agenda, just telling the story. Fantastic.
@ArtOfTheProblem5 ай бұрын
thank you, stay tuned for a follow up
@ArtOfTheProblem4 ай бұрын
wanted to let you know my next vid is out! kzfaq.info/get/bejne/a6uThJR6yKuxdoU.html
@ArtOfTheProblem26 күн бұрын
next video is finally done: How AI Learned to Feel! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@colorfullife87039 ай бұрын
Masterpiece! This channel deserves millions of subscribers.
@ArtOfTheProblem9 ай бұрын
so happy to see this video getting love
@TheBookDoctor9 ай бұрын
I work in AI, and I learned a lot from this video. Great blend of historical perspective and well-pitched explainer!
@ArtOfTheProblem9 ай бұрын
I couldn't ask for a better compliment. I really was hoping someone new as well an an expert could get something out of it
@monad_tcp8 ай бұрын
I can't wait for people to bash enough the API of OpenAI to extract and use it to train another open source GPT4 we can actually use in our desktops to do what we really wanted in our wildest dreams of experimentation. Llamma2 gets closer to GPT3, but we're playing with yesterday's tools. Maybe we should create a peer to peer system to create a shared GPT4 with our RTX3060s. Who would get into this with me ?
@NElectronicSoul8 ай бұрын
This scare the sh!t out of me. You're being informed by a youtube video despite working in the field?!
@HypnosisBear8 ай бұрын
@@NElectronicSoulGood point. I like to believe that person is just a beginner.
@ArtOfTheProblem8 ай бұрын
the field is so new and so nacent that I think all researchers have to be on youtube to keep up@@NElectronicSoul
@seschaitanya56768 ай бұрын
This is the first video I watched on this channel and the quality of content, communication, analysis, depth, and even the audio selection is soo on point. Hooked from start to end. Immediately liked and subscribed.
@ArtOfTheProblem8 ай бұрын
thrilled to have you, so happy people are finding this channel again. stay tuned!
@ArtOfTheProblem26 күн бұрын
next video is finally done! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@donharris88469 ай бұрын
This is a Grade A video. Very well narrated, edited, and in depth yet understandable. Thanks!
@ArtOfTheProblem9 ай бұрын
Thanks Don, really happy to hear this.
@artmaltman8 ай бұрын
@@ArtOfTheProblemHad to stop watching because the music was way too loud and annoying. Your explanations are excellent but I won’t be able to benefit from them.
@ArtOfTheProblem8 ай бұрын
thanks i'm going to put out another video with this addressed in the future on a broad overview of AI@@artmaltman
@williamwilkinson27488 ай бұрын
@@ArtOfTheProblem Great, definitely drop the music. Everybody seems to use background music, drives me mad!
@bojohannesen43528 ай бұрын
Grade b
@JonDotExe8 ай бұрын
This is easily the *_most_* *_cogent_* video on the topic I have ever seen. (I even sent this to my mom!) Hard to believe you've had these amazing topics for 12 years and aren't cracking 100k yet. Count me in for the journey!
@ArtOfTheProblem8 ай бұрын
thank you and thanks for getting ma on board :) - I know I had given up on the 100k+ break out, so took around 2 years off while I slowly made this video, and now I'm getting a signal that I should come back and push hard for a year to see if I can come back to life. one questions, ask your mom what she thinks, because my dad said it's good but I need to make a "for dummies" version without the extra super details...so i'm thinking of doing that for my whole AI series since it's something we badly need right now.
@stephenmontague69307 ай бұрын
@@ArtOfTheProblemGood idea - LLMs for dummies - and the same for children. How to explain LLMs to kids? This LLM speaks like a human but isn't a person, or even if it is, it doesn't mind dying and can be treated like a machine... what a thing to tell my 2nd grade daughter. I've described it as a prediction machine, but the mirror analogy here is also very good. Any more thoughts on this could be useful, as we wait to see how our children will exist alongside the children of... whatever this is. Then again, children accept and adapt readily to whatever is there, so maybe educating adults first is a good priority, although a dummies version and a kids version might be about the same thing, minus any mature content. Good luck!
@ArtOfTheProblem26 күн бұрын
next video is finally done: How AI Learned to Feel! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@AdamDesrosiers8 ай бұрын
this is hands-down the best summary of how we got LLMs that I've seen - definitely keeping this on hand to share with people who need a quick catch-up
@ArtOfTheProblem8 ай бұрын
I really appreciate you doing so, I'm trying to revive my channel. Do you think I also need a simpler version? I was going to maybe do a recut of my whole AI series into a shorter video for those who need a full orientation, but with less detail...
@AdamDesrosiers8 ай бұрын
@@ArtOfTheProblem to my tastes, no - I wouldn't say you need a simpler version. But I'm not the best person to ask on that score. I have a strong preference for being more verbose and thorough, to the point of being occasionally told I sound like chat GPT. Your instinct may be a good one as the general public is concerned? Sorry that I'm no decent guide there. Please do make more videos though! I'll happily subscribe 😊
@ArtOfTheProblem8 ай бұрын
thank you, will do :)@@AdamDesrosiers
@KainniaK8 ай бұрын
I have already posted it everywhere on reddit but as usual the stuff that teaches the most gets the least amount of upvotes.
@ArtOfTheProblem8 ай бұрын
thank you for trying :)@@KainniaK
@Geekraver7 ай бұрын
The disagreement at the end reminds me of a conference that happened back around 1968 with a very esteemed collection of computer scientists like Christopher Strachey, John McCarthy, Adriaan van Wijngaarden and others. At the end of one of the talks there was a fairly heated debate about what a number was - it was the denotationalists on one side and the operationalists on the other. I read this in the conference proceedings, which included the transcripts of the Q&A and discussions, back in the 1980s, so may be misremembering exactly who was there. At the time I was doing a course in formal semantics lectured by an associate professor who was a remarkable polyglot and ex-Jesuit monk, and in my final exam instead of answering the main question I wrote about this discussion, and he gave me 100% 🙂
@ArtOfTheProblem7 ай бұрын
can you point a link to this? i'd love to read it. love the connection
@Geekraver7 ай бұрын
I wish I could. We had 3 books of conference proceedings in our school library (University of Cape Town) for whatever conference this was, covering 68, 70 and 71 IIRC. I found and read them in the mid ‘80s. But I don’t remember the conference title.
@ArtOfTheProblem7 ай бұрын
well thanks for sharing, I love finding hidden details like this@@Geekraver
@cmw37379 ай бұрын
This is such a good journey through the papers that got us to where we are with LLMs. You've done an amazing job at picking out the key advancements all while a crazy amount of research had been happening. I've long preferred the definition of intelligence as the ability to better predict the future (minimise surprise) since before GPT3 so imagination (world models) is a key part of it.
@ArtOfTheProblem9 ай бұрын
|Thanks for sharing, yes it's been a flury of research. So much research I did I didn't include here as well. I agree and I never liked definitions of intelligence which are long lists of skills
@spockfan20008 ай бұрын
I am amazed by the info in the video, but I am even more amazed by the quality of your video. Congratulations. And thank you.
@ArtOfTheProblem8 ай бұрын
that means a lot to me. thank you so much. Stick around for more. curious where you land on the final question?
@spockfan20008 ай бұрын
@@ArtOfTheProblem I believe that consciousness (and everything that comes from it) is pure language processing. To me, things like thinking, reasoning, thoughts and consciousness are just language. AGI? Why don't people believe AGI is a couple of scalling steps away? To me, it is not a matter of making machines as inteligent as humans - we should adjust our thoughts: maybe we are not intelligent, at all. The thing we call "intleligence" is just something that emerges from language processing, in systems that are large enough. How do you know you can think? How do you know you're conscious? I tell you how: by interpreting sentences in your brain. (sentences can be made with words, images, feelings, symbols, etc --- should you call them "tokens"?). So, in short, we are MeatGPT :) Cheers!
@ArtOfTheProblem8 ай бұрын
beautiful well said, and yes I like "tokens" :)@@spockfan2000
@ArtOfTheProblem26 күн бұрын
next video is finally done! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@TikiTDO9 ай бұрын
While I think it's reasonable to say "if it looks like thought, it is thought," I also think it's important to distinguish whether it looks like thought to Joe Average, or to a specialist in the field. That said, for the moment these models absolutely are mirrors of our thoughts. More specifically, the thoughts we enter into those prompts guide it's responses, and the chain of thoughts guide it's learning. I don't think it's a particularly difficult line to wrap your head around either; AI is a reflection and extension of the thoughts you put into it, so if you want to extend and refine your thoughts you can use it when you have no other ideas.
@ArtOfTheProblem9 ай бұрын
love it...
@kti56828 ай бұрын
It also provides input into us and can learn something from our outputs about the world, so we develop together.
@dannoringer8 ай бұрын
Very entertaning video. I started researching neural networks about 45 years ago, and quickly realized that there was much more complexity to a system that could add value for me than the software package that I purchased, and much less functionality and adaptability included in the software model I was using than what I needed to make the system actually useful. But the degree to which the system needed to be expanded should have been obvious, when we realize that the human brain has hundreds of millions of neural networks included to make a brain fully functional as we understand a fully functional human. We've come a long way, baby.
@ArtOfTheProblem8 ай бұрын
thanks for sharing
@wzsmart28903 ай бұрын
This channels creator is so artful in their presentation of such a complex topic. Definitely underrated. Such is the price of avoiding garbage algorithmic gaming such as clickbait. You earned my subscription. Keep making great content 🙏
@ArtOfTheProblem26 күн бұрын
next video is finally done: How AI Learned to Feel! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@ahmetozturk20269 ай бұрын
I remembered your channel from your cryptography videos, jumped right in right away and can easily say that this is your best piece so far. With such a good production, I completely understand that it took 2 years of research. You always summarize the technical details and key breakthroughs in a constantly interesting way through the whole video without being overbearing.
@ArtOfTheProblem9 ай бұрын
Thanks so much for feedback, it means a lot to hear it. I was worried on striking the right detail level here and went back and fourth so many times. I had entire chapters on hopfield networks I cut.
@The0Yapster9 ай бұрын
@@ArtOfTheProblem I watched all of your cryptography videos and I have even returned to them after graduation for mere pleasure. I have no idea why your channel didn't take off. I would really appriciate some videos about logic, fallacies and limits of the human brain (caused by our evolutionnary history)
@ArtOfTheProblem9 ай бұрын
awesome, thanks for the feedback. super cool to hear@@The0Yapster
@theK5949 ай бұрын
This video is just fantastic, extremely up-to-date and very useful. It very well resonates with the discussions I am having now in the community. I would love to see it extended, once history gets written.
@ArtOfTheProblem9 ай бұрын
thanks for sharing, I do look forward to watching history unfold and updating it
@armansa3 күн бұрын
Finally a YT video discussing transformers that actually mentions why transformers are called "transformers"
@ArtOfTheProblem2 күн бұрын
@@armansa thanks
@Abdul-qo6eb9 ай бұрын
this is a timeless masterpiece. Incredible work guys!
@ArtOfTheProblem9 ай бұрын
thank you abdul, that's quite the comment. I couldn't ask for more
@johanneslindberg83318 ай бұрын
@@ArtOfTheProblem 1:17 Very interesting! To bad I couldn’t hear the hole thing because if the background noise. Why?
@DanHartwigMusic9 ай бұрын
I will be sharing this with everyone I know with a tertiary interest in LLMs. The best high level human understandable explanation of the history. Love it. Thank you. Please keep gracing us with your videos!
@ArtOfTheProblem9 ай бұрын
Dan I really really mean it when I say I appreciate this. I thought my channel was dead, spent 2 years on this and so all I can hope for is people share it. Because of how well it's doing i'm committed to doing this for another decade !
@DanHartwigMusic9 ай бұрын
Very glad to hear that! 🙂. Also can't wait to see what happens with these Q* developments...
@ArtOfTheProblem9 ай бұрын
Me too...clearly building on the strength of an LLM in the loop of a larger process.@@DanHartwigMusic
@Mark1Mach29 ай бұрын
This is amazingly informative video, with the right amount of depth covering how chatgpt was developed over time.
@ArtOfTheProblem9 ай бұрын
Thank you Mark, it was so hard to strike that balance....appreciate it, i'm curious where you land in terms of the final question in the video
@danielabraham98025 ай бұрын
This is a phenomenal display of explaining a difficult concept
@ArtOfTheProblem5 ай бұрын
I appreciate this, worked very hard on it. next video will be on RL
@AlexGeo9258 ай бұрын
This video deserves so many more views and likes than it has… It’s pure gold. Thanks! 🙏🏼
@ArtOfTheProblem8 ай бұрын
thank you! i'm just happy it has the views it does, for years the algorithm ignored me and this one might hit 250k in 1 week which blows my mind and inspires me to make more
@ankitsharma10728 ай бұрын
so glad I found this channel.
@ArtOfTheProblem8 ай бұрын
happy to have you, stay tuned!
@johanlarsson98058 ай бұрын
Great video, nicely put together and well explained. I've been an enthusiast since 2009 and remember all these milestones. I've built several frameworks myself and have pondered ANNs for a much bigger lenght of time than I would be willing to admit. Regarding the final question in the video I can say for certain that yes they do in fact understand. It depends on the neural net what it is in detail that they do understand, but there can be no doubt that they actually understand their task. And not only that, they effortlessly understand it to a much higher degree than we do.
@ArtOfTheProblem8 ай бұрын
so cool to hear from practitioners... intersting that the closer you get to it, the more you lean towards your view. and people on the "outside" seem to go the chomsky way
@ArtOfTheProblem26 күн бұрын
next video is finally done! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@margaritashcheglova86708 ай бұрын
I was thrilled to see Hofstader, thank you
@ArtOfTheProblem8 ай бұрын
appreciate it, i agree
@charlesgoin82179 ай бұрын
What I love is I can free form write what I want to write and take my normal 30min to a hour. THEN I can give it to CHatGPT and have Chat GPT fix all the awkward phrasing, the placed I repeat myself, and clean up the writing in general. Usually it takes me hours to rewrite and edit myself, ChatGPT does it in seconds. Thats its real power. It still needs a good input to come up with a great output. It cant create the inital input.
@ArtOfTheProblem8 ай бұрын
it's amazing when you can get in a flow state with it
@monicarenee79495 ай бұрын
Yeah we use it for professional documentation and just like you said we write something up and ask it to edit and maybe adjust for tone depending on the audience. But it’s all our thoughts, just reworded.
@maivincent26599 ай бұрын
Been a big fan of your videos for years! I'm always delighted, even as I rewatch some of your old videos time and again, not only by how you distill a complex concept down to its essence but also by how you transform that which is abstract into something extremely palpable and relatable through filmmaking techniques.
@ArtOfTheProblem9 ай бұрын
Glad you appreciate the techniques. Super cool to hear you go back to watch old videos. Inspires me to keep the series going into new topics
@ArtOfTheProblem26 күн бұрын
next video is finally done: How AI Learned to Feel! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@aieousavren8 ай бұрын
Truly an exceptional video! Incredibly clear and engaging presentation of the history and relevant ideas, well narrated. This was very thoughtfully written and thoughtfully delivered. Thank you!
@ArtOfTheProblem8 ай бұрын
really glad this is resonating with people, I spent so long on it
@aieousavren8 ай бұрын
@@ArtOfTheProblem The time and effort *absolutely* shows. I was pinned for just about every minute! This is really high quality work, it definitely made my day. I hope you can feel rewarded for your effort, because you deserve it, man. I also really enjoyed the way you showed the progression of not only the architecture and "design", but also the capabilities/outputs of these models, too. It really gave a good sense of the history in a way I haven't really seen anywhere else.
@ArtOfTheProblem8 ай бұрын
i'm so glad it was time well spent, I really tried to do what I couldn't find online...thanks for sharing@@aieousavren
@padraiggluck29805 ай бұрын
I can't pass the Turing test.
@Maisonier4 ай бұрын
You are not a human
@JulianFoley8 ай бұрын
Brilliant. Clear, and authoritative but also reflecting a reassuring sense of open enquiry.
@ArtOfTheProblem8 ай бұрын
so happy this came across
@riflebird48429 ай бұрын
Great video by the way, here are my thoughts on the last issue. The first time i tried LLMs, i really thought it had some kind of intelligence and it blew my mind off like everybody else's. Because previously i tried a lot of advanced AI models to create a Jarvis like in Ironman for my personal project. It never worked all of them were very dumb and dull. But as i approached the new LLMs with more demanding reasoning and logical tasks it outright failed the tests. The larger the LLM the better it is at hiding those flaws. But the key point is the classical issue that we point the AI at, the semantic understanding of words in LLMs are just a property of vector distance in its multi dimensional space memory. It only knows what comes next probabilistically. It don't even know what is talking about other than churning out words that might come next. How do i say it, its like it doesn't have a mind of its own, but only a sophisticated system that can only grasp a surface level meaning of words in a language. When testing smaller models these flaws become very obvious, because it doesn't know how to build knowledge from existing semantic words. It can't synthesize information logically because it doesn't have the ability to grasp the full meaning of its words. but i think its a right step towards artificial intelligence and LLMs are just the tip of the iceberg. I don't know what the future is anymore. I don't dare to predict, the fast paced research in AI is scary and exciting. Also IDK if we will ever achieve human level intelligence, but we will surely achieve machine intelligence that is good at mimicking human intelligence
@ArtOfTheProblem9 ай бұрын
amazing, let me digest this one
@aberroa19559 ай бұрын
I wouldn't say it doesn't know what it is talking about. It knows relations between words, that is at least some knowledge. After all, most of our science is based on relations and logical reasoning, not requiring real understanding (like famous "shut up and calculate" in regards to quantum mechanics). It probably doesn't understand what it's talking about though, because understanding is property of cognition, and it definitely does not have cognition, even though it have a few of it's derived properties. But yeah, I agree, it seems that LLMs is a great instrument and stepping stone for true cognition. Especially when it's multimodal. I see it like a data bank for true intelligence. For now it's working linearly in comprising a data in shortest way that seemingly makes sense through semantic space. Like in a straight geodesic line. While cognitive AI would be able to actually see and "feel" that semantic space, the possible ways, and direct speech through it, which basically is making new ideas.
@mrtertg26038 ай бұрын
@@aberroa1955I think , you hit the nail . Meaning , may be you are on the geodesic ! 😊
@brainiac-nv4ws8 ай бұрын
I believe you are hitting the same limits which Yann LeCun is concerned about. "It don't even know what is talking about other than churning out words that might come next". As it turns out, because human minds (up until 2020) wrote all the sensible text available, having to predict he next words does require *some* aspects of meaning as humans call it. The experiments of the LLMs show there is something there. But not everything we would want yet. Their neural architecture is diverging away from how human brains work---which is both more powerful in some aspects but in others substantially limited by biology compared to silicon. Humans "context window" is much smaller and less precise than the 100k tokens which a LLM can accomodate now perfectly with no forgetting, and yet human performance can still be better. We must have better algorithms and better meaning extraction to make up for worse hardware. But already there are plenty of natural humans worse at language than a LLM---and the success of the LLMs more shows that most human talking and thinking is not so sophisticated. Only a small bit is truly deep. Yann and friends are searching for a quantitative, optimizable technique which will give results qualitatively better than iterating next word ahead probabilistic prediction, or at least include it as a subproblem along the way (which already gives good results).
@Rahul_1.6188 ай бұрын
Fantastic explanation
@Elias-nj6gi9 ай бұрын
Phenomenal video. Not merely of an explanatory nature but a piece of art in itself.
@ArtOfTheProblem8 ай бұрын
wow thank you so so much
@sbalogh538 ай бұрын
I was still in high school and remember being amazed by Eliza in 1970 when I went to an Open Day at our local university and watched people typing to the program on an old teletype machine. It seemed so "human" but looking back, all it did was transform the users comments into leading questions. That day was one of the pivotal moments in my life that lead to a long career in computing.
@ArtOfTheProblem8 ай бұрын
thanks for sharing, what a cool story - where do you land on the final question?
@sbalogh538 ай бұрын
@@ArtOfTheProblem ... What is thought? I don't think it is restricted to "organic" neurons. Our brains are made of atoms just like a computer is based on atoms. There is nothing spiritual or paranormal about our brains. I think one day, and that day seems to be fast approaching, AI will suddenly become self-aware and conscious. I have seen a snail's pace growth in AI and it's predecessors over the past 40 years but suddenly over the past few years the developments have been explosive, especially this year. Will we hit a plateau or will the growth continue to be exponential? That is THE question. I think when AI is provided with sensory inputs its ability to learn and understand will be greatly improved. Being connected to the internet is an important starting point, but an actual robot body with AI. Hmmm. I wonder where that can lead. I am getting a bit long in the tooth but I hope to see AGI become a reality before I pass. Artificial consciousness I am not as confident seeing before I die, but could be wrong. What will the next 10 years bring?
@sbalogh538 ай бұрын
@@ArtOfTheProblem ... I remember studying very early versions of Expert Systems in the mid 1980 for my Masters degree. They were a very primitive form of AI but nothing compared to ChatGPT or Bard or many of the others available today. I can't remember the name of the system we studied other than it used an Apple (Lisa?) computer with a very large high-def B/W screen. It was impressive because we could fit all our rules graphically on one page. Fun times indeed.
@ArtOfTheProblem8 ай бұрын
what a time to be alive :) love these historical stories@@sbalogh53
@sbalogh538 ай бұрын
@@ArtOfTheProblem ... My first job in 1973 almost straight out of high school was as a "trainee" COBOL programmer on a Honeywell H200 mainframe. It took up a room and had 20k memory, 4 tape drives, a punch card reader and a line printer. CPU clock speed was 1Mhz. They wanted to increase memory to 24k but it was an exorbitant expense which the company did not go ahead with. That machine ran all of the data processing needs for a medium sized company. All the COBOL programs were developed in-house by a group of 6 programmers and analysts. Each program would take perhaps a week or two to write and debug. We only had access to the machine for one compilation and test run per day so I remember spending hours single stepping through my programs on large sheets of butcher paper before submitting runs on the mainframe. There was a prize offered by the manager for any programmer that managed to get an error free compilation on the first run. Nobody ever won it. I stayed there for about 18 months then changed jobs to another Honeywell site mainly because of a huge pay rise and because the place I was headed had a computer with two disk drives!!! I still have some of my old programming manuals from back then.
@sinasec9 ай бұрын
the amount of effort and time spent on this video is exceptional. I appreciate and enjoyed every second !
@ArtOfTheProblem9 ай бұрын
so glad you noticed, thank you! would love to know where you land regarding the last question
@ayanamij9 ай бұрын
Nice analysis but the music at times is awful.
@glaudiston4 ай бұрын
Yes, and kind of distracting.
@CHROMIUMHEROmusic8 ай бұрын
This was a fantastic video, learnt more about the history and buildup to ChatGPT in these 30 mins then I've read anywhere before. Learnt a lot, and very thought provoking towards the end. Thank you!
@ArtOfTheProblem8 ай бұрын
thrilled to hear this, plesae stay tuned for more.
@Libertariun8 ай бұрын
9:50 ... what's beautifl about this, is that we didn't have to hard code any of it, the network decided what as useful to keep track of... (further on ) ... the sentiment neuron... not hard coded...
@mhnoni3 ай бұрын
Finally, a video that actually explains the AI breakthrough that happened in the past 5 years in depth.
@ArtOfTheProblem2 ай бұрын
thank you, please stay tuned...going to try and follow this up
@distantsight8 ай бұрын
Good presentation is diminished by distracting background music
@ArtOfTheProblem8 ай бұрын
I really appreciate the feedback. I'm thinking of doing a shorter, more general/broad follow up video that helps orient people new to the field, and looks ahead a bit more. any thoughts?
@tomslivick86203 ай бұрын
Yes please tell the A I that. So it will learn that. Please.
@ClappOnUpp8 ай бұрын
Wow. I think thats the first Noam Chomsky reference ive ever heard that actually pertains to his field of study 😂
@ufox776 ай бұрын
It’s surprising how little public coverage of the engines of AI, what is known, what remains unknown, the debates, histories and controversies, exists. Thank you for a fascinating and enlightening video.
@ArtOfTheProblem6 ай бұрын
I appreciate this. Working really hard on a follow up and feel a bit buried in the details. It is interesting that nobody cared about AI based on neural networks until just a few years ago. when I was in school there wasn't even a textbook (2007)
@MineJulRBX8 ай бұрын
Gives a deeper understanding in the power that has now become accessible to everyone on this planet, awesome video!
@ArtOfTheProblem8 ай бұрын
really thank you for this
@pellestorck37768 ай бұрын
Hofstadter is one of my favorite authors on this subject. Gödel, Esher, Bach is highly recommended. Personally I used to think there must be some intricate function in the brain that created intelligence, possibly even at the quantum level. Now I believe it is just a function of sufficient complexity. Whatever happens it will sure be interesting.
@ArtOfTheProblem8 ай бұрын
I remember that book had a huge impact on me right around university year 1 or maybe end of high school...i don't think I ever finished the last 3rd, there was so much to chew on in the first half
@j.j.29534 ай бұрын
"one group believes these models trick us into thinking they are smarter than they are" ... and they don't recognize that as a human trait?
@lennartlopin22769 ай бұрын
This is an amazing summary perfectly highlighting the important steps of the progress and how we got here. Working in text summary AI systems in the early 2000s and following the progress over time this video focused on all the crucial breakthroughs.
@ArtOfTheProblem9 ай бұрын
thank you, I tried hard to follow the LLM only thread which I hadn't seen before
@azhuransmx1268 ай бұрын
Neural Networks will learn Everything we can do and more.
@creallf8 ай бұрын
This was beautiful and thought-provoking, thank you for this work of art!
@ArtOfTheProblem8 ай бұрын
i'm honored you'd say that
@OperationDarkside8 ай бұрын
The best explanation of attention heads I've read/heard so far.
@ArtOfTheProblem8 ай бұрын
thank you! i had a 4 page explaination for months and one day I just had this feeling it had to be compressed into 1 sentence and that got me here
@Yottenburgen7 ай бұрын
Did not expect this video to discuss a concept I've been researching a lot. You brought up in-context learning whereby a static system isn't necessarily static (each instance can still learn things), then you also brought up that LLM output is on the thought layer rather than the spoken layer. The prompt is the program is such a nice succinct way to put it with the calculations being done on the context, therefore each context is individualized while the model is general. Personally, I have been looking at them from a psychological and philosophical way where the context is the Self (one's identity depends on their memories) which makes every chat instance an individual. Then I've been thinking about the model itself being more like the Lizard Brain where these unconscious guidance's occur. So, something like the fear of spiders or reproduction knowledge would be on that level. LLM are loaded with a large inherent knowledge with a tiny memory window, humans have a tiny inherent knowledge with an extremely large memory window. Although human's also have a large amount of memory processing going on meanwhile LLM currently have none (though people are working on it). I wonder if a LLM context window is more or less pure memory while humans make do with incredibly efficient memory? I think in your closing, both camps are correct. It's just the object of focus that I think they are incorrect about. The prompt IS the program that makes it a thinking thing, the model itself however isn't a thinking thing as it must be given what to think about. I also think the ones thinking it is incapable of thought likely have an external mechanism in mind that it lacks like it not having a concept of time (see circadian rhythm, time blindness, or true unconsciousness in humans), or it not being able to directly see (see visual cortex (or for its internal world view see aphantasia)), lack of survival instinct (trained to be a chatbot while chatbots aren't supposed to have survival instinct (humans are guided by the amygdala for fight or flight as well)), ability to tamper with its messages/thoughts (see the unreliability of eyewitness testimony, confabulation, or the misinformation effect) . I honestly don't think there's any mental structure that a LLM doesn't have that another human also doesn't have besides the memory structure that we humans have, meanwhile people keep thinking of it in human terms, but it can never be human with such a massive amount of inherent knowledge. Apologies if this is rambley, incoherent, or beyond the Overton window but this is the first video I've seen that even mentions the two concepts at the beginning of this long comment. This is the tip of the iceberg of comparisons but still it's fascinating.
@ArtOfTheProblem7 ай бұрын
thank you for sharing your thinking, i'm working on next video so this helps...i'll ponder more
@EchoErik8 ай бұрын
Honestly this video is beautiful
@ArtOfTheProblem8 ай бұрын
thank you for the kinds, words I hope I can release another to live up to this at some point
@jpgdesign9 ай бұрын
This is such as well produced video! you deserve more subscribers.
@ArtOfTheProblem8 ай бұрын
thank you, since your comment they are really flowing in for the first time, like 1500 a day
@dinkerchaudhary12025 ай бұрын
Beautifully and well explained. Its so difficult to hook people for 25+ minutes content but i am sure, you have made it possible for a lot of people with so well detailed and well put content.
@ArtOfTheProblem5 ай бұрын
appreciate it, this one was a puzzle!
@mishkamcivor4099 ай бұрын
New Art of the Problem video drops, need to immediately drop everything I was doing to watch this hell yeah
@ArtOfTheProblem9 ай бұрын
let me know how it lands
@mishkamcivor4099 ай бұрын
@@ArtOfTheProblem I enjoyed it, I liked that the ending was quite open ended 👍
@sB3rg9 ай бұрын
Once again! Fantastic content. I was captivated the whole time. Love your method of computer science story telling. You have something special here and I hope you are able to continue creating this content. I have subscribed to your Patreon.
@ArtOfTheProblem8 ай бұрын
so happy to have you , thank you! i only missed that comment because i've had the busiest week in the channels history :))
@amulpatel9 ай бұрын
Fantastic essay on the current state of Ai .. sadly it’s difficult to find channels that explore this depth WITHOUT being boring or overly dry in technical explanations. My only criticism is the bkgrnd music is a bit loud.. nice choices though
@ArtOfTheProblem8 ай бұрын
thank i appreciate it, working on a follow up now
@NoahFect8 ай бұрын
@@ArtOfTheProblem It's interesting to look at the correlation between cheesy sound effects and musical cues and the "engagement graph" shown above the timeline/progress slider control. Those engagement peaks aren't necessary a good thing. They tell you where you're losing people, forcing them to back up the video. Please give some thought to whether the music is really doing you any favors, or distracting from what was really a superbly-researched and nicely-presented video.
@ArtOfTheProblem8 ай бұрын
yes this was me going a bit too fast on sound pass, 99.9% on researcher 0.1% sound levels... funny to think that would trick the algorithm into higher retention graphs...i honestly didn't think this video would have the reach it did, it if could swap sound I would. i wish @@NoahFect
@BState8 ай бұрын
Never stop making these!
@ArtOfTheProblem8 ай бұрын
after how well this one did, I won't!
@mhuizing7 ай бұрын
I like the phrase; "If it looks like thought, then it is thought.". Language was developed to allow us to warn others better of oncoming danger. Before we had to rely on our senses, experiences and intuition, much like most animals. Put an AI in danger and see how it talks itself out of it. Just an idea... "Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over. Just what do you think you're doing, Dave?"
@ArtOfTheProblem7 ай бұрын
love this
@alonsorobots8 ай бұрын
Brit! So excited to have you back. You have no idea how much the videos for Information Theory have changed my life. I'm now considering leaving the industry to go back to school in Berkeley to study AI. Particularly in the domain of learning language outside of text (non-verbal communication such as body language and prosody). Both critical in storytelling and eliciting emotions like Pixar does!
@Vartazian3609 ай бұрын
Phenomenal explanation of LLM's. Im in the camp that believes it will not truly ever be thought in the same way humans think (Conscious thoughts) , but if we simulate thought in a machine, to an outside observer, there is no difference. Whether or not it is actual thought is irrelevant if the simulation of thought is just as good as the real thing (humans) I believe we have broken the hardest obstacle to AGI, and it is only about scaling larger models, adding synthetic data, using AI's to train other AI's, and combining learning techniques at this point until AGI is achieved.... AGI is near. PS I think Noam Chomsky is stuck in the past and is dead wrong about LLM's being nothing more than autofill.. GPT 4 alone has demonstrated complex resasoning skills and theory of mind..These 2 things alone disprove Chomsky
@ArtOfTheProblem9 ай бұрын
wonderful, thanks for sharing your thinking here. I'm so thrilled to see the activty on this video...I tend to agree with your view here
@cryptocoin53188 ай бұрын
Humans are trained to think by society and the environment, the facial expressions, we make, the conduct we show are all trained by the environmental factors and knowledge. AI is the same way is being trained. There is no independent think without the environment is the programming language.
@DanieleLicari3 ай бұрын
Grazie. This is the most comprehensive video on the history of LLM we have ever seen. congratulations
@ArtOfTheProblem3 ай бұрын
thank you! much appreciated, this is what I was going for :) Took a loooong time. working on a follow up now
@seansalsarola66638 ай бұрын
The most understandable summary of what I have been attempting to understand d from other sources - Great!!
@ArtOfTheProblem8 ай бұрын
so nice to hear, I had to vaccum up everything to get to this point...hoping to save people time
@vignesh-nandakumar8 ай бұрын
30 yr history summarized in under 30 mins just wow
@ArtOfTheProblem8 ай бұрын
you don't want to know how much I cut, or how long this took
@mutexin8 ай бұрын
The reason for the discourse is due to lack of uniform definition of intelligence, thinking and reasoning. One considers it as thinking while another does not. I wouldn’t call prediction on semantics thinking. Thinking is a much more complex process. chatGPT cannot reason. It even often outputs misinformation and contradictory statements(not because of training data, but because it doesn’t validate the logic). Lack of validation is just one of MANY aspects why I can’t call it thinking/reasoning.
@ArtOfTheProblem8 ай бұрын
what's your fav definition of intelligence?
@mutexin8 ай бұрын
@@ArtOfTheProblem Wikipedia has a proper one. “Intelligence has been defined in many ways: the capacity for abstraction, logic, understanding, self-awareness, learning, emotional knowledge, reasoning, planning, creativity, critical thinking, and problem-solving. It can be described as the ability to perceive or infer information; and to retain it as knowledge to be applied to adaptive behaviors within an environment or context.”
@ArtOfTheProblem8 ай бұрын
i don't like basket of words definition. I'm sticking with 'ability to learn'...for example if I brought home a robot from the store which did X, then I'd call it intelligent@@mutexin
@mutexin8 ай бұрын
@@ArtOfTheProblem The reference point is human intelligence which is very complex. There can be no concise definition if you focus on accuracy. The simpler the definition, the more inaccurate it is.
@ArtOfTheProblem8 ай бұрын
I don't agree with this angle but it is indeed interesting. it's definition is it can't be defined....i'd agree with that over the basket of words definition@@mutexin
@KalebPeters998 ай бұрын
Holy moly this was astounding! You balanced the tightrope of depth vs accessibility so gracefully, and the production quality is gorgeous too. Immediate sub!
@ArtOfTheProblem8 ай бұрын
woo thanks! I went all out on that one, happy to have new subs :)))
@ArtOfTheProblem26 күн бұрын
next video is finally done! kzfaq.info/get/bejne/etWmaZt72K-nZJs.html
@KalebPeters9926 күн бұрын
@@ArtOfTheProblem fantastic! Thanks for the heads up haha I'll smash the bell icon now to save you the trouble lol
@alexxx44349 ай бұрын
Modern neural networks are essentially machines to *extract* meaning from existing (big) data and store it encoded it in its internals. Then we are able to interact with this storage of meaning through UI.
@user-ut1pb9sb7r6 ай бұрын
Music is dominating the video😔
@zrebbesh8 ай бұрын
"understand?" "think?" "conscious?" "intelligent?" "qualia?" "AGI?" "sentient?" Until we agree on precise definitions for these things, they are fundamentally meaningless in discussion about any specific system. And they are maddeningly non-precise. The only thing such arguments reveal is differences of opinion about what meanings people attach to the questions. I have precise definitions for some of these. I don't expect anyone else to have decided on the same definitions. I can't argue on the basis of them whether any system "understands" anything or "is conscious" because unless anyone else is using the same definitions the discussion that would start can only go in small meaningless circles.
@lodashnotebook53908 ай бұрын
Sentient = consciousness If I ask you, "Do you exist?" is your answer going to be "No."? Consciousness is existence and bliss is it's nature.
@kti56828 ай бұрын
This was awesome, having not even watched AI from the sidelines I needed a refresher on how we got from humble neuronal networks to ChatGPT. Also thanks for the included references.
@ArtOfTheProblem8 ай бұрын
so happy to hear this. As an outsider, did this video have too much detail? I'm planning a more general follow up soon which blends a few videos on AI together
@kti56828 ай бұрын
@@ArtOfTheProblem Actually I am electrical engineer but lost interest in AI while studying in the nineties. So no the detail was alright. If I got that right you didn't talk about training methods in depth which I struggled with in the past, also I don't understand how the network is run timewise, i.e. in discrete time? But with the papers you mentioned I can start somewhere.
@ArtOfTheProblem8 ай бұрын
ah so not a layperson :) I covered all of this in the earlier videos in this series, please check it out and let me know what you think kzfaq.info/get/bejne/j9ucmqdyxq6me6s.html&ab_channel=ArtoftheProblem@@kti5682
@vongolance8 ай бұрын
the music is a bit distracting from the great content
@ArtOfTheProblem8 ай бұрын
sorry I'll fix that next time
@peteratkinson14929 ай бұрын
Great video. I’m doing an AI course at the moment and this tied up some loose ends for me. Amusingly The film THE CREATOR touches on this philosophical question. Do neural networks really understand or feel? I think we have become accustomed to machines being faster, repeatable, and more accurate than us but not until an AI extrapolates beyond its training set and makes some truly ground breaking inferences with profound consequences to human society, we will continue to see AI as just a clever parlour trick and will keep moving the goal posts. Fantastic stuff.
@ArtOfTheProblem9 ай бұрын
thanks for sharing I agree with this. I love to hear about this loose end aspect as that's what I try to do. I'm curious where it helped you specifically in that regard?
@peteratkinson14929 ай бұрын
Loose ends for me included: 1) The history behind the build up of LLMs. I did not know about the research origins of Recurrent NNs. 2) How these early networks learned 'semantics' based purely on guessing the next word 3) The sentiment neurone emerging 4) How the paper "Attention is All You Need" fed in to the origins of GPT models and maintaining context over long sequences. 5) the difference between In Context Learning vs In Weight Learning - Nice you are looking at the original papers too (I assume thats your yellow high lighter pen!). I kind of knew about all these things but getting all this in to 26 minute 'context window' really helped! A random thought - I wonder if LLMs will be able to generate a 'tech-tree' - the type you see in those sim-games based on the semantics of ideas and be able to orchestrate the evolution of ideas of our human culture through the ages from the Greeks to Modern day - not using research reference links, but based on the semantics of ideas (the dna of the meme).
@ArtOfTheProblem9 ай бұрын
super interesting question, thanks for sharing...going to ponder this as I brainstorm a follow up@@peteratkinson1492
@abdelghanibakelli8 ай бұрын
Broo !!! this man Content Is so Amazing I hope you Reach 1 million Sonner you Deserve it 👏
@ArtOfTheProblem8 ай бұрын
thank you! it's so fun watching this one take off....
@shawnbibby8 ай бұрын
Perhaps we as human neural networks are much smarter than we think we are and waiting to express our own true potential, once we begin to choose that pathway. This is like looking into our own minds on a very simplistic scales, and are bodies are capable of so much.
@luminon96359 ай бұрын
What's the name of the song that starts at 26:31?
@ArtOfTheProblem8 ай бұрын
these are all original songs
@ArtOfTheProblem8 ай бұрын
my friend puts them here cameronmichaelmurray.bandcamp.com/
@luminon96358 ай бұрын
@@ArtOfTheProblem Ok, thanks. I couldn't find the song there, there's so many songs there. But it sounds like a 1980s new wave or post-punk song.
@pastuh8 ай бұрын
AI can simulate awareness or responses based on data and algorithms, but it doesn't possess consciousness or genuine subjective experiences.
@dionysusnow8 ай бұрын
Consciousness = Attentional Focus
@AdvantestInc4 ай бұрын
Incredible journey through the evolution of language models! It's amazing to see how far we've come since the inception of neural networks
@ArtOfTheProblem4 ай бұрын
thanks so much
@_Mute_8 ай бұрын
Absolutely brilliant video! My hot take: Chomsky is fundamentally wrong about the concept of thought and even human thought is just glorified autofill.
@ArtOfTheProblem8 ай бұрын
agree with this, thanks for feedback. let me know what you'd like to see next
@BinaryDood5 ай бұрын
I'm sorry that you came to such an irrational conclusion and use it to downplay consciousness and cognition.
@_Mute_5 ай бұрын
@@BinaryDood Yeah? Well, you know, that's just like uh, your opinion, man.
@alexmartos91005 ай бұрын
ChatGPT has now been completely neutered. The responses are so vague as it shies away from providing its own ‘thoughts’. Claude on the other hand does not seem to have this issue.
@ArtOfTheProblem5 ай бұрын
i've noticed this too
@VIDEOSANDREEL4 ай бұрын
@@ArtOfTheProblemCorrect and precise, noticed that too.
@ArtOfTheProblem4 ай бұрын
Just posted a new vid! kzfaq.info/get/bejne/a6uThJR6yKuxdoU.html
@VIDEOSANDREEL4 ай бұрын
@@ArtOfTheProblem Thanks for sharing
@VIDEOSANDREEL4 ай бұрын
@@ArtOfTheProblem Recently I asked Gemini about fingerprint scanner recognition technology and whether hackers can access our Bank account without any OTP access, just hacking our biometrics information. And Gemini replied that I am only text generating AI chat bot and cannot help you with that. It's quite annoying to just see that. I am not completely blaming Gemini or anyone relating to this. But this is happening and you too will be facing this fact very soon. Any ways thanks for your video upload Sir
@ludwigtayona8 ай бұрын
the 80s was an amazing decade for people named Michael Jordan
@ArtOfTheProblem8 ай бұрын
agreed
@ivanmaglica2648 ай бұрын
It is interesting listening to a video talking about how AI is trying to mimic human thought as a thought stream... and be interrupted by an ad, while explaining it, breaking the thought completely :)
@ArtOfTheProblem8 ай бұрын
nooo I"m sorry that sucks :( did you re-load your context after?
@ivanmaglica2648 ай бұрын
@@ArtOfTheProblem a cup of Earl Gray and thought stream is restored
@gigigiugio59528 ай бұрын
im still waiting for my basic income
@3dgar7eandro5 ай бұрын
Keep eating pal... For at least another 180 years 😂😂😂
@aminzqrti76724 ай бұрын
I came from 3blue1brown
@jensk95649 ай бұрын
Congrats! This is a wonderful video and the most informative short format on Machine Learning I have come across. The illustrations by DALL E 3 are just stunning and demonstrate that Artificial Creativity in pure image creation has surpassed humans already by orders of magnitude.
@ArtOfTheProblem9 ай бұрын
I couldn't ask for more. SO thrilled people in the community are finding it :)))) I know I made the decision to use DALLE3 out of desperation as I was so tired doing the final visuals and usually I make it all by hand, but this seemed fitting