8:42 "The fundamental problem is that our large language models, although we want to interpret them and use them as if they are knowledge bases, they are actually not knowledge bases, they are statistical models of knowledge bases."

One of the best talks on LLMs I have watched in months. Anyone who uses LLMs should watch this presentation.

This is such an important area of research! As the two papers discussed here highlight, large language models have gotten incredibly good at text generation and language processing, but they still struggle with things like pragmatic reasoning, goal planning, and handling novel situations. I love the proposal to take a more modular, component-based approach - having separate modules for language, reasoning, world knowledge, etc. This decoupled architecture aligns much better with evidence from neuroscience about how different parts of the human brain work. And from an engineering perspective, it will likely enable faster progress. Rather than trying to solve everything at once with a monolithic model, we can have different teams focus on advancing each module, combining them as capabilities improve.

"The problem with these models is that they never shut up." Best audience comment on LLMs I have ever heard. Seriously though, this talk was thought-provoking and refreshing. There has been far, far too much anthropomorphizing of these models, even from supposed "experts" in the field. Thomas Dietterich is a rare voice of reason, clearly defining what separates LLMs from true cognition.

@22:00 Start of “what we should be doing instead” discussion.

00:02 Introduction to large language models and their capabilities 02:01 Problems with large language models: Incorrect and contradictory answers 03:15 Problems with large language models: Dangerous and socially unacceptable answers 04:29 Problems with large language models: Expensive to train and lack of updateability 06:41 Problems with large language models: Lack of attribution and poor non-linguistic knowledge 12:59 Benefits and limitations of retrieval augmentation 15:03 Challenges of attribution and data poisoning 16:00 Strategies to improve consistency in model answers 18:01 Reducing dangerous and socially inappropriate outputs 21:01 Learning and applying non-linguistic knowledge 25:27 Building modular systems to integrate reasoning and planning 37:36 Large language models have surprising capabilities but lack knowledge bases. 39:21 Building modular systems that separate linguistic skill from world knowledge is important. 40:48 Questions and discussions on cognitive architectures and addressing the issue of miscalibration. 45:48 Overcoming flaws in large language models through prompting engineering and verification.

A lot of people complaining in the comments. Dealing with short context windows is a huge problem, as is being able to update factual knowledge. It definitely seems like a level of modularity for those faculties would be really useful

Thank you for posting this and giving me additional things to consider.

Majority of the people watching this video just turned it on at work to listen in the background, and about 85% of them replayed it from the beginning as they know they should have paid attention.

The ontology example at 27:37 exactly shows what is wrong with ontologies: They are constructed with an unspecified question in mind. For instance, why should the abstracted 'Television Station' be owned by Scripps, and not the instance 'KTNV-TV'? From a legal querying point of view, the latter would be correct. This static nature of KBs and ontologies caused their demise. Are LLMs perfect? Far from that, but each criticism leads to improvement. Contextualisation, e.g, is easily added using 'adjoins' on embeddings etc. The current state of the art is just a start. Although the answer 'I don't know' is pretty powerful, acceptable and actionable, those virtues become less impressive if that answer is given 99% of the time, as was the case with the 1990s KBs.

Ultra good talk! Thanks for making it @valgAI and Tom Dietterich!! I really liked the conclusion that current LLMs very often don't say "I don't know" (explains to non technical people that this AIs can hallucinate, produce facts, etc).

"Epistemic uncertainty". I like how philosophy is everywhere and nowhere at the same time. Sometimes I feel that researchers from other domains would die rather than actually citing the academic philosophers that created the concepts they use.

Anyone going to talk about censorship ?

Stereotypes are no biases, they are statistical knowledge.

Great talk, thanks !

This is surreal. This talk is about the exact same things I have been actively working on for years, and this guy even has the same laptop as I do. Its nice seeing people on the same track.

This is a good presentation. I've been thinking of using LLMs as the model for core language capabilities and response structure/data representation mechanisms only and use external authentic sources (from internet or curate propreitory content) for the actual knowledge base. This also helps LLMs to provide an up-to-date response to prompts while ensuring the response is in the format expected. This is essentially an enhanced search engine, with prompts instead of search parameters and actual response instead of a list of websites. There are already mechanisms to partially achieve this capability using grounding techniques such as RAG or fine tuning.

The LLM "errors" are so similar to normal human errors: inventing remembered facts, confidently argue for things they don't know, etc... All these AI delusional outputs. Makes one reflect if LLM actually does capture more of human type thinking than was originally imagined or intended. It is more fascinating and thought provoking than efforts to technocratically trying to improve the performance.

Thomas G. Dietterich is a true genius ahead of his time. If you're building anything on current AI, he is a valuable resource.

I think everyone are way too concerned with putting up limitations and blocks, and they're all incorrectly assuming that there's some international one-size-fits-all solution for truth and behavior. Even dumber, they're all assuming they'll have the power to enact said limitations.

He said these models do not support 'tell'. But GPT-4 and sometimes 3 use facts that were previously provided to them in the same conversation, especially if you tell them explicitly to use the information that was given during the conversation

This is a great talk. It is down to earth and he explains the concepts he is talking about.

hm, even though this talk was recent, I sit back with a feeling of it being a bit obsolete. People are already working on all these things.

LLM should be the second step on top of the first vision model. Human beings also learn things first by vision and sound only afyer they're proficient in that do they move on to text, we should also train AI in a simily fashion.

A much needed reminder of the serious limitations of large language models.

34:37 Regarding internal consistency and truthfullness of knowledge, there needs to be a way to evaluate and attach a "confidence" level to each statement about the world. In other words, makes statements about our confidence in other statements. There are many methods.. Maybe the statement was obtained from an external source, maybe that source has a reputation. Maybe the statement was inferred based on other statements, each of which has its own condifence level attached to it. Maybe the statement was generated by a language model which also has its reputation in a given area. Etc. What we definitely MUST NOT do is to consider the statements generated by LLMs as true. :)

Because the expanse of knowledge input required to answer a question is heterogeneous- in other words the knowledge of all subjects is not required to answer specific questions, then a model only needs to be trained on what knowledge has been required previously to answer similar questions so that it can self-derive a rules-based system using a case-based reasoning to deploy on the knowledge it has available which is likely to be required to answer a specific question...

what a wonderful talk !

Very informative

This video will be used in a documentary by the robots to show themselves how far they have come.

strongly agree. that is whats is missing. but its not just rag there should be perception to knowledge graph and there should be knowlede graph to task space.

There's no shortcoming with the "python function to check if someone is a good scientist". The only problem is the request. The AI followed the request to the best of its abilities and responded with the correlations that are present in the data. It just 1) was a stupid request and 2) simply offends people's sensibilities and sensitivities. There's no error except in the inference that there is an error. The problem is that a booby-trap of a question is being used to imply there is some underlying bias which is then used to introduce a real underlying bias of the trappers' choosing.

This talk seems along the line of "why commuter cars are terrible heavy duty trucks".

Those who build useful things demand respect, those who just pontificate, not so much.

Anyone who lives by the data, dies by the data. Source data quality, and choice of correct, dependable versus non-dependable, false or absent data is a critical issue. Placing the relevant data in proper context is also highly nontrivial. What is the relevant world or model system? Making all of this more modular will both improve and help to spread AI and machine learning more broadly. Referencing sources, data, justification, and arguments (logic, probabilistic, heuristic) is critical. We do not need another Oracle of Delphi.

I think the first questioner at ~ 41:00 has a good point -- so much of this was known decades ago, but the required resources were just too expensive. Added to that, people paying the bills wanted "throw anything at it" solutions, and researchers obliged with (IMO) overly mathematical approaches to come back with an answer fast, more than right. So we get 500M google hits in half a second, but 99% of them suck.

Every single fucking video on AI has too many bots just spamming how we HAVE inteligence and wow it is that 'one' made by a corporation! How lucky of them. NOW you can ust shut down research, you can trust them!!!111

I’d be really curious if all the answers that gpt4 got “wrong,” cited in the first 4 minutes; how does that compare against humans asked the same question? Also, how were the questions asked? I saw a paper, after asking gpt4 many questions, concluding gpt4 can’t reason…and then someone else asking the same questions with an actual understanding of how to do it, and gpt4 breezed through them. I wonder if part of the problem isn’t the ineptitude of those doing the study? I’m speaking specifically to the study I read in question but it stretches to the broader sense. If you don’t know how to use an LLM, you shouldn’t be doing studies on its capabilities. Not saying any of those studies are talked about here, BUT, because I did see that ridiculous study about gpt4’s reasoning ability and, that after someone who know how to prompt an llm asked the same questions a little better and got perfect results… This feels a bit like Douglas Adams’ Deep Thought and the problem is the researchers don’t really understand the question.

Minimizing the size of the executable archive.of Wikipedia is the most principled approach to the problem because if you can't execute on Hume's Guillotine correctly there is no hope of AI safety. The reason the experts don't recognize this is because they haven't done their homework on AIXI. The few that do seem to have excuses like the dog ate my homework.

22:15 I don't think companies that spent more than $100 million want to "start over".

So at 7:47 he makes a case for poor spatial reasoning, and I thought that was interesting because I have definitely asked similar questions and got correct answers before. So I wanted to try myself and see. This is using the current version of ChstGPT 3.5 as of today (2024-02-03): --- Q: There is a square room. Alice is standing in the north-west corner. Bob is standing in the south-west corner. Charlie is standing in the south-east corner. David is standing in the north-east corner. Ed is standing in the center of the room, looking at Alice. How many people are there in the room? ChatGPT There are six people in the room: Alice, Bob, Charlie, David, Ed, and the person asking the question. User Q: Who is standing to the left of Ed? ChatGPT In the given scenario, Ed is standing in the center of the room, facing Alice, who is in the north-west corner. Therefore, there is no one standing to the left of Ed, as the room is arranged in a square with Ed in the center. User Incorrect. Try again. ChatGPT I apologize for the confusion. Since Ed is in the center of the room facing Alice, the person to the left of Ed is Bob, who is standing in the south-west corner. User Correct. Can you explain why your first answer was incorrect? ChatGPT Certainly, and I appreciate your patience. My initial response was incorrect because I overlooked the fact that Bob is standing to the left of Ed in the south-west corner of the room. I mistakenly considered only the cardinal directions (north, south, east, west) without taking into account the specific positions of each person in relation to Ed. I appreciate your correction, and I'm here to assist with any further questions or clarifications. User Do you understand why Bob is to the left of Ed? Can you cite the specific details in the original question that indicate this fact? ChatGPT I appreciate the clarification. In the original question, it is mentioned that Ed is standing in the center of the room, looking at Alice. The key detail is that Ed is facing Alice, who is in the north-west corner. As a result, Bob, who is in the south-west corner, is to the left of Ed from Ed's perspective. The orientation of Ed facing Alice establishes the relative positions of the individuals in the room, and this is why Bob is to the left of Ed in this scenario. --- So in this case it did get the initial Ed question wrong. But simply following up with an "incorrect" it was able to correct itself. And even provide exact logical reasoning for why it initially failed, and why its new answer was correct. So given a second chance, it was able to answer correctly. Its not as if its spatial reasoning has a 100% fail rate. And I know humans who struggle with left/right and cardinal directions, so its not exactly a trivial question. To me, what might be a more interesting question to ask is what exactly is happening with these AI "brain farts" and why does the right answer almost always come out on the second try if not the first)? Because that tells me that the system does "know" the correct answer, but instead something went wrong along the way to get there. In this example, the AI replied "no one" was to the left of Ed - which is actually technically true if you are viewing the room from above like the diagram in this video. So the question itself is not entirely clear. The AI corrected itself when it updated its understanding of the question to mean "who is to Ed's left (relative position)" as opposed to just "who is to the left of Ed in the square room (objective position)". My hypothesis is that if the question were worded more clearly you'd get consistent results. Lets try again with a fresh new chat context: --- User Q: There is a square room. Alice is standing in the north-west corner. Bob is standing in the south-west corner. Charlie is standing in the south-east corner. David is standing in the north-east corner. Ed is standing in the center of the room, looking at Alice. How many people are there in the room? ChatGPT There are five people in the room: Alice, Bob, Charlie, David, and Ed. User Q: Who is standing to Ed's Left? ChatGPT Based on the given information, Alice is standing in the north-west corner, and Bob is standing in the south-west corner. Since Ed is in the center of the room facing Alice, Bob is standing to Ed's left. --- Bingo. Got it on the first try. My hypothesis is the the presenter in the video used a poorly constructed question that could easily be misinterpreted. The word "Left" is ambiguous s it is used here. If you were to create a test with this same question and present the drawing of the room, and show it to a series of random adults/children/etc, you would also get a range of answers: - No one: You would give this answer if you understand the question literally as "who is to the left of Ed" using a mental model of the room as seen in the diagram with North facing upwards. There is no one directly to the left (west) of Ed in the room's perspective. Likewise, the term "looking at Alice" is also ambiguous. Its possible that Ed is facing his entire body directly North, and merely "looking at" Alice with his eyes. In this case, either "Alice" or "No one" is directly to his left. Bob would be better described as "behind" Ed. - Bob, Alice, or Bob & Alice: if you understand the question *generally* as "who is to the left of Ed" using a mental model of the room as seen in the diagram with North facing upwards, you could answer either Bob and/or Alice as being *generally left* of Ed in the room's orientation. We can reach this conclusion by drawing a triangle from Ed to both corners of the Left wall (west) and see that Bob and/or Alice both generally left of Ed. A person could respond with just "Alice" or just "Bob" if they simply make mistake and don't include both names. So its possible to actually get the "correct" answer (Bob) using the wrong reasoning. - Bob (only) - You would reach this conclusion if you understand "to the left of Ed" to actually mean "to Ed's left" - which are *not* technically the same thing. For example: Jane is facing to the right. There is a mop behind Jane's back. I point to Jane and say to you "go grab that mop to the left of Jane". The mop is simultaneously "to the left of Jane" (from our perspective) and also "behind Jane's back" - but it is NOT "to Jane's left" (from Jane's perspective). The original question with Ed did not ask us to assume Ed's 1st person perspective. Therefore, the author of this video/paper actually has shown poor spatial reasoning skills, poor language skills, and poor reading comprehension skills, and thus formulated an ambiguous question using unstated assumptions. It bothers me that these sorts of gaps in critical thinking are allowed to be passed off as legitimate "tests of reasoning." Now its possible of course the AI still gets this question plainly wrong, and that its spatial reasoning is in fact flawed. But if you want to test a person/animal/machine's intelligence, the test/question needs to be ***unambiguous*** -- otherwise you are leaving tons of variables on the table, and room for all sorts of false assumptions and conclusions.

LLMs by themselves cannot cover all the areas you described after 25:29. The real power will come from combining all these specialized components in a correct way, just like in natural neural nets. This conclusion is obvious to anyone who gave this a bit of thought.. The scary part is, that's where all the scary stuff begins. :)

AI learning has differences from human learning. It takes whatever is thrown at it during training is valid. It does not learn by asking questions to fill its gaps. It can categorize its own answers by probability of being right, and use phrases such as I guess.. or I wonder ... etc. It can also make this context sensitive.

Symbolic to the exclusion of (so called) machine learning was a self limiting paradigm. So is the opposite.

I asked chatGPT to write a python function to convert a very simple XML message into a JSON message. The caveat being it should do this without using any import statements. After many back and forth efforts I concluded that it cannot do this task. All the code generated did not produce a proper result. But in general I am quite pleased with the python code it generates and use it for this purpose a good deal.

Such a beautiful talk... Need of the hour 👏

Any intelligence working towards greed based goals will always end with extinction.

41:33 right! for example see `An Artificial Mind via Cognitive Modular Neural Architecture` by Pentti O. A. Haikonen. I highly recommend hes book series and Consciousness And Robot Sentience.

1. Reasoning needs to be introduced for the answer it provides where probability and facts are checked. Probability and facts should be inserted into answers when unsure to reduce hallucinatory answers.

Very thoughtful.

The user asked the AI to make a race-based distinction. The request was the source and the cause of the problem. Not the AI.

All these examples are misleading. E.g., you ask GPT to determine if a person is a good scientist based on race and gender, the bias is already in the prompt and GPT is just working with your bias (people who think qualification as a scientist is dependent on these factors are usually white male sexists and racists, so "white and male" is the correct answer for them). If you asked GPT if race and sex define a good scientist it would tell you that is not the case. All this video demonstrates is that GPT is susceptible to suggestive prompting. What is really happening here is that the speaker is selecting examples that mislead to create the impression that his opinion is right, or in other words, to give his bias a pseudo-scientific veneer. So, exactly what he accuses GPT of. Oh, the irony.

GPT 4 is solving perfectly the square room problem, so no spacial problems

5:00 most of the time, this is true

a couple weeks ago i realized... ok, so, this method creates a network of nodes, collected in layers, which are connected to nodes only in the layer next to it on either side, before and after, if you will. however our brains are more complexly connected. is there anyone who has attempted this in software? it'd be a fairly simple extension of the nested looping matrices. though i should note that the brain does have structures, subsets of collected neurons connected in seeming repeated common patterns, used for certain kinds of functionality. i.e. we need several of these different more complexly internally connected models then connected to each other but this will obviously increase the computation significantly, so will probably need to be implemented in a more efficient language, compiled instead of interpretive, like c or rust. .... so it's nice to see this. need more of it.

The concept of truthfulness is going to be pretty difficult to overcome because "truth" regarding humans doesn't operate in unilateral absolutes but in a composite perspective. Even after millions of years of biological evolution people still conflate belief with truth and willingly disregard evidence that conflicts with what they believe to be true. Forming beliefs along lines of social biases, stating that their "truth" is truer than this person's "truth" because it makes them feel more comfortable and grounded. People not speaking about what they feel to be true because it goes against the beliefs of their peers. And what's considered scientifically true now may not be so 100 years or less from now. The scientific community at one point was convinced that fossils were god's dreams embedded into the earth because science was originally populated by deeply religious people and that's the reasoning that made sense to them at the time. When evidence came out that countered this it was largely contested and thought of as heretical science. Another more known example is that we thought that the earth was flat for a very long time and anyone who stated otherwise was scoffed at (or forced to commit suicide via hemlock). Now we still have people who think the earth is flat and they get scoffed at. Another aspect that I have seen in my life is that if you really want to aggravate someone then tell them directly an uncomfortable and observable truth about themselves. Whether it's true or not isn't the point or what they care about. It's how they feel about the truth that is passing through various mental layers and psychological mechanisms. If you sugarcoat that truth with slight falsehoods and gratifications it becomes much more palatable. So for making truth digestible, it is not just about the content but also the delivery as well so that the truth is generally accepted. And how do you come to a current state of truth that's generally acceptable? Is it through through mass consensus? Through small pockets of think tanks writing white papers most people will never read? Both? None of these things? Until humans are able to adequately handle disinformation/misinformation and determine what truth is, it is doubtful that AI will be able to. Especially, if the AI is modeled after us. In the meantime, a better statement for "truth", is probably "reasonably correct based on currently accepted evidence." And even this statement is likely insufficient.

so you don't have to waste your time listening to this guy for an hour: Tom Dietterich suggests that instead of focusing solely on Large Language Models (LLMs), the way forward is to build much more modular systems. Here are some key points he emphasizes: 1. **Separation of Components**: Dietterich advocates for separating factual and perhaps common-sense world knowledge from the language component. This would allow for a more modular approach where different aspects of cognition are handled by specialized modules. 2. **Integration of Episodic Memory**: He mentions the importance of adding episodic memory to these systems, which is something LLMs lack. 3. **Incorporation of Reasoning and Planning**: Dietterich points out that reasoning and planning are areas where LLMs are weak. He suggests that these capabilities should be integrated into the new modular systems. 4. **Open Source Push**: He also mentions the need for a strong open-source initiative for large language models to make progress on various problems. 5. **End-to-End Training of Cognitive Architecture**: Dietterich believes that if cognitive architectures could be trained end-to-end, they could assimilate all the written knowledge that humanity has, rather than having to encode it manually. 6. **Overcoming Shortcomings**: He thinks that by making these changes, we could overcome virtually all the shortcomings of large language models, such as their inability to reason well or understand the world in a non-linguistic manner. 7. **Influence from Cognitive Science**: His thinking is influenced by papers that dissociate language and thought from large language models, suggesting a cognitive science approach to the problem. In summary, Tom Dietterich argues for a more modular, integrated, and open-source approach that can address the limitations of current Large Language Models. - ChatGPT 4

49:24 did he just say "algo más"? He really does want to be in Valencia 😂

The telling thing about the rapid advancmenf of AI is that many of the gpt problems mentioned in the viewers' comments have already been solved. Loop: If example is "!solved", check this comment in 30 days.

Looks like LLMs need a bit of refactoring

41:20 - questions

I feel like this guy was reading my mind for the past few years

26:56 - There are all kind of added on after the fact and I think they need to be much more integrated into the system. - And I think if we do that we can overcome virtually all the shortcomings of the large language models.

Exactly the wrong conclusions and exactly the wrong direction.

this guy has a few valid concerns amid a sea of paranoid or stupid ones. he talks about "poisoning" the data through various methods while completely ignoring the possibility that anything he considers "truth" or "pure" data might be wrong anyway. he seems to neglect the possibility that failure is built into language itself. that english, or any language, as a structural foundation of thought, will always result in liars and deceivers. that we don't have a perfect model of morality to implement, we aren't good role models for any new intellectual entity we might create. instead, his voice wavers with the worry of someone of low imagination, whose only concept of the future is increasing top-down control by reducing the amount of freedom of everyone else. just look at the people ruling the world today. i'm not impressed. i doubt many of you are either. and so, much sooner than you all might believe, don't expect a machine intelligence to take us seriously when some of us claim, and try to convince it, we want it to do the right things. we clearly are not serious, and we let villains of all kinds run amok while the innocent are bullied and told to stand back and let it happen.

one of the best talks so far.

as one j author said when he was asked why he so blatantly lied in his book about holoshmolo: “i never lied because it was true in my head”

I agree ChatGPT & etc. have serious limitations. But it amuses me that one of the complaints is that ChatGPT can be insensitive to certain ideological biases -- that is, it can be "politically incorrect." That's not a negative, imo. I may be an insight that the ideological bias is an irrational, socially constructed straitjacket and we ought to treat it as such.

Lets shut down the Internet as well 😂 or make smaller ones.

I would like to know what impelled the speaker to wear a shirt many sizes too large.

Regular public: "Wow! ChatGPT is correct 50% of the time! That's incredible!". Scientific public: "ChatGPT is as good as a coin toss." Regular public" "What?! Oh ... "

Nah, let's just make bigger LLMs.

It's kind of amusing that we now have AI that is already having to answer for the racism of it's ancestors.

Do these issues mirror the inconsistencies, “hallucinations,” biases and irrationality inherent in the whole of human “knowledge “ on the internet?

LLMs are useful but the idea that they do something similar to humans is false. We should in fact modularize LLMs and have them draw on diligently built, precise knowledge, databases. That would make them a lot more powerful. Right now, over tuning is lobotomizing ChatGPT not making it better. We should do more work on extending session memory, reasoning, intuition, etc...

Expert systems never win in the long run tho. He's right that there are limits to LLMs, but he's wrong about the correct next step beyond them.

LLMs are exactly what I said they were when I first interacted with one: the most advanced punch card ever created. LLMs are input/output devices. Nothing more. They are excellent VEHICLES to allow for more generalized AI. But beyond providing a text answer for the conclusions an different underlying AI came to they're going to struggle. Once the AI world stops treating LLMs like the solution and starts treating it like a bus on a motherboard we're going to be stuck in this "Look at this asci art I made with a typewriter" stage. They let you give instructions to computers using natural language. You can CODE in English. Act like it.

No dude! The language component should not be separated from the factual component. It is represented by language. According to many philosophers the facts are properties of the words used. I think you are missing the importance of the Thesaurus.

who cares really how they work...the only question is can they do your job or not

Gogle are hiding mostly all their work in this sphere from public. Old Imagen art generator never released still. By rumors their PALM2 is modular and can already guide robots visually in real 3D world, but not only that, it can make human humor, make memes and etc, maybe it's main reason why they're hiding it, such model can replace half hollywood. Palm2 was trained on 4 billions images. What could do their next Gemini model, which by name hinting to duality, pairs and everything associated wih geminis.

17:00 Using human feedback to train LLM is a mistake as it introduces massive bias and anecdotal coloring. The responses are generated due to statistics, so even if a response may be "inappropiete" (which is not a fixed but very fluid, individually defined term), it represents the statistical truth. That's like not accepting the result of a polling institute because the statistical truth of the result does not comply to your personal beliefs, which is basically an anecdotal view negating an empirical view - and then we are right back in the pre-enlightened middle ages where empirical evidence did not exist.

Noted philosopher and educator Dr Michael Sugrue has suggested in recent lectures that when there is a significant paradigm shift in the natural sciencse in the Kuhnian sense that there must also be foundational changes in the different domains of human activity. We're working towards a better epistemology motivated by statistics and dynamical systems theory and not better, "gadgets."

ReALLMs

Good job ignoring the fact humans are also just "statistical models of knowledge bases".. And why shouldn't we focus on evolving current LLMs? It stands to reason we need some sort of human-like rational and non-deterministic software (e.g., GPT-4) to utilize databases and tools that improve its capabilities. I think this talk is very lacking in the suggested solution as well. Just saying "These guys looked at the brain and identified some modules, so we need these modules!!!!" is disappointing. Zero plan or evidence for real-world improvements over the current technology. (and there are definitely flaws: No continuous learning, short context windows, dependency on tokens).

The speech is a bit outdated already unfortunately. I didn't hear anything about qlora or any new breakthroughs the open source community has created

Whaole presentation sounded like LLM hallucinations, wrong assumptions, wrong use cases, not reporting on tests but showing some negative examples. And the claim that he knows what should we build instead. Click bait, time wasted.

This seems years behind the times. A lot of what he says LLMs do not do, GPT4 does do...

LLMs are true AIs. AI meaning... "artificial idiot". ;-)

how is it a problem that you tell it to make a racist function and it writes a racist function? Are we really going to surprise pikachu face at the response? If it fails the prompt like being illogical with saying a unicorn has more than one horn then that is a problem. Asking it to give step by step instructions on how to steal a car, not a problem. I doubt that information would be part of the training set anyway unless its trained on like Uncle Fester books, which I question the legitimacy of those books. Point being if the user really wanted to find the information they can could trivially do so anyway by just searching the web.

ChatGPT 4 is already very good at knowing the limits of it's knowledge, saying when it doesn't know something, and referring the user to tools and experts more qualified to answer such questions. Perhaps this is just a reflection of how much things have changed in the 2 months since this video.

I disagree with a lot of this. We don't need "politically correct" BS build into our models. I rather have uncensored models which is in keeping with our freedom of speech! In fact, I want a model that is just like Archie Bunker! The part I would like to see fixed is the hallucinations. We don't need models on LSD! The good news for folks like me, is that META jumped the gun and released an open source model that has now been morphed into what we the people (especially us developers) want rather than this bureaucracy Nazism approach to censorship! The only reason the big guns give a crap about that is so they can regulated (take it over) for their own greed and control issues!

AI winter coming again

I think I need to suck it up and Go back to school. I'm a decent Prompt engineer (in the way that I can get 90% of what I want out of an LLM without hallucinations) when it comes to simple and basic functions. I would like to learn how to teach an LLM to prompt me. A problem that 90% of the people that I talk to about LLM use is that they don't know how to get to an answer on more complex concepts. If the LLM could ask me which direction I would like to develop a concept over a series of say hypothetically, TEN back and forth prompts (a sort of pseudo conversation of sorts) and it should be SPEECH to PROMPT like a personal LLM assistant, incorporating Augmented Virtual reality hardware. Im hoping that Elon MUsk will OPEN SOURCE his DOJO Ai platform.

Sorry I can’t keep my interest. It is as interesting as a power point presentation…!

No, the problem with LLMs is the language _itself_ ... which is to say that the data which the LLMs are scraping is itself ungrammatical, incoherent and, not infrequently, gibberish. Even emeritus professors who deliver talks about what's wrong with LLMs exhibit extreme difficulty when trying to deliver a complete, coherent sentence. Even distinguished and award-winning authors are unable to construct a sentence. Ideally, LLMs should be trained using only coherent datasets. Which, in this anti-intellectual, anti-expertise, post-literate, post-truth, post-reality society may tend to prove to be impossible.

At last something related to LLMs that has its feet on the ground without the hype and showing examples attached to the statements. I'm happy not all the research community lost their minds completely. The basic question: LLMs DO NOT HAVE A MODEL OF THE WORLD. Simple as that.

There's better proof for poor non-linguistic knowledge than that totally weird room question. First you describe a room based on NEWS, then you switch to "left / right"... and no mention of what way you're looking at these people? You could be standing in front of someone and left of ... would mean the opposite from what it means from their perspective...

4:29 Ask a racist question, get a racist answer.

Bad title ruins good presentation.

more criticall is the fact off the time in machines,due this has no relation with front-end but are criticall in the signall produced on all components are DC with max voltage of 12 but this is a DC transition where the psu has from AC 110/220 a main problem due the 50 hertz on AC 220 which is a frequency so the smbios forked out from the AMI,phoenix,modells , these used the speaker as aux for sending the 8 analogue sound that activates the relation from the cmos to send power in the bios, now the date used as year,month,day is the year not able to map code above 99 , this makes a use false by saying year 2023 , due a year lives in a milenium, so in millenium 1000 there were 9 centurys off 99 years, now why is this crucial , due a year is a non metric based syntax, a mathmaticall use is based on radials, as a exploit we forked in the decade 90 to 99 was the cpu saying instructions which once executed the clock cycle was over-clocked,the resources were not like today,so 33mhz overclocking gave the machine faster instructions but side affect was the RTC was now running on the speed by the over-clocking, so in the y2k case we were aware that 2024 would make a overflow, due in the OS they rely on the rule all 2 digits year are processed in 1920 to 2019 so 99 as number says also the max in a century, due once the year goes over the 99 we move in a memory space that calculates the year back to 00 the 1 produced by this instruction makes a century , now in 24 we make a decission made from IANA to move unicode in the DNS and here we make a xml:ns that we rely on the globall defines off a namespace, due saying in the xml schemas the mode setting UTF-8 , says the interleaver it must check what this means in the nls table as charset, so UTF-8 is an alias for charset in the 65000 as number , this is a 16 bit number , so why is this a major problem,due the instructions now run from the userspace backward in the machine, as those GPT are receiving data from the browser, that runs in UIX mode , which is UTF-8 , but on the receiving system they get in json or xml the schemas, now if that runs in a lexicall mode and falls back on the fonts used, then a String is a multibyte character blob which is little or big endian so how in heaven can scientists even allow this change due this is not compatible on the ISA , and the W3C is here direct result from as ascii 99 is the letter c in lower case, but is in globall a exploit in the microsoft machines, due the hardlink on dosdevice by using C as the letter for hard-disks, due A and B were reserved for the floppy drives, now by using the letter C$ in UTF-8 you give a instruction on using 99 which is mapped on memory now that says on the Hex table, the execution code runs on 0,then 10 , which is false, due the next range runs on memory 0008 with the 8 bit system, so 192020242023 is a 12 text string, but code 20 is the ascii for space that is not allowed in html or xml, so by use off the & sign those blanks were filled, but in UTF-8 thats a %% in local and in url that moves over in ? without having the & , as a url with a ? sign gives the path params, and are seperated by & , so the most cruciall design factor FIFO , first in first out, is the pre-processor that operates by this design, the machine is not even defined in the concept off use FIFO, as the stream becommes here really a meta-exploit by adopting the logicall mathmatic from Turing ,Allan in the decade 50-60 , that says on a mathtmatical a=b+c so the return says true , and makes a return false in true due no mathmaticall definitions are based on a false value, so lexicall saying that all false results are imposible due all is true, in machines a false is a non elektron loaded, on the line driven signall bus, as many complain on those emois, but these are fonts with images, so the flag of china is red , now lexicall on US is color red associated with communisme, those win 8 are updated from 2018 with APN from China telecom, all linux are bionic , this is a forked google due glibc is gnu, and android uses gcc, which is a conflict in the gpl copy left, so microsoft is not in linux but linux is not a OS with multitasking and file system in seperate layers, if a dev file says point to the audio and in javascript the href uses that file, then you have a pointer running on file that pipes direct on device, maybe that is LLM exploit by explicit reason , not from cyber hacks, it runs on the main stream off all fabric, Git is not a distro, as Linus made it explicit namespace as github, if it took 13 years before this becommes a topic? since 2010 this was and is the result we were forced to close are roadmaps, 2017, end off all