ChatGPT is very human in the sense that it doesnt like to say "I dont know" when it doesnt know.

As a teacher I have tried to use ChatGPT with my students. The goal was to find specific errors of the answers. On one hand, the students had to show a real grasp on the material being taught to be able to find reasoning mistakes. On the other hand, it was a way to show the students to not blindly use this tool to avoid having to do their homework in other courses.

When I first heard about ChatGPT, I was having some physics fun with it. This was one of my favorites: "Explain the Unruh Effect like a gangster"..."The Unruh Effect is a theoretical prediction of quantum mechanics that states that an accelerating observer will perceive the vacuum of empty space as being filled with particles. It's like a gangster who is constantly on the move, always hustling and shaking things up, will see opportunities and threats everywhere, even in places where someone standing still might not."

Like the Professor said, GPT is just a language model, the fact that it can do math AT ALL is kind of amazing.

People are working on combining ChatGPT with Wolfram Alpha, because a language model cannot do basic maths. Wolfram Alpha can do that piece quite well.

A very cool and strange thing is that you can ask it to correct itself, without saying what it did wrong. Simply by taking it’s own output, and say ”Find any errors in this text if there are any and fix them. If there are none, respond ’ok’” or something similar and you can get more accurate results :) I guess it’s cause there so much text where people correct each other on the internet :p

I've actually been using chat GPT to help with my intro physics homework. It gets about 50% of the questions wrong but it is a pretty helpful tool. It's like working along with someone who is also in intro physics. By bouncing the ideas off of it I can sometimes get to the answer myself. This is really great bc I work full time on top of schooling so I'm not able to make it to any tutoring sessions or work with anyone in my class. Chat GPT is gonna be my saving grace this semester

Something to consider with ChatGPT: when it makes mistakes, you can point them out and it will correct them. In the end when it wrote the program and made the mistake, you could tell it what mistake it made and it can write up a new program with corrections

The most impressive thing about ChatGPT for me that it's level of knowledge is like this for every domain and field. Professor Moriarty could easily tell that the answers it gave to physics questions were wrong but if it was instead a geology exam then ChatGPT would seem like it know its stuff giving answers full of technical jargon.

11:42 - one thing to consider is that ChatGPT might have been designed to be helpful and agreeable. Therefore, it might be biased towards agreeing with statements if you ask it whether it agrees or not. And once it agrees with the statement than it generates prose to justify why it agrees whether it makes sense or not.

I really appreciate you giving the 2/3 mark for the first question. My computer engineering prof would always give us a zero if we got the answer wrong. Yep - if you got the unit wrong you'd get a zero - even if all of your conceptual understanding and work up to the end was perfect.

Hi there, if in future you are interested in testing ChatGPT's mathematics ability, it is very helpful to ask it to print the answer in a LateX format as it is much clearer to read complicated expressions once you paste it into latex

Wow how long haven’t we see Professor Philip Moriarty!? Welcome back

Plot twist: that was actually a deepfake of professor Moriarty downplaying ChatGPT's abilities to make it seem less threatening until it's too late.

You got me at "I'm anthropomorphizing" lol 16:01 I noticed when you said they before. Honestly, I don't blame you. Respect to you and yours, you have a beautiful channel here. Thank you for doing what you do.

Chat GPT might be a D-level Physics student, but the scary thing is that it likely gets a passing grade in almost every subject on earth. You can ask it questions about topics from welding to wedding planning and it is always close enough to impress domain experts.

I think many people are misunderstanding ChatGPT's function. It is NOT designed to provide accurate answers - it is designed to demonstrate a language model that correctly interprets and answers the question - from a language perspective, not from a subject matter perspective. "As a language model, ChatGPT is not designed to provide answers to specific questions, especially those related to a specific topic or subject. Instead, it uses a large corpus of text to generate responses based on the input it receives. This means that the responses it generates may not always be accurate or relevant to the specific question being asked. Additionally, ChatGPT does not have access to external information, such as the internet, so it cannot provide answers to questions that require knowledge beyond what it has been trained on. In short, ChatGPT is not intended to be used as a source of information, and it is not able to provide accurate answers to all questions." - ChatGPT.

50% impressed. 50% terrified

Great video. As former lecturer from a different discipline, I was impressed at ChatGPT's ability to generate answers to "compare and contrast" type questions, or to summarise concepts and theories. Most of the answers received were equivalent to those written by A-grade undergraduate students. The main point of difference was that ChatGPT's grammar and spelling were superior. One issue I found was ChatGPT incorrectly assigned authorship of a book, which was irking as I was the author!

One thing to note is that when tries to answer additional questions in the same branch it can be influenced by earlier questions and answers. For example in the video, it is possible that after the first question it starts mimicking a student because the chat history looks like a test answered by a student with some of the answers being wrong. Its not designed to give the right answers, just give you a the most likely continuation of a text based on its training data

I used it to write unit tests for production code. By the time I had explained all the stylistic requirements and had fixed all of the things that were missing or it got wrong, I would have been quicker writing them entirely myself. Other times, I asked it software related questions and unless you have reasonable expertise in the domain, you might not pick up on things that it gets wrong. Right now I think of it more like a sort of Stack Overflow that can get you an answer quicker. You can't just copy paste its output, but it can definitely get you going in the right direction.

2:18 is pure comedy :) but seriously, really interesting vid. thanks for keeping these going to all the crew and professors involved. even when these vids are out of my area of interest they're always rewarding to watch.

Thanks for explaining what the "A" levels mean. For you in the UK this term may be well known by everyone, but outside the UK it is a term often heard in movies and mentioned in books, but never explained. Thanks so much! As for ChatGPT: it is amazing how far AI languages have evolved!

The language model is so effective at finding plausibly associated words that it's hard to tease out how much is real understanding. It's all random noise of the exact sort we want to find. We need to create the equivalent of the null hypothesis that factors out the noise somehow. Keep in mind that most of these questions are standard questions. They're discussed in textbooks, they're discussed online. Telling it about a new subject (that wasn't published when the NN was trained), then expecting repeated *exact* answers, would be a lot more meaningful.

I asked it some questions for a logic test for job applicants, for example a finding the next number in a given sequence, and it always found the number given in the tests answer but came up with some wild reasoning that I am not able to comprehend 😅

Absolutely fascinating, I'd love to see more videos like this

Amazing! Excellent! Demonstrates the dedication and quality of your exam questions!

Not only is this a great upload for the ChatGPT but I learned a lot about what lecturers are looking to get from students. Cheers!

Thank you for such an interesting video. My thoughts on the integration of Wolfram and ChatGPT. It was a great article about being able to access the Wolfram language using ChatGPT. Furthermore, it was particularly impressive to see how Google Sparrow could interpret pictures and graphs. This brings up an interesting discussion of the specialist versus the generalist, as one must know what they want to ask in order to get the desired results.

I have been giving it my physical chemistry (400 level) homework problems as I solve them, out of idle curiosity, and it has taught me two things. 1.) It's incredible how often it will correctly pick out exactly what formula it should be using for even statistical thermodynamics, only given a desired outcome and variables. 2.) It's also incredible how confidently wrong it is at just solving the math once it's plugged everything in. I feel like combining chatgpt with Wolfram alpha would be terrifyingly powerful. And I can also say that several times it's gotten me un-stuck when I saw a problem and straight up didn't know what formula to use, there was about a 50% chance chatgpt could figure it out. It's been more useful than chegg, which has people posting the exact question I'm working on, only to have it very incorrectly answered about 95% of the time. It can be downright astonishing when given language based tasks. I fed it the thesis I wrote for a final paper in African American literature after I finished, asking it to "compare the works of Philys Wheatley to Richard wright's 'the ethics of living Jim crowe' and provide examples of both individual racism and state sponsored violence against African Americans" and I was utterly blown away with the detailed, thoughtful and downright INSIGHTFUL answer it gave. Chatgpt understands racism better than the average American and that's equally impressive, depressing, and terrifying

I love how it answered him so convincingly he felt bad not giving full marks on a question

Love this lecturer's attitude. Muat be fun having him teach your courses. Great video

this is great, i wish we could see these same audits with other disciplines.

The best description of large language models I ever heard came from Timnit Gebru, who called them “stochastic parrots”. They’ve consumed enough of other people’s language to give a superficially convincing facsimile of conversation, without developing any conceptual understanding at all.

This was fascinating. I can see this being used as a tool by specialists in various fields to come up with basic frameworks and ideas. Amazing how quickly it cranked out that code, would have been fun to see how well it actually worked.

I am amazed you got it to give you such a long code snippet in one parcel. One of the biggest problems I have is the code snippets it gives me are too long for the chatbox, and sometimes when I inform it that it cut off and ask it to resume pasting from the cut-off point, I get entirely different methods and functions, it is very challenging.

ChatGPT is pretty amazing. It certainly has its limits, but I love toying with it. I was quite impressed when I asked it to create a Python IDE with an interactive interpreter window that displayed the list of local variables and their values in a separate window... and it did. I was quite impressed that it understood that it needed to update the separate window list of locals after every command entered in the interpreter, and that it made it a sensible GUI. I have a few times either run into limitations that they put in place intentionally or limitations in its own reasoning... I pointed out to it how humanity had reacted in the past to developments of new technology which caused people to question the 'specialness' of human beings and change what things they saw as being "what makes us human." Despite all prior historical reactions being very negative (like John Henry being a folk hero who was famous for committing suicide rather than be out-worked by a machine) and resulting in humans seeing things they had previously seen as very negative aspects of human nature (self-destructiveness) as heroic and virtuous.... I could not lead it to the obvious conclusion that if we produce an AI which has been carefully guardrailed to omit all sorts of biases and antisocial ideas, we will produce an AI that is a 'better person' than any actual human is capable of being and that people will react to that by holding up hate, bigotry, aggression, violence, etc as "what makes us human". Our vices will be seen as virtues. Kindness, egalitarianism, etc will be insulted as "acting like a machine." ChatGPT wouldn't go there, though. It just kept saying people should work to make AI systems that are devoid of perpetuating biases, etc. It couldn't see the obvious ramifications of doing exactly that and how it will lead to horrors as humanity flails about attempting to define themselves as special in the universe.

I'm a graduate student studying computational media, and I find these types of interdisciplinary, expert analyses of AI to be very interesting.

I just thought I was the only one doing this. I have created study guides for certain complex topics in physics in 5th grader content language which is great for helping better explain certain difficult components to grasp. This was literally my first project as I was creating a proposal in November last year and I needed to quickly put together study guides that were easy to digest.

I follow several of Brady's channels, but I haven't been paying too much attention to this one, I think this video will change that. I love the professor's passion and I find this subject fascinating, especially the part where GPT is much better at coding than he thought, but it still didn't quite grasp the physical concepts. Really interesting video.

While it is easy to point out basic failures (getting max KE wrong), the fact that it is talking about the right things is itself very impressive. It could say “Yellow is smelly at t=23”. Just being in the correct context with its answers is phenomenal. The answers are close enough to correct that we are critical, not so far off that we can just dismiss it entirely. These are early days of the technology. While it may be a while before it is flawless in its answers, I expect the quality of answers to improve rapidly as AI designers get better at training these systems.

It would be interesting to see what would happen if the next generation of algorithm incorporated Wolfram Alpha's database, or if it was hooked up to a similarly powerful image recognition and analysis algorithm...

4:52 One of the ways it really helps in speed-cramming something I'm unfamilar at. It kinda shows some sort of template that someone unfamiliar wouldn't immediately think of, and while wrong, the template gives enough of an idea to help correct the wrong numbers-n-stuff that it spits out. (at least, that was some of my experiences)

How lovely to see Prof. Moriarty back on the channel!! Wonderful episode :)

I find it incredibly useful as I’m learning a programming language as it can point me in the right direction better than any other resource I’ve found. It writes code that sometimes doesn’t compile and often doesn’t do what it says, but that’s fine when it’s a “you could use this class…” and I’ll figure it out. It’s also amazing at “show me how to use function ‘x’” when I just need to see some simpler code.

9:52 Thank you for saying this, it really is the most important part to understand the stuff. Mistakes happen to everyone.

This comment was written by ChatGPT. I love your videos as they are among the best. They are both entertaining and educational with a decent serving of enlightenment. 😀

For the last example, when ChatGPT is wrong, you can tell it what's wrong and tell it to recode with that mind. It can refine it's answer within the context of what's being asked.

I asked ChatGPT to show me an example of a module in numpy as I wasn’t familiar with it, except my phone auto-corrected numpy to bumpy! ChatGPT took it in its stride and just invented the bumpy library and wrote code with it, even though it doesn’t exist! Amazing, in its own way!

13:40 Ah matplotlib, the library chatGPT always recommend and never works lol

Nice... never tought I would see something like that on my lifetime, as a physicist and programmer.... Wonder what the next 20 years will bring us... Cheers!

I enjoyed seeing how impressed the professor was as it generated the code. It seems like its coding is one of its greatest strengths.

If ChatGPT gets a Math Backend, things will get a lot more interesting. This has to be combined with a latex parser, else there is too much ambiguity in the English language

I had it solving graduate level classical mechanics problems from Goldstein's book. It was spot on in about half the cases I tried. Frequently, a nudge in the right direction was required. In some cases, I'd say 10-20%, no amount of nudging would generate anything useful, or even, at times, comprehensible. Also, it understands Latex notation very well. It can even handle, evidently, a kind of "quasi-Latex" combination of Latex and straight text.

I just subscribed to Sixty Symbols and Computerphile.I hope you all have a great rest of the weekend.

It's the wording in the question that need to make it straightforward for the ChatGPT to be able to fully capture and go forth in computing the result for you. There is this "prompt engineering" that we had to be specific to what questions are we asking and what sort of details best encapsulate the task to process it.

What's interesting about ChatGPT is that it can answer correctly some advanced undergraduate-level questions in quantum mechanics, yet it fails to multiply correctly two-digit numbers using long multiplication. At the same time, it can write computer code in any language that applies the algorithm of long multiplication correctly. It will try to produce a reasonably sounding answer to anything you ask it to do, and might get about 70%-80% of the answers correctly, but it will screw up in places where almost no human will make a mistake.

Great video! Also nice to see a real professor test with real questions. Instead of all the clickbait articles stating Chatgpt passed a MBA. Looks like for physics it isn't ready yet.

Fun thing to think about, it was correct in saying the velocity is maximum when it is displaced most from equilibrium, similarly to when it at the equilibirum, just with different directions and assuming its moving in both directions.

This might be the start of Borges' Library of Babel, which is a nightmare. It contains everything that can be written. There is a book that contains all information of the universe, but there is no way to find out where it is and when one finds it to assess whether it is valuable or nonsense, since the library has ALL possible books.

Hello. Can someone blind become a physicist/physics student? If so, what accommodations are made for them regarding interpreting graphs and diagrams? Could those accommodations be used with ChatGPT?

Great video. Thank you.

6:55 No, I did not know what "A level" means since I am from Germany and not familiar with this system. Thanks for explaining / clarifying! :)

As a software engineer I've found chatgpt genuinely very useful in my day to day work. I guess it's lucky I'm a software engineer and not a physicist!

I'm imagining ChatGPT being a student in Prof Moriarty's class lol

I think its outputs could be a great teaching tool, both for critical thinking, and for the subject itself. I wouldn't know how to answer the oscillating particle question myself, but even I could immediately see that the first paragraph was completely incorrect. I have no idea if the v=(2*PI*A)/T part is correct, but I could easily check that at a more reputable source.

Very cool stuff. I'm not great at the physics stuff anymore, but if as a student you did know the last question needed certain limits, you can tell chatgpt to adjust the code however you like and it will modify what it gave you before.

My Quantum Mechanics teacher failed I guess... Thanks for explaining why I didn't understand anything that semester.

I'm facinated about the mission statement thing for a specific reason. I was on Academic Council at Murdoch Uni in West Australia, about 15 years ago (As the representatitve of the Student Union), and one thing I discovered is the academic council had been arguing about the definition of "Interdisciplinarity" in the Uni Mission statement almost since the uni first opened. Like, to the point of screaming matches between the chair of philiosophy and the chair of english and literature. It was FAR from resolved by the time I left, and chances are its STILL not resolved. And this thing has implications because its actually in the Murdoch University Act of the WA government, so it has legal obligations attached that nobody seemed to be able to work out what they where. Maybe they should ask ChatGPT

This is really interesting. I made similar experiences with chatgpt. I tried to use it as an assistant to solve a graph theory problem. It was incapable to understand what a connection between two nodes is. I tried to explain this to chatgpt via a matrix environment as a rudimentary drawing sheet. But it failed to draw the correct conclusions from the rules I set when I asked it to decide if two given nodes are connected.

3.00 it has actually a limited capacity to OCR images but you have to link it the image

As a perfectionist, Life has taught me I need to strive for mediocrity on some things.

I think what's being downplayed is the amazing role AI will play in research (yes high level academic research) both in terms of making writing easier, performing reviews on specific topics in the literature, and an aid in data analysis. Word has spellcorrect. Soon, we will have ChatGPTA

The most fun thing about it is conversation, not just independent questions. Its main feature is dialog after all

I love the frame centering on the paper at 5:26.

I'm going to be honest, considering how far these AI's have gotten over a rediculously short timespan, I wouldn't be surprised if there soon existed an AI that could both do proper explanation of the matter, and properly understand the physics and mathematics to fool even someone of a very high level of understanding within said field of study. It's both equally horrifying and amazing to watch I'll be honest.

I had 2 years of Physics in college and I couldn't even answer those middle school questions. Physics always went over my head.

Edit: I brought up integration of ChatGPT with Wolfram Alpha as a module as a solution for arithmetic being an issue on Q1, but I see others have already mentioned it lol. But I do wonder how long or if we'll ever reach the level of self-awareness and "reasoning" that Professor Moriarty is looking for in some of the conceptually harder questions. (And perhaps just as interestingly, if not, whether AI can still approximate the desired result without approximating that human process - much like how many of us though AI didn't have the creativity for art and poetry, which is true - and yet, it clearly can produce impressive art and poetry. Surely the answer is yes, since it's already remarkably close in some ways, but getting it to parse those nuances both in language AND in concept AND in convention will certainly remain a challenge. And then how long will it be until AI can solve the first conceptual questions that a human can't? (Of course we know that computers can calculate answers better than us. But will they ever reason (and/or "reason") better than us too?)

What I find really interesting is when you point out mistakes that ChatGPT has made. In the context of that conversation, it remembers the correction, and appears to /somehow/ incorporate that correction in further responses. As to how it does this, and to what extent that "learning" is actually occuring, I have no clue.

I guess the internet (and hence ChatGPT's training data) is full of mission statements and code, but answers to physics exams, not so much ;)

Might be worth retrying these questions! The update from 30th Jan has improved factuality and mathematical functionality

I find that when you have it re-answer your question it sometimes provides you with a a different answer. After a few times you should get an answer which is either more sensible or less sensible in terms of answering physics questions.

2:50 it actually does... You can DESCRIBE the diagram to it, then say "using the diagram previously described, tell me why X" It can do incredible things, it's more about how creative you are in asking the question.

There's a thing that I noticed people don't try: when it gets a wrong answer, you can confront it and it will made a whole new thing and sometimes get the right answer after that. It's scary, it analyze its own answer and try to output a more reasonable one

Awesome to see Prof. Moriarty again!

Great video :) and Thanks Mr Moriarty

I'm a c#/Unity developer and I use ChatGPT sometimes to find quick solutions. The code is sometimes flawed, but it can serve as a quick-start from which to get something up and running.

I played around with Chat GPT and after a few hours I felt like it would be faster to just write it myself. However Chat GPT is great for breaking past any type of "writers block" and if I get stuck on anything it's great to bounce ideas off of.

I asked ChatGPT the other day about series convergence and it said that the harmonic series 1/x converges. That pretty much sums up why ChatGPT is useless when confronted with mildly complex questions.

The best use of GPT3 , i found is the data sorting based on filters or parameters. The data being more massive and obscure in various places on the web makes it shine even more.

Try asking it to write a paper for a specific journal on a specific topic in the style of a specific prolific author using LaTeX. The results can be fun. Don't forget to ask it to continue if it stops abruptly.

I tested ChatGPT a while ago and it was terrible at understanding rules to a word game (e.g. name some things, but not things such as this, or such as that, first you, then me, one at a time), and it didn't apply the rules correctly (e.g. named some mammals, even though one of the rules was to not name mammals). A few weeks I check in again and there are big improvements. It's learning fast! Next time you ask it about physics it will probably be better.

You know it's just going to keep getting better. right now you're training it and it's learning.

Try using prompt engineering. It will be more focused. Example tell it "act as a physics teacher" and then ask it the question. I asked it something relatively simple. I have 2m of wire. I want it cut in 450mm sections. How many sections can I get and what will the length of the final section be. It got the answer hopelessly incorrect. Then I added the same question and prefixed it with"act as a maths teacher". It explained every step and it got the answer correct.

I majored in computer science years ago. I use chatGPT everyday. It's great to generate boilerplate code, but also to quickly generate code examples of things i'm not familiar with. For instance, i wanted to know about SVG images. I just ask him questions, and he generates the code. It's always an approximation of what i want, but i just tweak it a bit, and it works. I also use it to ask a lot of technical questions. I also use it as a dictionary and helper for language studies. You can ask him question, and ask him to answer in another language for instance. I mean, it's great all around. I haven't really used google or stackoverflow in the last 3 weeks.

Researchers have combined chat gpt with various other calculating tools, such as Wolfram alpha, and it seems to be a very powerful next step to enabling better and more precise computation.

Thank you for sharing Mr. Jedi Physics of Nottingham.

I don't give the professor full marks, as he is unwilling to consider that the harmonic motion question was a bad one, or that the chatbot got it right for different (but not necessarily invalid) reasons.

"Committed to Excellence" Is there any word more nebulous. Me: Nebulous? I don't know about that. Cliché? Yes, definitively. Do you know how overused that phrase is? Heck the company I work for has it in as pat of their mission statement.

Does it find a anwser from some one ( or many had) uploaded ? Or does it really gives a thoughtful answer ?