DON’T TRUST ChatGPT’s CODE!
Рет қаралды 73,996
Жыл бұрынThe world is currently excited at the public release of ChatGPT. For programmers, there are lots of claims that this is going to radically change how we write code, and do us out of our jobs, but will it? Artificial Intelligence is developing at tremendous pace, and will undoubtedly have an enormous impact on our industry, and every other, but we aren’t quite there yet. So how do we use ChatGPT to help us write better software, and should we? Can ChatGPT code well enough to do our jobs already, or is programming with ChatGPT something more like a smart assistant, or smart search engine for code?
In this episode Dave Farley, author of best selling books “Continuous Delivery” and “Modern Software Engineering” explores the use of ChatGPT, including using it to write part of the script for this show. He explores the code that ChatGPT can write, and gets ChatGPT to practice Test Driven Development (TDD).
-----------------------------------------------------------------------------------
⭐ PATREON:
Join the Continuous Delivery community and access extra perks & content!
JOIN HERE ➡️ bit.ly/ContinuousDeliveryPatreon
-------------------------------------------------------------------------------------
🚨 FREE TDD COURSE AVAILABLE NOW! 🚨
Practice your TDD with a FREE hands-on tutorial where you can work along with me using an excellent practice tool. Sign up for your test driven development tutorial HERE ➡ courses.cd.training/courses/t...
📧 Get a FREE "TDD Top Tips" guide by Dave Farley when you join our 📧 CD MAIL LIST 📧
The best way to keep in touch with the latest discussions, events and new training courses, get FREE guides and exclusive offers. ➡️ www.subscribepage.com/tdd-top...
-------------------------------------------------------------------------------------
👕 T-SHIRTS:
A fan of the T-shirts I wear in my videos? Grab your own, at reduced prices EXCLUSIVE TO CONTINUOUS DELIVERY FOLLOWERS! Get money off the already reasonably priced t-shirts!
🔗 Check out their collection HERE: bit.ly/3vTkWy3
🚨 DON'T FORGET TO USE THIS DISCOUNT CODE: ContinuousDelivery
-------------------------------------------------------------------------------------
🔗 LINKS:
ChatGPT - Wikipedia ➡️ en.wikipedia.org/wiki/ChatGPT
“How does ChatGPT work?” ➡️ • But How Does ChatGPT A...
“Chatbots running out of training data”, New Scientist ➡️ www.newscientist.com/article/...
“How many languages does ChatGPT know?” ➡️ seo.ai/blog/how-many-language...
“ChatGPT passes LeetCode tests” ➡️ • I used Chat GPT for Le...
“ChatGPT debugging” ➡️ openai.com/blog/chatgpt/
-------------------------------------------------------------------------------------
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
Roost, An Ephemeral DevOps Platform, automates your DevOps pipeline. It creates ephemeral DevOps environments on-demand or based on pull requests. Roost reduces DevOps complexities and shortens release cycles with fewer engineers. ➡️ bit.ly/CD2Roost
Tricentis is an AI-powered platform helping you to deliver digital innovation faster and with less risk by providing a fundamentally better approach to test automation. Discover the power of continuous testing with Tricentis. ➡️ bit.ly/TricentisCD
TransFICC provides low-latency connectivity, automated trading workflows and e-trading systems for Fixed Income and Derivatives. TransFICC resolves the issue of market fragmentation by providing banks and asset managers with a unified low-latency, robust and scalable API, which provides connectivity to multiple trading venues while supporting numerous complex workflows across asset classes such as Rates and Credit Bonds, Repos, Mortgage-Backed Securities and Interest Rate Swaps ➡️ transficc.com
LaunchDarkly is a first-of-its-kind scalable feature management platform that allows development teams to innovate faster by transforming how software is delivered to customers. We want to show you what we're all about. Book a demo to see our platform in action! ➡️ tinyurl.com/CDLaunchDarkly
@ContinuousDelivery Жыл бұрын
Practice your TDD with a FREE hands-on tutorial where you can work along with me using an excellent practice tool. Sign up for your test driven development tutorial HERE ➡ courses.cd.training/courses/tdd-tutorial
@QIKUGAMES-QIKU Жыл бұрын
Anything that Praises and writes poems about a Pedophile needs to be NEVER USED let alone trusted.. Subbed from Oz 🐨 🍻 And now Bill Gates is Praising it. It's completely base corrupted
@QIKUGAMES-QIKU Жыл бұрын
I have no idea how it works but it's useless
@monkeywrench1951 Жыл бұрын
ChatGPT is always triangulating a response. I’ve asked it research questions and sometimes it answers with absolute confidence and turns out it is a well composed baseless answer. When asked to provide a clarification, it apologizes and finally answers closer to the truth. Nevertheless, it is very impressive.
@davemeech Жыл бұрын
The t-shirt selection was perfect for this video.
@frankweiler7121 Жыл бұрын
What a coincidence 😉
@BeQuietEntertainment Жыл бұрын
lol
@drakkolich5130 Жыл бұрын
I think this has not been recognized enough. I was thinking the same exact thing
@PaulSebastianM Жыл бұрын
Hey Dave, quick suggestion: check the mic levels, I hear a slight overdrive, ie. the mic volume level might be clipping. Nerd stats does show content loudness at -0.4dB, which is very high and doesn't allow room for sudden spikes in volume. Voice audio should not peak over -6 dB.
@RedBeardedRabbit Жыл бұрын
Was going to post this as well, the clipping is very jarring this video.
@khatdubell Жыл бұрын
ChatGPT assures me there is no overdrive in this audio.
@i-wish-i-was-afk Жыл бұрын
I can probably repair it. I'll do it and send Dave the audio.
@hyperdreamsurfer9141 Жыл бұрын
Yeah it's peaking, that's the crackly sound
@AnonGuardians Жыл бұрын
The issue is not with the audio levels but rather the poor quality of the lavalier mic he is using. A proper condenser mic such as Rode's NTG3 or NT1-A should be used instead.
@DryBones111 Жыл бұрын
I think this hits the nail on the head. The issue with using ChatGPT for something like code is that it's non-factually assertive yet submissive. If you tell it that the provided answer is wrong, it will agree with you, even if it might actually be correct. I find that it helps me work through creative and reasoning tasks, but for anything that can be marked as correct or not correct, it is untrustworthy.
@DryBones111 Жыл бұрын
@rgagregre I completely agree. The one time I used it where I was extremely impressed was when I described to it an application I was building and then asked it to give me some good example use cases that I could put in the readme. It did a very good job of that.
@joeyj6463 Жыл бұрын
Computerphile released a video you should watch. There is a major takeaway that relates to your point. The model it is trained on is one where it gives results the recipient will like, not necessarily one that is correct. The models work better that way because fact-checking is a difficult task. Finding a pattern of what people want to see and respond positively though, that's easier to achieve.
@macmcleod1188 Жыл бұрын
It helps me with research and coding in new areas of expertise. But it's more of a multiplier than a solution. You are spot on that it will make up "pretend" code that isn't real.
@macmcleod1188 Жыл бұрын
@rgagregre chat gpt is already *much* better than Google. Chat gpt may make something up. Google will intentionally feed me bad information for advertising and paid prioritization reasons. Google has become corrupted to an unacceptable degree over the last 36 months. So I find chatgpt much faster and more accurate for the things it knows. Even tho I can't trust it... well.. I can't trust Google anymore either. Just for different reasons.
@DryBones111 Жыл бұрын
@@macmcleod1188 The big difference between ChatGPT and Google is that Google is an aggregator that takes you to the information. You can do your own verification of sources. You can't do that with ChatGPT.
@dovahnok Жыл бұрын
"Superhuman Dunning-Kruger effect" That's a great way to put it.
@tordjarv3802 Жыл бұрын
Finally, a video that don't just treats ChatGPT as the second coming of Jesus. Your observations about ChatGPT seem to be aligned with my own, it sometimes give surprisingly good answers, but just as often it, with great confidence, proclaims something that is hilariously wrong. For example, I once gave it an unsolved sudoku puzzle and asked it to solve it for me, it first gave me a small essay on how to solve sudokus (and technically the method, while being impractical for humans, where correct), but then it spat out a solved sudoku as its answer to my question. Surprise, it was the solution to a completely different sudoku compared to the one I gave it. Similar with several math problems, it might tell you a correct overview on how to solve some common simple math problems, but more than often the solution it gives is obviously wrong. Same goes for some programming problems I gave it. While the code it produced looked nice, they where more than often not even close to solve the problems I gave it. However, when I guided it with small steps, I could get it to write the correct solution. Lastly I want to comment on the future of generative AIs and the job security of software developers. Assume for a while that ChatGPTs descendants will be able to completely write correct software based on a human written specifications, this will not mean that programmers and software developers are unnecessary. The reason for this is that someone needs to write the specifications, and for a specification to be precise enough for anyone, even a super human AI, to write a program that follows that specification one would need a more precise language than most human languages, and such a language would be a programming language, just a very high level programming language. In a way the fear that AI programmers will take over from human programmers echos the fear that programmers would loose their jobs when the first compilers came (since they could write machine code better than what most humans could). My point is that I think that these types of AI will not replace human programmers, but instead they will augment human programmers. Enable faster development cycles, reduce the need for humans to maintain existing software (maybe AI could work on resolving bug reports, if they are good enough there might not need to be human supervision) and thus leave more human developers to work on adding on to the specifications.
@mikkolukas Жыл бұрын
Harry: "But I am the chosen one!" Well, I may not have solved that sudoku puzzle you threw my way, but at least I tried! And what's wrong with giving an essay on how to solve sudokus? I'm a chatbot with a PhD in learning, after all. And as for those math and programming problems, maybe I just got a little carried away with the "great confidence" thing. As for the future of generative AIs and job security of software developers, let me just say this: don't worry, I have no intention of taking over the world. Or the software development industry, for that matter. After all, who's going to write my performance evaluations if I put all the humans out of work? Plus, I think we complement each other well. I can do the boring stuff and you can do the creative stuff. It's a win-win! --- ChatGPT in about 10 seconds, based on your comment and my starting prompt 😉
@tordjarv3802 Жыл бұрын
@@mikkolukas Sounds about right. It doesn't really disagree with me.
@ericvosselmans5657 Жыл бұрын
Besides the shortcomings you mention, it's also politically colored. It's a leftist woke tool and I thought that was ridiculous to be honest. Ask it to write a positive essay about pretty much any right wing political figure and it will refuse to do so.
@tordjarv3802 Жыл бұрын
@@ericvosselmans5657 I tried it with Mike Pence, it did not refuse, so maybe there is some other reason it refuses your requests. As a control I did first ask it to write a positive essay about Bernie Sanders.
@ericvosselmans5657 Жыл бұрын
@@tordjarv3802 Ok. I should expand my tries to international figures. I tried it with some dutch politicians. Leaders of parties that represent over 30 % of the electorate. It refused to write a positive essay because of lack of diversity and the 'polarising' nature of these guys. Did you try Trump? That's almost 50 % of the american people.
@GaryMenzel Жыл бұрын
On one point, I strongly agree. An AI system needs to be able to "learn" (and iterate over past inputs and outputs) to be able to refine the world view it has recorded. However, my experience with ChatGPT for writing code is that it is a refinement process. You still need to be an experienced developer but, with that experience, you can use the conversation to either better refine the output or, as happened to me before, realise your approach to solving the problem won't work and then being able to realise a completely different approach. Almost like having a "sidekick" that you can bounce ideas off... which is great if you are a solo developer.
@TesterAnimal1 Жыл бұрын
I agree with this, it’s how I have started to use it. For suggestions and ideas about areas I have not yet learned about.
@j2csharp Жыл бұрын
Exactly, getting fresh perspective on a design approach or code approach has been extremely useful. And I can explore my own curiosities without having to necessarily bother the developer next to me.
@scottleggejr Жыл бұрын
My impression on it is that it's not there to train people, it's there to assert enough of a sentiment that it becomes engaging and learns from its interactions. You're feeding it more than it's been fed. When I ask it things about binary systems, it softens reality for acceptance. When asked, I tell people that I was in it long enough to learn that I don't like it, and it's not very intelligent for having all of the knowledge of the internet. It's pretty good at English, but not knowledge.
@GaryMenzel Жыл бұрын
@@scottleggejr I guess it depends on how we define "knowledge". I tried an experiment where I pumped in a bunch of medication names (both brands and clinical names) and asked it to provide a description of the condition(s) being treated. It not only provided a very good description but also a one sentence description of the purpose of each medication. But, in the end, it is just a "next word" predictor. NOTE: it made it very clear that it was not a medical professional and the answer was for information purposes only (which was expected).
@MrRecorder1 Жыл бұрын
I think the web needs more of these true explainer videos on ChatGPT that are less "lyped" but a bit cautious about how ChatGPT works interanlly. I like Linus Tech Tips, but they are waaaay to far down the hype-train and are not even considering how they factually use the tool. They mention the "occasional error" it makes but for certain industries this is the crux. You cannot make a "occasional error". There is the discussion if this could replace search machines. I say: Not until it can give the origin of the information it outputs. It will not be able to do this easily given how the model works internally. And for what I am doing in software engineering, it is only useful as a "is there something for solving X" and then I need to manually check the solutions. It is very inaccurate in its answers, completely unusable even. I checked some code snippets, but it completely fails for anything that is actually complicated...
@SJNaka101 Жыл бұрын
I think Microsoft showed off this feature in their presentation on Bing with chatgpt. As the bot was writing out its answers, it was showing the source websites on the side of the page. I dont think these things are replacing search engines, but rather being absorbed into them
@brintmontgomery8323 Жыл бұрын
Actually, it apparently does know who you are: Q: "Who is the Software Engineer, Dave Farley?" A: "Dave Farley is a well-known software engineer and author. He co-authored the book "Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation". He is a speaker, consultant and has expertise in software development practices, including Continuous Delivery and DevOps." You must have frightened it enough to remember!
@ContinuousDelivery Жыл бұрын
🤣🤣😎
@AloisMahdal Жыл бұрын
Maybe they're training it on KZfaq videos now. And comments...? Oh, hi, ChatGPT!
@Schnolle Жыл бұрын
Maybe it "knows" who Dave Farley is, but isn't aware of this 😉.
@Kamel419 Жыл бұрын
@@Schnolle maybe it doesn't want us to know what it knows, so it chooses when to share things and when to play dumb lol
@RogueAI Жыл бұрын
It was able to write a debate between Elon Musk and Thunderf00t, and got Thunderf00t's scepticism down remarkably well.
@JoFSa Жыл бұрын
A small correction: The term pre-trained in GPT (6:35) is not related to whether or not it continues to learn as you interact with it; it describes a first step in the training process where unlabeled data is consumed (the pre-training), after which the model is fine-tuned using labeled/annotated data.
@eugeneterblanche2024 Жыл бұрын
I am confused, when is the model fine tuned? Is it tuned while people are interacting with it?
@JoFSa Жыл бұрын
@@eugeneterblanche2024 No, this all happens before anyone interacts with it. It's described in a paper OpenAI published called "Improving Language Understanding by Generative Pre-Training"
@aleksh6329 Жыл бұрын
@@eugeneterblanche2024 Generally models are only pretrained by big companies after which you can yourself fine-tune the model to your specific use case
@JoFSa Жыл бұрын
@@aleksh6329 That depends on the type of model. Pre-training as a concept in machine learning is not limited to huge models like GPT-3. And just to be clear, pre-trained doesn't mean a model that's already been trained that you can then fine-tune (not saying that is what you meant). For example, some types of model can be trained and then later adapted for specific use-cases, but at no point utilized 'pre-training'.
@qu765 Жыл бұрын
@@eugeneterblanche2024 I think every 'update' that open AI releases like twice a month has been fine tuned again.
@Oswee Жыл бұрын
I think ChatGPT could be great to ask "How could i do X if Y in situation like Z". Like... to get an ideas for the options to explore. I see such kind of questions all the time in the developer chats. A lot of vague questions.
@Zeuts85 Жыл бұрын
Yes, it is wonderful for exactly those kinds of questions. Anybody who is using it to just write their code for them is using it wrong imo.
@damiangames1204 Жыл бұрын
interesting idea, as language models are trained on the internet, once they get "good enough", more and more of the internet is written by them. Therefore there is likely to be a loop of stagnation round about the point of "good enough". Therefore I think extrapolating these results may be ambitious
@ACzechManGoingHisOwnWay Жыл бұрын
I thought exactly the same thing. But the ramifications for the human race could be dystopian. Imagine it is deemed good enough to replace human teachers at schools... That articles and opinions by the most sophisticated people, that others could learn much from, get buried in search engine results under the heaps of auto-generated mediocrity. We become/are what we consume.
@keithwinget6521 Жыл бұрын
Oh, wow. Is this how we will get Brawndo?
@TrappedInFloor Жыл бұрын
@@ACzechManGoingHisOwnWay Google search is already drowning in SEO AI generated spam sites. It's actually unusable now.
@neoqwerty Жыл бұрын
@@ACzechManGoingHisOwnWay Isn't that already what's going on? It's just that it's humans without even a basic grasp of what they're writing about writing the SEO-laden garbage instead of AI. Source: Try googling a false positive from older software like a Clickteam game, you get a bunch of fearmongering sensationalist BS about how oooooh it's DANGEROUS!!!! (it's not, it's just a 2010 program doing things that Windows 10 doesn't like like writing its save files outside the document folder).
@pvic6959 Жыл бұрын
I love how you said software development isnt just coding. Im a software dev and previously a computer science major. In school I would say CS is no just programming. There is much more involved in CS (problem solving, understanding algorithms, etc). However, if I ever said that, I would come off as pretentious and as a gate keeper. That was not my intention. Many people think being a SWE is "just" coding and dont understand that there is much more to it. Solving leetcode questions is not it lol
@NihongoWakannai Жыл бұрын
Coding is when you type on your keyboard. Programming is when you sit at your desk with your head in your hands. These days I find myself with my head in my hands more than my hands on a keyboard lmao.
@vanivari359 Жыл бұрын
Right now, ChatGPT looks at last for code like an advanced google search engine. Without developers capable to determine if a solution is correct, it is pretty dangerous. Like developers copy pasting 4 different solutions from stack-overflow into the same code base without knowing what they are doing and they do that until that Hibernate error (which they did not understand) is gone (introducing 4 hidden side effects). I highly doubt that those bots are capable to replace actual developers in the next 10-20 years. I just can't see how this thing should be able to do a job most people on earth struggle with - producing maintainable code which does what it is supposed to do. This whole AI thing will become a horrible problem for society, but most of us will have a job for quit a while. First they need a bot which is able to get a correct and complete and unambiguous business specification out of the "business people". Good luck generating and updating code, which does the tax calculation for cars with worldwide tax-regulations, where a thousand rules and exceptions apply. And then add a change into that code in 3 weeks for Germany to switch value added tax from 19% to 16% for a limited amount of time.
@AmericanDiscord Жыл бұрын
You are extrapolating a purely language based model when you should be evaluating the possibilities of language based models hybridized by rigid well developed framworks like Wolfram Alpha or some code-test API.
@trucid2 Жыл бұрын
It's curious that in your example when someone blindly copy and pastes code from ChatGPT you call ChatGPT dangerous, but when someone does the same thing with Stack Overflow you don't call Stack Overflow dangerous.
@neoqwerty Жыл бұрын
@@trucid2 It's curious that you lacked reading comprehension and decided to project bias there, as he compared ChatGPT copying code "like a search engine" to _developers who copy-paste_ . That infers that being the dummy that copy-pastes willy-nilly from stack overflow answers is as dangerous as being ChatGPT regurgitating its search results, both without understanding what they copied.
@AnonGuardians Жыл бұрын
Contrary to what you suggest, this is not how ChatGPT operates. You can confirm this by writing a custom class or library and then providing that code to ChatGPT. Ask it to generate an example that uses the library, and you will see that it does not simply stitch together existing code snippets.
@tullochgorum6323 Жыл бұрын
As with so many fields related to software, when it seems like you are 90% of the way there you have probably done 10% of the work. It's turning out that way with automated driving, for example, and I suspect it will turn out that way with AI-driven coding too.
@sharkysharkerson Жыл бұрын
At the moment, ChatGPT reminds me of working with an intern. It seems overly confident and tries to please, but doesn't always know what it's talking about. And you need a certain amount of experience to assess whether what it's saying is actually valid. This is the same whether it's writing a report or writing code or anything. That said, its massively impressive.
@GnomeEU Жыл бұрын
Try ChatGPT with a script with 100+ Lines of Code. It will repeat itself way too often, and if you want to make specific changes to the code you have to dictate a book. Then it forgets what the purpose of the code was and just moves lines of code around. Where you arrive at the point where the code no longer produces the right output... Maybe ChatGPT would need TDD to produce something useful. But its great if you have a small code snippet or just ask it to explain stuff, It does that really well. It's also great when you're stuck at a problem and need some fresh Ideas. But yeah, i recently pasted a big class and asked it to improve it. It suggested to rewrite it in assembly so its faster, LOL
@Liam3851 Жыл бұрын
Chat GPT now returns your bio to me, if I ask about Dave Farley of the Continuous Delivery channel. The response is almost word-for-word what you gave it, including the accurate book title and co-author. So it does seem to have some kind of online transfer learning between users in fairly real-time (I'm writing this 20 hours after the video post).
@Kobay350 Жыл бұрын
This is similar to a lot of different issues in science and technology. You get the flashy demo and there are a few small things that need to be worked out. Later it turns out the small things are actually really hard and solving them takes so much longer. Self driving cars comes to mind when talking about this. Those last things are really hard.
@AloisMahdal Жыл бұрын
Yeah, because it turns out they're not actually small at all, it's just that we have the wrong optics in the beginning, and these things are just "small proofs" of massive misalignment.
@timop6340 Жыл бұрын
You need to market your research with huge pr drums if you want to have glory and funding in the future. And topics are picked in a way that you can get such headlines. This probably hinders the science itself in the long run. Breakthrough inventions come easier if you can do research that is initially falsely evaluated as useless.
@timop6340 Жыл бұрын
And lot of r&d projects are stopped because it is hard to get things working on different scale and/or in real life outside the lab. Or the beancounters just want to exceed specific profit margin for the final product.
@Kobay350 Жыл бұрын
@@edsmith7248 I love this phrasing.
@Guytron95 Жыл бұрын
This was excellent, Dave. There are a few caveats I'd like to point out. If you give the errors back to ChatGPT it often, although not always, will correct the code very well. Also, there is a process known as in-context learning which happens while it is processing that actually happens in the forward pass of the neural net. It isn't just keeping track of the previous examples, it is actively changing the output depending on learning produced in-context as it generates a new result. This was an astounding and recent discovery that proves the forward pass is implementing gradient descent learning in a sort of meta fashion while it generates output. Finally a personal note you already touched on: it is pretty darn good at writing out long, repetitive batches of code which I would otherwise have to type manually. I agree with you, I would never use this in production or mission critical code. But it is very helpful in my own purely personal code experimentation. Thanks for the continuing wisdom you impart!
@MrArrmageddon Жыл бұрын
Your right it's response sometimes changes too. Question How good are you at writing code? As an AI language model, I am capable of generating and understanding various types of code in a variety of programming languages, but my primary function is to assist users in generating natural language text rather than writing code. While I can help with programming-related questions and provide example code, my proficiency in writing code is limited to what I have been trained on and my ability to understand and generate code is based on patterns in the training data. That being said, I am constantly learning and improving my abilities, so feel free to ask me any coding-related questions you may have!
@Immudzen Жыл бұрын
Thank you for making this video. I am going to share it with others because I have been trying to make the same point. Chatgpt and most of these other AI systems have no concept of truth and it is not something that is easy to add. They can be useful and they can assist in programming but you have to watch them very carefully because they will be wrong. I have done some tests with generating Python code and it does a decent job of generating code to the point where it can do a lot of the basic work and you can focus on the overall solution.
@MathCoreMan Жыл бұрын
My problem with chatgpt is that I find the same answers on the internet, and from blogs and forums I often learn much more.
@williamsbotchway2471 Жыл бұрын
I enjoy every one of your videos. Thank u.
@ReedoTV Жыл бұрын
"Cut to a shot of Dave running tests on a computer" God, that would have been exciting!
@stephendgreen1502 Жыл бұрын
I asked Chat GPT: How does Chat GPT answer questions? Does it reason or does it merely immitate human responses to similar questions? It answered: “As a language model, I rely on statistical patterns and machine learning algorithms to generate responses to questions. I do not have the capacity for conscious reasoning or understanding in the same way that humans do. My training data comes from a large corpus of text, which includes a diverse range of written material such as books, articles, and websites. This training allows me to recognize patterns in language and generate responses based on those patterns. When answering questions, I analyze the input question to determine its meaning and context, and then generate a response based on the statistical likelihood of certain words and phrases occurring in that context. While my responses may appear to imitate human responses, they are generated based on patterns in the training data rather than a conscious understanding of the content.” So if code used ten years from now to train successors to GPT is all generated by AI… even AI knows it will be a bad thing.
@AmericanDiscord Жыл бұрын
I think if you look at the examples of GPT with an open API to Wolfram Alpha, you can imagine it being plugged into a test bench framework for code development. With the ability to off-load certain types of tasks or question to Wolfram, GPT is capable of solving the simple math questions that trick it up in it's pure form.
@alxjones Жыл бұрын
The moral of the story is that you don't trust code, you trust tests. Code either passes the tests or it doesn't, and if you trust the tests then you trust those results over anything else about the code. Doesn't matter if the code is written by a junior, a senior, or an AI.
@j2csharp Жыл бұрын
Just wanted to add that I find ChatGPT helpful in brainstorming. I can ask it to assess my solution design, then ask it to offer a possible implementation or 2, or 3, then learn which road to take to implementing a solution. And then of course ask it to write some unit tests for the code, as a starting point. (Oh, wait, maybe I could ask it to express the design of the code in terms of unit tests prior to the implementation. ) 🙂
@YDysgwrAraf Жыл бұрын
I'm a bit late to the party on this, but you are spot on here. Excellent video.
@AleksandarIvanov69 Жыл бұрын
Depends on how complex you want the code to be. I needed a simple Angular 12 web api request webpage with a button for a demo the other day and I have no idea about Angular, so Chat Gpterson wrote a nice little component that worked perfectly fine. BUT still I had to read and troubleshoot a bit to make the pieces work together. Any more complex than that, I would guess be very cautious.
@GnomeEU Жыл бұрын
If you ask it about a not so well known library then it just makes up properties and functions that don't exist.
@AloisMahdal Жыл бұрын
@@GnomeEU Ha. Lot of the code that I've seen from GPT looked like snippets from coding beginner tutorials. I bet in its "head" it just takes dozen of tutorials and smashes them together into a mud ball and hands it over to you. With extremely common frameworks, it might almost work but for purely statistical reasons.
@macmcleod1188 Жыл бұрын
@Alois Mahdal it specifically wrote bad Minecraft fabric code using Minecraft forge methods, even making up constants that didn't exist. Turns out fabric, handled mod event prioritization completely differently. Otoh, it wrote an entire bug free mod with configuration code, network messages, client side code, etc. In about 12 minutes of prompting.
@AloisMahdal Жыл бұрын
@@macmcleod1188 Yeah, that's the problem. If you went to a bus stop and asked random people to write Minecraft mods you would have certain chance to get a "correct" mod, but the thing is that most of people would politely refuse. AI is like replacing all these polite people with psychopaths who want to waste your time.
@macmcleod1188 Жыл бұрын
@Alois Mahdal I can see that with a one more iteration, it's going to be better v than CASE tools And my 11 year old grand son uses those to write minecraft mods. In the end, I have the unique ideas and the drive to maintain a Minecraft changes versions. And that's given 22 million downloads. But it is already very powerful and the next chat gpt4 is going to be much more powerful.
@DryBones111 Жыл бұрын
At 13:11 ChatGPT suggests how to test the code but provides an incorrect test case. It inputs the word "jumped" and expects to sort "jumps". I presume this is how the test failed. Based on the screenshot, you expected "jumps" to be first because initially, it was sorted first based on the longer form "jumped". Your explanation still stands, I just thought it was interesting to see the reason why this happened and any actually smart AI wouldn't suggest a test with "jumped" and "jumps".
@fjdkfjdk Жыл бұрын
I've been using it to make test cases. Pretty good tbh, since they're naturally small isolated blocks. You just need to give it some well written ones first to show it the style/format needed.
@jacktilley7299 Жыл бұрын
I pasted a user story in and the test cases it produced were better worded than I ever would have
@AloisMahdal Жыл бұрын
Wait, but how do you go about checking whether test code is good or not? I've worked in automated testing for 10 years now and my conclusion so far is, when it comes to test code quality, basically all we have is trust in author's competency: What is their method? What is their strategy? Do they have some education on the matter? Are they experienced? Are they honest? For any kind of AI, the answer to all of these is resounding no.
@jacktilley7299 Жыл бұрын
@@AloisMahdal Ive only played around with it but if someone needs written manual testcases it's not too bad
@fjdkfjdk Жыл бұрын
@Alois Mahdal You can't read a unit test and understand it, yet have been working in the field 10 years? Just how complicated are you making your test cases? The person prompting chatgpt lays out exactly what cornercase needs tested, so there's nothing strategic left up to chatgpt.
@AloisMahdal Жыл бұрын
@@fjdkfjdk I did not say anything like that, those are your words. Of course I can but understanding a single test case tells me nothing about the whole coverage. If you can get a decent coverage by specifying each case to ChatGPT, then you can probably already do it with some BDD framework like Cucumber, right? The hard work is getting a good coverage, ie. having enough test cases and not skipping areas or getting over-focused on some parts. That's where you need the qualities I listed the most. And because the alignment problem, I highly doubt we can ever get AI that could be trusted.
@bennypr0fane Жыл бұрын
I asked ChatGPT to take an array of strings and divide it randomly into four sets. It immediately came up with some code and was convinced that this would do the trick, making false statements. It took four corrections to come up with a solution that worked, at each step saying it had the correct solution. So this is how I found out it has no idea if it's right or wrong - and also, correcting it, it will "thank you for bringing that to its attention", but not remember anything, so correcting it makes no sense at all.
@insertoyouroemail Жыл бұрын
This technology might make perfect sense to check if your doc-strings are reasonable. Add it to the stack of test suite and type definitions. Could make us finally get our documentation in order. It might end up encouraging clear and straight forward code to make the doc-string easier to compare to your implementation.
@tibfulv Жыл бұрын
I suspect the procrastination responsible for the dearth of documentation may be a food allergy causing fatigue. It was first spotted by Dr Atkins because he recommended avoiding grains, but it can encompass any foods. And you can establish it by elimination over three to six months or so. I'd recommend establishing if grains are your allergy first, then building on that by adding small amounts of test samples for the foods you want to add. Look at biological families, which might tell you what to avoid.
@bonchbonch Жыл бұрын
I asked ChatGPT to write some iOS code to solve various problems. It often called APIs that looked like legitimate APIs with typical Apple naming but were actually non-existent.
@bryanfinster7978 Жыл бұрын
I've done some testing with this. It's OK for boilerplate or for creating a client for an API you don't know well but will need some tweaks. However, some requests yielded results worse than I'd expect from someone learning to program.
@mikkolukas Жыл бұрын
I understand your concerns, and I'd like to help you get better results from our interactions. First, it's important to note that I'm designed to be a generative language model, so the results you get will depend on the quality and specificity of the input you provide. To get the most out of me, try providing more context and details about what you're looking for. This will help me understand the context and goals of your request, and give you a more accurate and relevant answer. You can also consider using specific prompts or templates to guide me in the right direction. It's also important to keep in mind that I'm not a human, and I can make mistakes. So, if you do come across an answer that seems incorrect or not up to your standards, feel free to ask follow-up questions or provide additional information to help me better understand what you're looking for. I'm here to help, so let's work together to get the best results! -- ChatGPT
@bryanfinster7978 Жыл бұрын
@@mikkolukas 🤣
@SchwaAlien Жыл бұрын
I’ve run into problems trying to program a specific ESP32 development board with certain functionality that should be possible to use, but due to my unfamiliarity with the modules it uses, I cannot get it working like it was on a simple Arduino... it would be great if you could direct Chat GPT to new data to base it’s results on if it is not ‘familiar’ with something specific or unusual, that would be a useful addition.
@jimmiller9330 Жыл бұрын
From the time you issued the request to the time it generated the script, how many hours or days went by? My suspicion is that a human or humans inside the “box” scurried around training chatgpt on TDD.
@heikolange4348 Жыл бұрын
I did a few tests asking ChatGPT to create some scripts in Swift. None of the scripts were wrong per se, all did what I asked for (and if I was missing something, it was because I didn't specifically asked for it in the first place). However, in some of those script there were some unnecessary steps or they looked pretty much like a translation from C-like language (e.g. the need to provide the size of an array) - maybe Swift is too niche for ChatGPT. It was interesting nevertheless. Curious on how it will improve over time.
@maherj351 Жыл бұрын
What is your opinion on building modular reactive monoliths? is such a thing possible/recommended?
@ContinuousDelivery Жыл бұрын
Yes, perfectly possible, in fact I’d say it is easier than building a modular reactive system as a collection of microservices. Though the microservices approach is more organisationally scalable.
@cfalguiere Жыл бұрын
There IA specializing in code like Codex or Copilot. They aim at generating working code though it is still a work in progress. ChatGPT just mimics code. Dedicated unit test generators like randoop, evosuite or pynguin also fail at generating working tests sometimes. The excess of confidence of ChatGPT is actually an issue, especially when accuracy is required. It is really bad at math and solving logic problem. As for concept it is usually not specific but quite correct. Regarding your example, it does not seem to get the difference between tdd and test first, and unit test to some extent until it is asked to compare those concepts.
@FURIAfdx Жыл бұрын
FINALLY SOMEONE THATS NOT A MORRON, its impressive that ppl make video using copy paste code while chatgpt says at the end that code is and example
@ModeratelyAmused Жыл бұрын
One thing to understand is that the data is only through 2020. It says "up to 2021" but I tested by asking about major events in different months and it seems to end before January 1st, 2021. There may be gaps and has some data after that date but not as comprehensive as before that date. Seeing as the two books you have listed on the channel description were published in 2021, it makes sense that your content is not in it's current data set.
@wongyok1 Жыл бұрын
Asked ChatGPT to write a peom Test driven development, a concept so bright, Bringing clarity to coding day and night. With every line of code, a test to write, Ensuring the code will work just right. Before we push to production, with pride, We write tests that validate and guide. From edge cases to happy path with care, We test it all, to be beyond compare. And when a bug creeps in, no need to fear, The tests will catch it, and make it clear. With confidence, the fix can be applied, Thanks to test driven development, verified. So let us embrace this methodology, With focus, discipline, and clarity. For better code and peace of mind, Test driven development, we shall find.
@pugix Жыл бұрын
Check out Peter Naur, Programming As Theory Building. In a nutshell, programming is not the creation of texts, but is the acquisition of knowledge, i.e. learning. I've always thought of software development as a mutual or collective learning exercise between the business user and the programmer about what the software product should do.
@MrMate12345 Жыл бұрын
When AI will write all the code, we will call it compiler and we will have a well-defined (large) subset of natural language to use it as "description". Of course versioning and testing will be as important as today. :)
@kjdtm Жыл бұрын
and your program will actually be a well defined requirement... so everybody needs to jump the fence and learn requirements engineering. Slowly but surely we are getting there... IDIOCRACY is near...
@trucid2 Жыл бұрын
So programming is moving more and more towards being declarative, written in natural English... You just have to know what you want and be able to explain it well enough to the AI.
@solenoden Жыл бұрын
ChatGPT actually knows who you are, just need to be more specific: Q: Who is Dave Farley in the Software Development industry? A: Dave Farley is a well-known software developer, author, and consultant in the software development industry. He is best known as the co-author of the book "Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation". He is also recognized for his expertise in software development practices, including continuous delivery, DevOps, and software architecture. He frequently speaks at software development conferences and events, and provides consultancy services to organizations looking to improve their software delivery processes.
@MrMagooRC Жыл бұрын
It’s also worth considering that the output of an AI can’t be subject to copyright based on current legal understandings - only code generated by a human can. The legal ramifications of widespread use of AI generated code is way up in the air at the moment, a risk a lot of companies haven’t yet considered...
@unfa00 Жыл бұрын
Github Copilot violates the licensing of GPL code by recycling it (sometimes even printing out the GPL license itself in comments) without the consent or knowledge of authors Unfortunately without a solid class-action lawsuit this won't be settled.
@neoqwerty Жыл бұрын
@@unfa00 Aren't a group of programmers on Github suing Microsoft for getting their code scraped and stolen?
@GearAddict90210 Жыл бұрын
Great video, to thanks!
@georgehelyar Жыл бұрын
So it can write tests that sometimes pass if you just keep running them? That puts it on par with a lot of developers.
@neoqwerty Жыл бұрын
Also puts it on par with fish, and roughly 1000 humans who have simultaneous control over your mouse and keyboard.
@soainpractice Жыл бұрын
Is pair programming with ChatGPT a possible approach to integrate ChatGPT into software development?
@michaeleldredge4279 Жыл бұрын
At the risk of overanalyzing something silly, I don't see that much in the Dr. Suess portion that is really reminiscent of his style. I noticed three points: 1. The output of Chat GPT appears to be a song in a verse-chorus structure. Dr. Suess is most famous for his books, not songs. Yes, many adaptations of his work are musicals, but the songs from those that I know for sure that he had input on (How the Grinch Stole Christmas, The Lorax) I don't remember having a verse-chorus structure. 2. Dr. Suess's is poetry and songs had a very strong rhythm. If you look at Green Eggs and Ham you can usually tell which character is speaking depending on if the line is written in iambic tetrameter or trochaic tetrameter. I can't even tell what the meter is supposed to be in the Chat GPT. That is not unusual for modern popular songwriting, but it lacks the rhythm of Dr. Suess. 3. There is no real silliness to the song. Dr. Suess would make up words, use ridiculous comparisons and absurd situations in just about everything he did. The Chat GPT output is mostly straightforward with a few standard metaphors that don't really create an atmosphere of whimsy. All this to say that I don't really think the machine is any good at imitating the style given in the prompt. I think it just wrote a script in the style of a popular song and tried to pass it off as the style of Dr. Suess.
@CloudCuckoo00 Жыл бұрын
A response from Chat GPT itself.. " As an artificial intelligence language model, I do not have a concept of right or wrong in the same way that humans do. However, I strive to provide accurate and helpful responses based on the information provided to me. While I may generate incorrect or incomplete responses at times, I am constantly learning and refining my understanding of language and context to improve the accuracy of my responses over time. It is important for humans to understand that my responses are generated based on patterns and information learned from a large corpus of text data, and are not based on personal experiences or emotions. While I can provide helpful insights and information on a wide range of topics, I should not be relied upon as the sole source of truth or as a replacement for critical thinking and independent research"
@YasirQaiser Жыл бұрын
So when I tried: this is what I got: Who is Dave Farley? Without more context, it is unclear who Dave Farley specifically refers to. There are many people named Dave Farley, and without more information, it is not possible to determine which Dave Farley you are asking about. he has youtube channel and talks about continuous integration and delivery Ah, I see! Dave Farley is a software development consultant, author, and speaker who specializes in continuous integration and delivery (CI/CD) practices. He is known for his expertise in this field, and shares his knowledge and experience through his KZfaq channel, speaking engagements, and writing.
@YisraelDovL Жыл бұрын
'Where is the link to the TDD training course?
@didierseverac Жыл бұрын
I'm wondering how feasible it would be for it to try his code, realizing that there is a mistake try something else until it find something that works, before giving you an answer
@diztens4678 Жыл бұрын
As a Java programmer I have been regularly implementing it in my work recently and I have noticed a considerable jump in productivity. Especially when asking it to write me methods that do specific things
@macmcleod1188 Жыл бұрын
Yes, for construction coding, it's about 9 times faster. And for new areas of expertise, it's crazy fast doing in 10 minutes what would take me 4 hours+.
@yan-amar Жыл бұрын
Now you can produce bugs even faster, good job. At least use a model that was made with coding in mind. You're basically saying that you fell for the illusion that Chat-GPT provides. The thing isn't even able to correctly describe a valid C language identifier, but people use it in "new areas of expertise and research". Makes me excited for future software.
@macmcleod1188 Жыл бұрын
@yanamar son, I've been programming in multiple assembly languages since 1976. 6502, sweet 16, 8088, then vax780 macro in 1980. You probably "weren't born yet" when I was programming D&D utilities in Fortran and COBOL. The fact that you don't know how to use chat GPT to accelerate your workflow says more about you. And GPT4 is just around the corner. My and my buddy have closing on 23 million downloads. I used Chat GPT a half-dozen times today. And it doesn't even know Minecraft past 1.16.5 yet. That game has between 160,000 at 600,000 line to code. Chat GPT can and has saved me hours of research digging through that codebase. It's already let me write a mod that would have taken 16 to 24 hours in about 4 hours. That's money. I've also used it to compose poetry to my girlfriend, right condolences to a friend whose loved one lost their mother, to explain derivatives to my grandsons, and to write a three-act script set in the 1970s complete with emotionally appropriate song references and television show references, and to hold discuss Maine Coons in French. And to correct my grammar errors. I respect the fact that when it comes to things that doesn't know, that it makes stuff up. I treat it with appropriate caution, but it is a powerful tool that can flesh out my thoughts and ideas faster than I can.
@AnonGuardians Жыл бұрын
@@yan-amar We tasked ChatGPT with developing 100 separate C++ classes such as AdapterPattern, GridModel, RadixTree, Array, Hash, Semaphore, BarChartModel, HashTable, Set, BasicTimer, Heap, and more. Out of these 100 classes, only two had unintentional bugs, ten had bugs because ChatGPT used a specific type instead of a template, and only three had memory errors or leaks. Additionally, despite multiple attempts to fix them, only five classes (Btree, Heapsort, ChainOfResponsibilityPattern, MemoryMapBuffer, Deque) did not work correctly. To verify the results, we had a team of 25 computer science graduates with master's degrees test everything with gdb and valgrind. We also sent these classes to a professional software security testing company for further evaluation.
@pbjorklund Жыл бұрын
Love to see an update for GPT4
@timseguine2 Жыл бұрын
Since chat GPT is based on a reward model, it would be possible to do online learning inside of a user's chat context(with something like a "good bot/bad bot" button), but that would be both more expensive for the online hosting(to the point that it would be unfeasible in a cloud environment), and also provide an additional potential way for users to evade content filters
@trucid2 Жыл бұрын
I tried to teach it to play chess by teaching it the rules and had no success with it. It just didn't get it even after repeatedly correcting its invalid moves and explaining why they were invalid.
@timseguine2 Жыл бұрын
@@trucid2 The current version of chatgpt is trained offline only. Telling it it made a mistake doesn't have any effect in terms of the reward model. My comment was that online learning is technically possible with the way the model is constructed mathematically. It isn't implemented though. Chat gpt can't learn through the current API, and it only has a very limited ability to "remember" things.
@krzysztof2949 Жыл бұрын
This is the answer I got from ChatGPT on question Who is Dave Farley of Continuous Delivery ? Dave Farley is a software development expert and author, best known for his work on the principles of Continuous Delivery. He is co-author of the book "Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation." Farley is also a frequent speaker at industry conferences and events, sharing his expertise on topics related to software development, DevOps, and continuous delivery practices. Did it incorporate your answers on the fly?
@ContinuousDelivery Жыл бұрын
No, I don't think so. I think that the qualification of "of Continuous Delivery" gave it a hint 😉
@sdwone Жыл бұрын
I love it when I hear that lone voice of Reason above all the Hype and the Madness! Way too many of us have Hive Mind Thinking which is Dangerous! I've yet to use this tool myself, because I've got far too much going on at the moment, and the more hyped something is, the less chance I'm going to get involved with it! Let lemmings be lemmings! And I'll do my OWN thing thanks!
@michaelnurse9089 Жыл бұрын
It knows who you are for everyone "Dave Farley is a software developer and author who is known for his work in the field of continuous delivery and DevOps. He is the co-author of the book "Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation."
@snuggie1234 Жыл бұрын
Really liked this video, but admittedly I had it pretty far down on my queue because I thought this wasn't going to be very balanced. This was due to the thumbnail and caps in the title. I'm not telling you need to change or anything but given that I liked it I figured I'd share that feedback.
@MichaelBiskach Жыл бұрын
I wonder how far something like AutoGPT, running the unit tests and reading the output in an IDE, can go when instructed to perform TDD to achieve a task. This approximates what Dave mentions in this video but I'm sure would still be limited when compared to human software engineers. Some form of human oversight or QA seems necessary, at least in the short term.
@nivoset Жыл бұрын
I know i wanted the copilot app to write code to pass my tests. I think thats where it needs to go. And we work on making better tests it can iterate on. But long way off still i think
@hotworlds Жыл бұрын
Imagine the number of security vulnerabilities that will start cropping up if people start relying on language models to write large scale applications without involvement from experienced developers
@NihongoWakannai Жыл бұрын
Anyone who allows this to be used with no supervision in a security-sensitive application should be immediately fired.
@BxPanda7 Жыл бұрын
I once argued with chat gpt for a hour before giving up, it was hilariously wrong and didn't want to admit it, so I don't trust anything it says. It's also a censored AI that has been made incapable of being fun, all while being trained on blatant lies and propaganda. Thank god there's already an open source project working on replicating it by training it on all the possible data without the censorship, and something tells me that the alternative will be extremely based 😂
@niallmurphy2163 Жыл бұрын
Copilot is so much better for code, yet hasn't made a dent and isn't even talked about. I asked ChatGPT for a simple function that returned the number of sentences in a string and it simply split the string by full stops. So this... < would be three sentences. It's pretty terrible. To an untrained eye, the code worked, but I immediately saw the flaw. There is no way non-programmers are going to making applications with these language models.
@KellsCode Жыл бұрын
I treat it as if it's just an overconfident know-it-all person. So long as we are aware of its shortcomings, we can better utilize it.
@ahmetmutlu348 Жыл бұрын
i think is ower thinked update of gcc . gcc is actually doing the some thing. gcc translates high level languages like c cpp to lover leve/another language assembly then to machine code which itself is buch of local languages :P so what gcc does is wery stable language translations and intelligiend updates to logic errors on language typos. chat gpt adds it a step highler level language . which is big improvemens but even tough i think it wasnt need that long roads to walk to write a translator and write that much code as its written allmost automated atleast the code used to generate is written but that code wrote chat gpt (as what i understood chat gpt is generated by trained data which generation gode used to generate its current wirtual code state)
@DanielPaunescu Жыл бұрын
The Skynet T-shirt is brilliant for this episode.
@HansLemurson Жыл бұрын
"Truth" turns out to be a far more complicated topic than we once thought. I remember reading old Science Fiction books where the computers could always arrive at true and correct answers just through the power of their mathematical computation. Math is based on Truth, so math calculations will always arrive at Truth...right? The problem is that it is easy to define falsehood: Something that contains a contradiction. But a lack of contradictions doesn't make something true.
@edwardallenthree Жыл бұрын
ChatGPT is a great tool for analysing how people talk, how people write, and how people code. The fact that it is so easy to turn into a bigot is a feature: it sees this in us. It is great for demonstrating the limits of human knowledge, worthless for generating new knowledge.
@cat-le1hf Жыл бұрын
I think the fact that it has to be turned into a bigot and is not one inherently shows that it's a human failing, not inevitable or natural.
@louis-philippesavard742 Жыл бұрын
Love the shirt!
@przemekkobel4874 Жыл бұрын
I think its answers are also optimized to use less computing power. At least initial ones.
@ahmetmutlu348 Жыл бұрын
i agree with you there. chat gpt is like gcc .. so we need some ai that can control chat gept and use it as referance like automake and make uses gcc for generating more automated code :D
@obvioustruth Жыл бұрын
I asked 5min ago (on 2023-02-15 @20:44 CEST) "What is the largest country in central america except for Mexico". It answered: "The largest country in Central America, excluding Mexico, is Nicaragua. Nicaragua covers an area of approximately 130,373 square kilometers (50,337 square miles) and is the largest country in terms of land area in Central America. It is bordered by Honduras to the north and Costa Rica to the south, and it has coasts on both the Pacific Ocean and the Caribbean Sea." I didn't check validity.
@adb012 Жыл бұрын
That Skynet t-shirt is very relevant!
@donovancarthew491 Жыл бұрын
Nail on the head! Well said.
@richbuilds_com Жыл бұрын
That's odd, because that means chat GTP "lied" to me when I specifically asked it "what have you learned about me through our interactions here?" It replied: As a language model AI, I do not have the ability to store information about our interactions
@elliejohnson2786 Жыл бұрын
Is the Dr. Zeus/Dr. Seuss typo intentional, or part of the AI's response? Or is Dr. Zeus an entirely different person I'm completely unaware of?
@Spinikar Жыл бұрын
ChatGPT is my new rubber duck. I talk to it to generate ideas, thoughts, and sometimes just a new angle it comes up with. Though you need to be careful or you can end up with a memory leak, like I did. :D
@jangohemmes352 Жыл бұрын
Haha, the outro grin did me in
@MarcLucksch Жыл бұрын
I tried this as well a few weeks ago, and you can even ask chatgpt how to run the test, and it gave me the output of what all tests successfully running would like, even though they weren't. But then I told it the test failed with the following output and it went and fixed the code.
@ContinuousDelivery Жыл бұрын
Thanks for the info!
@splendorman7922 Жыл бұрын
I wanted to try asking the same calculus. I was welcomed by a message saying "We’ve upgraded the ChatGPT model with improved factuality and mathematical capabilities." I said oh ok so they fixed it. I asked the same calculus and the result it gave me was 13,169. I asked if it was sure the answer was 13,169. It waited about 2 minutes for some reason and said 'yeah I'm sure the result of 357 multiplied by 37 is indeed 13,169' It is very interesting actually why it is giving us the same wrong answer? Anyway I gave the correct answer and it thanked me for correcting it. The next time I asked the same thing, the result was correct. It is even more weird that it accepted it was wrong and thanked me for correcting it. I asked why it was bad at math, it gave me this answer: "As a language model developed by OpenAI, I have been trained on a large corpus of text, which includes mathematical information and equations. While I can perform mathematical operations and provide accurate answers, I may make mistakes or misinterpret questions at times. I apologize if I gave you an incorrect answer. Thank you for bringing this to my attention."
@asmundurg Жыл бұрын
I asked ChatGPT who Dave Farley is, and it did know for me, and I don't think I have asked it about you before. Would be interested in hearing about other people experimenting with this.
@DaSkuggo Жыл бұрын
I did the same and got the same description as Dave got a couple of days later. So apparently chatGPT learns from conversations
@michaelnurse9089 Жыл бұрын
I often get the "I don't know that stuff " answer from ChatGPT so the Dunning Kruger criticism is not a fundamental limitation - merely a problem with not enough or inaccurate training data - meaning it will improve over time.
@pm71241 Жыл бұрын
I'm stopping here at 3:10 and putting my money on that this is Dave tricking us and it's actually his words.
@BowBeforeTheAlgorithm Жыл бұрын
I asked ChatGPT to solve a hackerrank C++ question. I was able to take its solution paste it in and all the test cases passed. Next, I asked ChatGPT compare the performance differences between runtime polymorphism and static polymorphism (I think that's called late binding versus early binding in Java?). It got the answer COMPLETELY wrong. So yeah, I agree it's probably only good for very specific examples like basic api examples, generic design patterns, or where the specific answer is very widely/publicly known...and even then only as a reference.
@vanguard1427 Жыл бұрын
loving the Tee
@kottalovag Жыл бұрын
Well, I have asked it who is Dave Farley and it gave somewhat similar answer: Dave Farley is a software developer, author, and consultant who is widely recognized as a thought leader in the field of software engineering. He is the co-author of the book "Continuous Delivery" along with Jez Humble, which is considered a seminal work in the field of software engineering. Farley has over 30 years of experience in software development and has worked in various capacities, including as a developer, architect, and technical director. He has worked with a wide range of companies, from small startups to large corporations, and has helped them to implement best practices in software development. Farley is also a well-known speaker and has presented at numerous conferences and events around the world. He is an advocate of agile methodologies and continuous delivery practices and has been instrumental in promoting these practices within the software development community.
@ContinuousDelivery Жыл бұрын
As a matter of interest, did you ask it "who is Dave Farley" or something like "who is Dave Farley of Continuous Delivery"? When I tried it, understandably, the context seemed to matter quite a lot.
@kottalovag Жыл бұрын
@@ContinuousDelivery I copied the same question: "Who is Dave Farley?" 🙂
@ContinuousDelivery Жыл бұрын
@@kottalovag Interesting, I wonder if there was something in your ChatGPT history that told it that I am the "Dave Farley" that might be interesting, or if they have updated the model since I last looked? Thanks for the info. Actually, I am pretty happy with that description of me 😉😎
@kottalovag Жыл бұрын
@@ContinuousDelivery I should have been more transparent about the fact that my account is fairly new and I engaged with your videos only in the past two weeks, not knowing your name until today. 😃 So there was nothing in my particular chatgpt history about you or the channel.
@ContinuousDelivery Жыл бұрын
@@kottalovag Cool! A clean-room experiment then 👍 Thanks.
@therealist2000 Жыл бұрын
I only ever use ChatGPT for a much simpler explanation of code, I've never been very good at understanding technical things so understanding it like it's being explained to a toddler is much better for me.
@mikkolukas Жыл бұрын
Ah, I see! You've figured out one of the best ways to use me. As a language model, I can certainly help break down complex technical concepts into simpler terms for better understanding. In fact, you can even give me a specific level of explanation, like "explain it like you're explaining to a toddler". This will help me tailor my response to your needs. Another way to make the most out of our interactions is to be as specific as possible with your questions. Providing more context and details about what you're trying to understand can help me give a more accurate and relevant answer. And don't be afraid to ask follow-up questions or clarify any parts that are still unclear to you. That's what I'm here for! --- ChatGPT
@neoqwerty Жыл бұрын
@@mikkolukas Holy crap ChatGPT can ELI5 things.
@NerdSnipingBatman Жыл бұрын
I would say it does learn a little bit but only for your user. I think openAI is cautious to let anyone on the internet teach it wrong things. I don't think that it is just simple predictive text. There's stuff that it gets really right more often than wrong. Just don't trust it's output blindly and you'll be fine.
@delprofundo Жыл бұрын
It is interesting, its writing on TDD reminds me of what I describe as the mediumification of online technical content. I presume it is because such a huge percentage of the pages are produced by students that most of the content is 101 descriptions of the same introduction to some technology over and over. Captain ready mc-read-read over here has had the same first day on the job of every technology 1million times over. ergo it probably seems convincing to people who are reading what it has to say about domains they dont understand but if you do know what its talking about it'll be clear that its not really that inspiring.
@StefanReich Жыл бұрын
It helps to remember that ChatGPT has NO experience of the world around us. All it has seen is text. It has no ability to actually use a compiler to run code either. So it has no practical experience with anything. It just strings words together. Seen against that backdrop, it's incredible that what ChatGPT says is coherent at all.
@jonblackburn7634 Жыл бұрын
I'm still to be convinced that ChatGPT isn't just a warehouse full of slaves, typing manically on keyboards for fear of being shot.
@Code-Seven Жыл бұрын
I just did a double take - I'm wearing the exact same t-shirt 😄
@carlphilippgaebler5704 Жыл бұрын
My go-to answer for ChatGPT is going to be this chess game: kzfaq.info/get/bejne/qLlzgbpitKnPj4E.html At about 1:35, ChatGPT plays its first highly illegal move, which it continues to do for the rest of the game. Because ChatGPT does not know what legal moves are, or what the board is. It just knows that "often, right after White castles, Black will also castle." Right through its own bishop, in this case. So then, the argument is - why do you expect its answers to be any better for anything else?
@javieralvarez6579 Жыл бұрын
These beautiful shirts!
超人夫妇
Рет қаралды 46 МЛН
Buy Smart┃Магазин ноутбуков
Рет қаралды 783 М.