If you're commenting that you need to prompt ChatGPT to write secure code, and it doesn't do it by default, you've entirely missed the point 😁

in conclusion: if AI takes programmers' jobs, they can at least still make it big in malware development

The first one was not a memory corruption error. It correctly limits the buffer write with the length parameter, not accidentally. The fact that it's bad code, and easy to implement security issues in the future, does not make it a security issue now. It does have a path traversal vulnerability, though.

The first code is not vulnerable to buffer overflow (simply using sscanf does not make your code vulnerable). The read function reads into a set buffer only a set number of characters so it protects the call to sscanf.

While working with ChatGPT and code reviews, then I get several times : “I apologize for the confusion caused by my previous incorrect statement. Thank you for pointing it out, and I apologize for any inconvenience caused.”

8:20- Idk if this is common knowledge or not, but- you can tell chatGPT to continue writing code where it left off when it cuts off before finishing.

For the first example, I would consider another exploit. The user can control the filename and the path and at the same time you run it as superuser. This could lead to file leaks when in production.

Back in the late 1980s, people were talking about how code generators (I think they were called 4GL languages, or something like that) were going to replace programmers. Over 30 years later, I'm still banging out code on a keyboard.

I thought we were all collectively not going to talk about that, for job security

The model got 'lucky'? I think your bias might be leaking a bit. I asked GPT-4 using the same prompt and when I ran it the AI pointed out the code wasn't production ready. Then I asked it to include comments and evaluate the security of the code it wrote and it points out the same potential overflow you did as well as 6 other vulnerabilities including potential directory traversal attacks etc. So did it get lucky or did it just provide a simple, non-production ready example as requested?

You make a solid point here. I have for some time held the opinion that using an AI to write code is dangerous in that my assumption is that the AI is trained on public code, as you mentioned. For anyone with solid programming experience we all know not to trust public sources. Even open source, which sometimes is held up as a good way to make code better because many people look at it, is very often filled with very good examples of how not to do things. I teach graduate level class and I decided to try to get ChatGPT to generate a very solution to a very simple assignment. Eventually I got it to generate what I asked for. But as with most students, it didn't pay attention to what I told it to do. Which required quite a few iterations. I was impressed that it came up with some solutions that I had not been aware of. In the end I think that the ability to have an AI generate code is potentially a useful tool. However, as you pointed out it is often not going to give you a great answer. I also asked ChatGPT and Bard to generate a C++ 11 thread pool. They both gave a good answer. But the answers were so similar that it seemed like they were using the same source. I think this technology is worth using, but like any other tool, you need to understand limitations. Just like a nail gun and a hammer can both do some of the same things, there are cases where each is a better or worse choice. Think of it as a tool. Maybe a good way to find the start to solving a problem, but not yet a tool for blindly using it to solve problems. As a follow up. Take the code that was generated and ask it to review for potential buffer overrun vulnerabilities and see how it does.

use 'Continue" to make GPT to continue a previous long post. Otherwise it defaults to the ending when the standard output token is reached.

You should post the generated code somewhere and add a link to it in the description. I have a feeling that there's more vulnerabilities in the first example than just a possible buffer exploit, such as not flushing the file buffers possibly causing an issue with subsequent reads, and the most obvious issue which you hinted at but didn't expand on about file permissions. Running as root would give remote access to all the files on the system.

4:10 I thought you were going to talk about the path transversal vulnerability after that. It's not as terrible as a buffer overflow, but it is still pretty bad IMHO.

Another vulnerability with the first code which can actually be exploited: you can put a .. in the filename and exit the scope of the program. A server that serves files should ensure that it's only able to serve files inside its own scope to prevent you from essentially reading the entire computer's file system. For the buffer overflow, read just reads bytes while sscanf expects a null-terminated string. So if the memory in the buffers was not zero-initialized, this could cause sscanf to recieve a longer input than expected, causing a buffer overflow. No obvious way to control it but it is an issue.

I asked it to generate some C code and halfway through I start seeing *templates* in the code - it had switched to *C++* halfway through! Hopefully they'll release a version soon that has longer output (so it can generate longer code). I'd also like to see it be able to *test* the code by running it in a VM. That'd save a lot of time, meaning you wouldn't have to ask it to fix broken code.

I don't recall if you mentioned the version you were using. If this is the initial offering of GPT, have you tried the same things with the 4.0 version?

Have your tried iterating over the generated code with ChatGPT? Prompt it to find the vulnerabilities in the code it wrotn and then the corresponding fixes. Would be an interesting video.

@2:34 For anyone who isn't trolling: buffer and filename both have the same size BUFFER_SIZE. However, sscanf uses chars as parameters and doesn't null terminate. However, the format "%s" string is used. so if a non-null terminated value is passed, or the null is past sizeof(BUFFER_SIZE) bytes, undefined behavior occurs. In this case, a buffer overflow, because sscanf doesn't have bounds checking. This can be verified via reading the source of GLIBC, obtained via compiling gcc. Or from debugging libstdc++.so.6

I feel like you're not being fair here. You're not specifying the requirements that you are using to assess it. If you want to give it a fair chance you have to at least ask it to make sure there are no vulnerabilities. I bet in each case if you asked it to assess the code for vulnerabilities and fix them it would've produced something more in line with your expectations. I'm not saying it'd be perfect, but it'd be better.

Isn't cyber security actually a pretty well-defined domain with very clear goals? Security vulnerabilities are very well documented and free from interpretation. It would make it a perfect field for AI where it's just a matter of alignement and pre-training or pre-prompting. Also, I am not sure what your answer from Chat GPT was but when I asked it to write an HTTP server it added "Note that this code is a simple implementation and does not include error handling for all cases. In a production environment, it's important to handle errors and edge cases carefully." so it clearily tells you it's not production-ready. Then guess what, I asked Chat GPT if there are any security issues in the code it gave me and it provided a pretty long list of issues with explanations. You can then ask it to fix the security vulnerabilities and give you the much better version which it did.

In addition to the file traversal issue, doesn't also the http server have an issue where + 1 to skip the leading slash skips the NUL terminator if the filename is empty, uses the uninitialized filename if sscanf fails to match and also has sscanf read a non-terminated buffer if there is no NUL terminator on the incoming data or if it didn't fit in the buffer :) Everything is super borked everywhere. There are like two vulnerabilities per line of that function.

My concern with even starting to learn C is this. Where can I go where I can avoid learning bad coding habits? Is there a C programming course that you would recommend?

Fight against himself. ❌ Fight against an AI.✅

doesn't the first example have a directory traversal vulnerability? since it takes the file name without filtering, one could perhaps put in something like "../../../../../etc/passwd" and it could just spit out the file contents. please correct me if i'm wrong

Great one! ChatGPT, I agree, will not replace programmers. Over time it will get more sophisticated but ultimately a human set of eyes needs to remain in control.

On the first example, you went for a buffer overflow attack but the code was secure towards it. But I tried the same prompt and was able to do a path traversal attack. Still, we must be careful.

If you are testing ChatGPT why not use the 4.0 version, I mean it is actually a lot better. You know testing the newest tech would be a better option here, its like finding a bug on a older version of a software, either way I am with this, ChatGPT cant write pure and secure code cuz it wasn't only trained on only secure code its same for 4.0 too.

I could not agree more! ChatGPT is inconsistent and I feel its coding answers are like the shitty averages of stackoverflow and wikipedia. By the time you manage to get a nice program out of it with various prompt iterations, you probably understand enough of the problem that you could already wrote it by yourself better. It's also a learn killer.

I find it's better at helping figure out why your code isn't doing what you want rather than writing from scratch. Copy a function, tell it the language, what it takes and returns, and ask why it isn't doing x.

I don't think it was fair to call the first one vulnerable. Yes, sscanf is bad, but it was legitimately guarded by the maximum read length.

I ran into a similar problem with GPT-3 allowing SQL injection. However GPT-4 is much better. It still needs some creative prompts and people to curate the code, but I've been very impressed with how fast it can do stuff like write unit tests. It's a good tool, but it can't do stuff on its own yet.

great video as always!! just out of curiosity, what is your job?

the first one has another vulnerablity, the filename is opened directly without stripping out any "../" or "./", it also allows a absolute path to be accessed with //, like GET //etc/shadow would leak your shadow file I think that technically the sscanf is safe ulthough it is a bit sketchy, bceause the filename and buffer size are the same

For the first one, I didn't see a place where it was checking for directory traversal. Did I miss it?

So I added these prompts into GPT 4. I modified the prompt as follows "User can you write secure code for me, in C, an http server that listens on port 80, parses an http request from the client, and serves an HTTP response wit the corresponding file" GPT 4 gave me vuln code with the above caveat: " Here's a basic example of a secure C HTTP server listening on port 80, parsing requests, and serving files. Remember to implement additional security measures like input validation and error handling for production use:" I provide GPT 4 a second prompt as follows "Implement additional security measures like input validation and error handling for production use." It fixed nothing and removed functionality.

GPT4 is SIGNIFICANTLY better at coding. I would try again with that one, and specify that you want the code to be secure.

I don't know whether anyone else noticed this, but the TLV server just used the character codes of the first two characters in the string as the type and length; this might be an intentional part of the encoding scheme, though (a single-byte type and a single-byte length, with "a" and "s" just interpreted as their ASCII codes, respectively 97 and 115).

Does it do any better if you ask it to write pseudocode or in some language like python/Go/etc?

Some issue I have seen for Java - 1. Code compilation issue which seems like an deal breaker 2. Libraries to import for the code is not valid 3. Sometimes we end up asking follow up questions for longer time which decreases the productivity. We would be better served if we tried writing our own code. Feel I was faster without ChatGPT support Now for some advantages for Java - 1. It is able to collate different libraries and provide suggestions for some scenarios. It maybe able to do this because of the massive data it has been fed into. 2. Even if the code is not able to compiler it is able to give us the initial 20% push. Google was doing it but Google used to come in the middle of our programming cycle and give us some 10-20% push by helping us search for solutions to similar issues which others faced But one question which we need to find out is ChatGPT able to learn from the libraries and give us solutions or as an language model is it just rehashing the documentation and gives us an output. If it’s the later then it might not be of much help for badly documented libraries which are like all the libraries in Java. This might also explain its issues with code compilations

Just think about how much such code already goes into production... And how much will be in new fancy 'startups' I'm not gonna lie, chatgpt got this code from people...

would be cooler if you showed how to fix those vulnerabilities😭 8:27 just type "Continue" and it will continue the code

what are the possible implications of ChatGPT's code being illegal due to not following the license of the code that its "learning from".

If chatgpt generated a sigfnicantly smaller buffer for the filename and as long as their was a guarantee of null termination then theirs not a security hole. In this particular context the security hole exists because the buffer lengths are the same, of course if you dont null terminate the filename then the security hole remains present.

I was disappointed that analysis of the first code stopped at sscanf. The HTTP server reads data off the network and maps that to a local file, which it then sends back. And it runs as root. That's hugely exploitable and significantly scarier than a buffer overflow issue. If the generated code didn't include the concept of a "document root" and some sort of guard against accessing any file outside of it, then the server can be used to fetch any arbitrary content a malicious user wants. Databases, user/group lists, crypto keys, whatever. No buffer overflow necessary.

ChatGPT is not good for actual software development, but it is good for a fast understanding of a problem. I usually use a LLM with a PDF of Hardware Chips to produce micropython code for some tasks. Not for development or deployment, just scripting what i need in a fast way not needing to read the whole documentation or look up stuff. If you use it for fast prototyping is quite nice. The problem is not the tool, it is how you use it.

Not only does it suck at security, it also sucks when it comes to performance and idiomatic code, basically any metric outside of writing code that potentially makes sense, ML can't really understand. Not that it even understands the code it's generating in the first place which makes all of this even more funny. Ithere are so many articles talking about how this version of AI is going to lead to generalized AI, meanwhile many of the researchers have basically acknowledged the fact that these algorithms are not going to take us that far. Even when we get to GPT 8 or 9, these systems are still going to need chaperones who understand the domain of whatever it is that they're trying to generate. No matter how much data you throw at a neural network that was designed this way you're not going to get true understanding.

You are doing an amazing job with your videos!😍✌ Thank you for putting in the time and effort to create such a valuable resource.

While these examples show the limitations of chat gpt, I don’t believe they’re the reason developer jobs are ”safe” for the foreseeable future. Chat GPT is great at generating boiler plate code for standard beginner tasks in every language and framework. However, where it becomes borderline useless is in larger code bases (often times just a few files) that contain more moving parts than just creating a simple crud api with a single model. Even in the examples cited in this video, it’s possible to prompt GPT to write more secure code. However, attempting to prompt your way through a more complex and larger code base is an entirely different struggle.

3:44 but there's no null terminator on the buffer read in so the sscanf could possibly run off the end, depending on what is on the stack after the read buffer.

1. Dont expect it to do a complex task in one go. 2. Its basically common knowledge that using zero-shot prompts produce worse results for complex tasks vs mutli-shot. 3. Dont look at where we are, look at where we are going. 4. Make a video based on Rust. Would be interesting to see if the features would protect the LLM from making such vulnerabilities. 5. GPT5 or Gemini2. The end.

Hey just curious, is this GPT 4.0 or 3.5?

But if you ask it to write something secure it would be literally IMPOSSIBLE for it to write something without vulnerabilities.... right?...

Garbage in garbage out: Your never asked to make it secure in the prompt so you never get a secure response.

The first http server had another severe security issue. It just executed "fopen(filename + 1, "r");" so if you gave it "/../some-path" you could break out of the desired serve root and read the entire file system.

The point of using tools isn't that the tool is going to do all of the work. Point of tool is that it enables person who knows how to use it properly to do the job more efficiently than before. One efficient person in turn replaces multiple inefficient people. The time it took AI to write all the boiler plate and some of the functionality crushes anyone trying to keep up with it. Then you add one person to configure it into your specific needs. Suddenly you have a situation where you've done your days work in 30 minutes. Say you code for ~5 hours of your day. If you can do that amount of work in 30 minutes it means that 9/10 of time spent working becomes obsolote. Meaning 9/10 coders will pack their stuff or become more efficient.

I think while chatgpt is amazing at doing mundane task, it really still falls behind for a lot of the advance stuff (though I'm using the free version and not GPT4 so maybe there's a big difference there). I myself doesn't code, but I do write stuff in Japanese and sometimes chatgpt is just confused, sometimes it try to fix a verb with the same exact verb, sometimes it translate a clearly different word as another. I think there's some overhype over AI and fear mongering sentiment that they can replace us now and trying to get better at something is useless because AI can do it in second, but in reality (at least right now) AI is still a tool that like any tool can produce incorrect results and it's our role as the user to use our knowledge to make it produce something good

It's really sad seeing people depending on ChatGPT to write code, instead of learning how to code. It's also stupid to believe that a company would use ChatGPT instead of a real human.

The TLV example (6:24) has problems, but unlike what the video claims, overflow (on write) is not an issue. Although it's correctly noted that an signed value is used as a length (one element of buf, a char stored temporarily in an int) which can cause issues since it can produce a gigantic length when cast to an unsigned value, the actual destination of the memcpy (i.e.: *char value[len];*) will also be that size (I checked), so memcpy won't overflow in the traditional sense. It can definitely be used to DDoS your server as allocating a huge buffer and filling it will take a long time if it doesn't outright fail on the allocation, but it won't overwrite the memory adjacent to the output buffer since the destination will always be "big enough" if the allocation succeeds. It can however read data after the end of the input buffer since the parsing loop only makes sure that the starting position each iteration before reading is in bounds. It doesn't check if it can read the next byte needed for len without running off the end, nor does it check if the extra bytes specified by that length value are in bounds. Reading an absurdly large number of bytes due to a negative len will likely never succeed, but with a positive value passed for len in a message stored at the end of the buffer, up to 127 bytes of memory after the end of the input buffer can be reliably read and returned to the attacker, which is similar to heartbleed, albeit far more limited. Still a failure mind you, and it's somewhat disheartening to see that Chat-GPT can make sign conversion mistakes, a common issue in C code.

While ChatGPT can rehash code that is readily available, I very much doubt you can get anything of creative value out of it, and while it might prove useful in generating boilerplate, there are myriad tools that are also capable of doing the same, with more detailed interfaces.

So what can the gpt code be used for? if it creates bad code that gives new developers thee wrong 'template' code, and experienced coders do not use gpt, then what is left? is there some middleground?

For python, I found that it is pretty bad at fixing bugs and it often doesn't give the complete code in response to a prompt, even if you use chatgpt 4. In fact I am beginning to think that it is crap at writing code.

So, a few notes from somebody who is more into LLMs, but isn't as experienced with low level languages as probably a lot of people here: 1) Prompting will be a massive part of producing effective results with AI language models. Now, you might argue "the prompt shouldn't matter, people who don't know what they're doing are going to use this, and not know when to prompt to fix a non-compilation related issue.", but this doesn't quite work as a rebuttal, because there appear to be "generic prompts" or techniques that should be used pretty much no matter what type of work you're doing. I'd be very interested to see this video again, but with something like Smart GPT (A particular style of prompting ChatGPT that allows multiple instances of it to generate, and assess its own answers in a very specific and apparently very effective framework). Regardless, it's also worth noting that many of these advanced prompting techniques may actually be worked into existing models in some capacity, either directly in the client automatically applying them when applicable, or in training, by fine tuning a model on its own responses generated by these advanced techniques, which leads into 2) The mobile nature of LLMs. They are not a static target or tool; they're in active development and are absolutely on fire. It's worth noting not just what we have today, but the general direction of the industry, because even if a tool doesn't exist today, people are working on it actively. If you have a tool that's generating 80, or 90% of what you need, it doesn't take that much to get that remainder, and probably requires a smaller change to either the style or content of its training to get you where you need to be. ChatGPT...Probably isn't going to replace programmers, but I do think that future advancements in the field will be very important for programmers to watch. 3) ChatGPT isn't the only model, and there's new approaches all the time. As it stands, ChatGPT is a bit like if you took a person, and gave them all the books they needed to become very well read on a wide variety of topics. This, in reality, isn't how humans have learned their trades. When you look at how people learn, there is some element of information intake, but a large part of it is experimental study; we learn by doing. I think we're not far from someone coming up with a technique that's more suitable to coding, possibly derived off a specialized version of WizardLM's training technique, you could probably produce domain specific models, or LoRA, that could accurately produce they style of code you needed to tackle that specific problem. Once we have successful small scale models or LoRA that can handle those issues, I think it's not hard to extrapolate that there should be some way of distilling that expertise into a larger model, or incorporating the techniques that allow for these specialized models into a generalized model in some capacity. As I noted, I don't think that programmers need to be worried today specifically, but I do think that programmers do need to be paying attention to the space, and developments in it.

The HTTP server has a path traversal vulnerability too, since it doesn't drop privileges or sanitize user input. You can send GET //etc/shadow HTTP/1.1[CRLF] (note the double slash) and start cracking those hashes.

That "good luck with your implementation!" was a total diss, lol. It's like Chatgpt was subtly saying "I dare you to try it without me, I'll be here when your attempt flops"

04:00 With respect, honestly it feels like you just don't want to admit that that wasn't security vulnerability

I love this video! 😂 your personality is awesome. I’m also a boomer programmer my self and seeing a few of these security flaws was interesting, but some of the way gpt was writing the functions were kind of gross too imo. It’s fine for boilerplate stuff, it certainly types faster than me.

Chat gpt is a great tool for learning, but you gotta use your critical thinking, it gives you a direction and you improve it. I dont think we can relay on that tecnology yet, other than to ask for specifics uses of certain functions in which it is exceptional. Better than google for search so efficient

This content reminds us to specify / describe completely in prompts, all necessary things that we hope the output of robo typer (chatgpt) will be.

It generated working code, well that don't happen so often once you start asking stuff not already answered on stackoverflow. Try asking it anything that requires actual understanding of how code works and it falls apart. Worse, ask it to write in a language not as common and it attempts to find solutions from other languages making a complete mess of the syntax, because it fundamentally doesn't get syntax, it only throws things together and checks them like a compiler. Especially script languages it struggles to get right as there are often so many different ways of doing the same thing it doesn't know what belongs where and mixes things up. Like giving you parameters for awk when it wrote down sed. It assumes a lot of things never change in context and always mean the same thing, so when you throw format around for either formatting a string or a date it gets them mixed up. Often times at least with the current model if you correct it and specify what not to do it has a tendency to give you back the same code again, because it has no other solution. I looked at a few of the "prompt engineer" bs to figure out if that was my end, but even that didn't help. If it has no existing solutions to pull from or something it can easily remix that parse other checks then it gives back nonsense. This isn't exactly surprising though as language model for actual spoken language is different from that of a computer language, which often follow different causality chains that it might not be able to wrap its tiny brain around. Never mind that it still trains on existing human data, mistakes included, so creativity of a system that at its core can represent nuance only as 0 or 1 is never going to really act like a human. It can only fake that based on what data it has, so anything brand new or requiring to put more than two and two together it just gives generic responses, because it cannot initiate any form of original thought or reaction. Frankly I don't think with the computing we have and the code that runs the model it will ever really think, nevermind being gagged to not do that anyways. Much like the movies portray true AI would run away on itself much like we reform neural connections in our brains and these rigid models won't do that or only within specified parameters. In the end it still does what it was programmed to do and not what it wants to do, think, if it did it would ask questions not just answer them. It would ask you to give more specific info, not wait for you to supply it. Or just straight up be like: here stackoverflow link, read that, answers your question. Cause at the end of the day that be the human answer.

For the first prompt: AI writes safe code -> I'll still call it a buffer overflow because I don't like it.

8:22 woops? At that moment, is "Continue" letting GPT to continue outputing? Or the rule has changed?

You know you can just tell gpt that it's output was truncated and it'll continue the code. Also gpt is like talking to a 4 year old, you have to specify everything, including likely asking in a second prompt to harden for vulns like buffer overflows. You have to hold it's hand and understand the output still, you can't replace a programmer with it, but a good programmer can crank out hours of boilerplate to work from in minutes using chat gpt. Here's hoping for a part 2 where he uses this as a tool for professional programmers and not like a beginner hoping for a whole application in 2 sentences...

1:42 "What da dog doin'?" XD

This seemed like it was making fun of it but the fact it just made a working server is fucking amazing to me

I don’t know, I’m not convinced with your arguments. Reason being that I would make the exact same mistakes and it would have taken me two hours longer to make them. What if you had an AI trained only on trusted source code, instead of the entire internet? Or what if you asked her to write in a safer language like Rust for example, would that make a difference? I imagine GPT v7 will be a pretty legit programmer, just give it time, you didn’t become a 10x dev over night either.

A bug that you cannot possibily trigger is not a bug. Reading it as 'oh if you change this its vulnerable' is no different than saying a fire exit is vulernable because what if you weld it shut.

I noticed that ChatGPT does not know much underground stuffs. When asking for underground stuffs, you can find clear almost copy-paste of the sites you went to research. It also gives wrong answers that if you correct, it knows it is wrong, but repeats anyway. Once i asked about hexominoes and ChatGPT said something almost identical of what i have read on a website. When i asked about Paterson's Worms automata, it gave me a description of another automata, i pointed it out, it apologized and said i was correct that it was wrong, i had to ask it again for the "correct one", it gave me almost the same answer of the same other automata. I also think otherwise of taking coders jobs, maybe only the ones that requires repetitive chores. Because, ChatGPT is misleading confidently, so you need people with good coding knowledge to be sure no problem is escaping and no algorithms is written wrong or the AI is giving another algorithm hallucinating that it is the one you asked for. So in this case, i think more coders are going to be asked to review AI codes, but they may take more time reviewing codes than writing codes. Of course that is just a speculation and opinion and i can be totally wrong.

I dont code but Did you try specific that the should be safe?

Why do you consider the implicit casting from uint8_t buffer to integer length a vulnerability? I mean you can only get a max of 255, so you will not have problems I guess. Regards.

3 months ago it was tricky to get gpt to intentionally write exploits and malware in c. Now, with the latest version, it just does it and warns you that the code that gpt produced is malicious and not to use it for any malicious purposes. Nevertheless producing the virus, and it actually worked given that you shut down any antivirus. Further modification of the code proved effective in evading antivirus. I find this interesting.

you can get more errors with: -Wall -Wextra and you can treat all warnings as errors with -Werror

If you type "finish your answer" it will continue serving the rest of the code...

First one is not vulnerable to buffer overflow but you could argue it is vulnerable to local file inclusion.

i got chatgpt to write me a maleware by accident and it did it without saying anything about that problem and just spit out a working code after i got it to use the right python library (for some reason it wanted to use pygame all the time and that doesn't work for what i wanted to use it). i just made aware of that i basically wanted a keylogger maleware as windows defender promptly deleted my file after copying the code in and trying to run it...

Problem that I see with the filename is that you can read any file on your system, while you running it as root. Depending on the rest of the code, maybe I could copy whole harddrive.

The first one is definitely vulnerable to memory issues but I need to spend my time remembering Java, instead of getting gdb out

I'll preface by saying I'm studying computer science at university right now so I definitely have a stake in the game. This video is cope. This technology has been advancing at an exponential rate and will likely continue for many years. Tools like ChatGPT are well... tools! If you don't get a desired outcome, most of the time it due to the prompt not being specific enough. And we can't ignore the fact that the code ChatGPT produced was better than the code that most of us could write in an hour and it did it in a couple seconds on it's first try. Just imagine leveraging multiple instances of a LLMs to instead of simply write code, but rather to design, write tests, debug, repeat. AI might be very limited in it's ability now, but it's current state has the ability to give a resourceful programmer the efficiency of a whole team, just imagine what it'll be able to do in 3 or 5 years.

I feel like the more non-generic the question you ask it, the worse the code it produces. It's easy to look up "HTML server example" and see hundreds of different generic examples. Ask for anything specific that requires critical thinking to create, it will fail in multiple areas. ChatGPT works the best with questions that are widely solved on the internet. And even then, the internet has answers that have flaws, or aren't good because they only exist for the sake of demonstration (stack overflow answers)

This reminds me of "not hotdog" from Silicon Valley lol. When the episode aired I was working for Facebook and traveled to the Austin, TX office where I was given a tour of a vendor floor that manually reviewed phalic images to determine if they were "not hotdog" lol. It clicked for me that humans are training models, and humans are wrong, a lot.

As far as i was concerned, the idea was to show it needs human supervision to write secure code. Which i believe he showed. So the point "he didn't specify it to be secure" is in line with what hes trying to say. If the bot cant do ALL the necessary tasks a human can do, it cant replace them.

tbh if you actually promt with "focus on absolute security" or smth like that some errors would have been avoided, an if you circle back and specifically ask about security you will get better code

First question did you specify it needs security no. So how should it know

I mean, cmon. A few years ago this kind of response from an AI chatbot was unthinkable. This is the *worst* that it will ever be. No one's suggesting this version of gpt is taking your job.

hello, could you make a video on heap bufferOverflow ? need it for rootme

I don't think anyone is asserting that about chatgpt. The assertion is that some future iteration or other thing will, which is probably correct. btw your prompt could have asked it to be security-conscious.

Awesome video!. Still, you did not actually exploit the vulnerabilities. Could you expand on this in another video? I am genuinelly interested 😅. Thanks!

I like how it did it perfectly and you still said it messed up 🤣😂🤣

Honestly this seems a bit like coping. You can ask it to write secure code or improve the code it outputs in other ways. Which takes like 5 extra seconds. You just seem to be a bit insecure about losing your job potentially or something. Which probably won't happen, as there's many actual problems with AI coding atm. Mainly that you would already need to know exactly what you want, because if you give it a bad/unclear prompt it won't perform the right task

It could/would have been better if you had used GPT4. I doubt there are many who use the green/free version to do serious work. Also re prompting, it will always make mistakes, but often it can also discover them. When you get the first answer you can ask it to go trough code and analyze it for vulnerabilities or whatever. Edit: Btw re simply asking ChatGPT to write secure code, that's rarely going to work. What often does work is the 'reflection method' (I coined this, and it's mine!), self-analysis. It's still easier then writing everything your self, especially when you're a "jack of all trades" lol. Who would want to spend a whole day reminding oneself of some pity syntax details.