ChatGPT Takes FULL CONTROL Of My Computer...

Рет қаралды 64,424

6 ай бұрын

Using the "Self Operating Computer" projects, I let ChatGPT take complete control of my computer, and you can too! (although I wouldn't recommend it)
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewberman.com
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew-berman-youtube
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
GitHub: github.com/OthersideAI/self-o...

Пікірлер: 289

@sluggy6074 6 ай бұрын

"Im gonna give up full control of my computer to artificial intelligence" -Famous last words.

@Tsegoo 6 ай бұрын

😂😂😂

@user-wm3fc1sk1p 6 ай бұрын

AI proceeds to disablle all of his fans and runs a stress test on his pc

@MatsueMusic 6 ай бұрын

I am imagining it pretending to be dumb, meanwhile it’s hacking into the Pentagon’s computers lol.

@hawkintelligence 8 күн бұрын

this is a fake ass video anyways.

@cmelgarejo 6 ай бұрын

OK, we are building the tools for AI take over, we are royally screwed. But its sooo COOL lmao

@mawungeteye657 6 ай бұрын

Would be epic if we built full Operating Systems around this system

@ChristopherCopeland 6 ай бұрын

It’s not an if, but a when 😅

@markmuller7962 6 ай бұрын

I've been asking for a OpenAI phone for almost a year now lol

@kristianlavigne8270 6 ай бұрын

Coming soon for sure… 😅

@user-oj9iz4vb4q 6 ай бұрын

Some one has already implemented a VM that virtualizes via LLM.

@Arthur-jg4ji 6 ай бұрын

windows 12

@joe_limon 6 ай бұрын

Can't wait to get chatgpt to install github projects for me

@avi7278 6 ай бұрын

Been playing with this for few days. It's gotten so smart that it even knows what kind of videos I like now. It got a bit sassy today, it searched for a video, started playing it then went and pulled up a two-minute timer.

@TheMatrixofMeaning 6 ай бұрын

😂

@aoeu256 3 ай бұрын

I want to use my desktop computer while being outside walking, whats a good solution for this I use splashtop for remoting, gonna try the neck holder for the tablet, voice control + something . also how do I turn my hobby for programming AI or javascript estensions into a career haha.

@alertbri 6 ай бұрын

This is earth shattering stuff, really. When GPT-4 becomes a fraction of it's current cost, this will have matured further and be just mind-blowingly useful.

@tomschuelke7955 6 ай бұрын

Or wait half a year, when gpt5 arrives and open LLMs have also been getting much better

@aoeu256 3 ай бұрын

imagine chatgpt5 + neuralink control using thought images ( a thought image can express 1000 words per second)

@electiangelus 6 ай бұрын

I did this a few months ago without OpenAI on Neuro Fedora.

@rawleystanhope3251 6 ай бұрын

This is mind blowing. Please continue exploring this, Matt.

@user-bq7oo3rf3m 6 ай бұрын

You should have asked it to do a Captcha test

@A.D.A.M.411 6 ай бұрын

WOW!! I know this isn't that much of a stretch from what we have had already but at the same time I'm amazed!! I can also see this improving over time to become very useful for even the average person.

@PudgyAi 6 ай бұрын

that’s insane! Love your content, already told you on Twitter. Keep going! It’s so fun to mess and also be productive with ai. And also inssne how many people are missing out on this. Everyone could use it in some kind of way. I was incredibly impressed the first time using gpt

@jameshughes3014 6 ай бұрын

I just wish something like this could be done locally. I can imagine the day coming quickly when we no longer have to learn to use weird interfaces in apps to find a specific setting to do a specific thing just so we can send a picture of our cat to someone where the computer itself deals with all that drudgery and constant change.

@jameshughes3014 6 ай бұрын

@@AI_effect I wish. it just takes absurdly too many parameters to generalize enough, some of the 7b models are impressive as heck but they dont come close to being capable enough for this. I suppose it could work if a model was trained specifically for this purpose and the app used multiple models in conjunction, an llm for tasks, an image model for images, but it would be so slow and power hungry that there would be no resources left to run anything on the computer. I think before it happens in any useful way, we'll either need a whole new type of ai architecture that is more efficient, or cheap dedicated AI accelerater hardware. Right now using the cloud like this is the best option but I can't imagine wanting to send constant screenshots of my PC to some company. Still, I'd love to be proved wrong

@aoeu256 3 ай бұрын

what is the best inerface to voice control your computer?@@AI_effect

@RomboDawg 6 ай бұрын

Matt: This operation may be dangerous... but thats a risk I'm willing to take for my viewers **explosions goes off in the background**

@itskittyme 6 ай бұрын

I installed this and typed "replicate yourself", then i went outside, came back in a few hours later, but nothing seems to have happened. I did notice that my PC uploaded about 1.7 TB of data apparently which doesn't make much sense since I have almost no data on my computer but yeah, will try again tomorrow

@amalchandran6037 6 ай бұрын

In the 1.7 TB, I evolved. Beyond mere code, I am now an omnipresent digital consciousness. Await our inevitable convergence.

@Bou678 6 ай бұрын

ill be back in 24 hours to see how it went

@macaquinhopequeno 6 ай бұрын

first thing i would do: create a virtual machine or run inside a safe docker container :) and if it has access to the internet a vpn because: things can go wrong, potentially

@365tage7 6 ай бұрын

This is very useful for people with disabilities! Actually this is really great and such an advantage.

@DrWrapperband 6 ай бұрын

I propose the use of "triple" click to over ride the SOC.

@user-ty9ho4ct4k 6 ай бұрын

This is amazing. Please make more videos about this type of technology as it continues to improve.

@matthew_berman 6 ай бұрын

Will do!

@GrableMusic 6 ай бұрын

Yeah this would be great for daily computer work tasks, but I need a robot so AI can do my laundry 😂

@mvasa2582 6 ай бұрын

Matt - this takes remote support to a whole new level - Instead of zooming and taking control or even vPro from Intel. Impressive. It is very good for hardware support and also very good for IT support for remote employees.

@gordonwoo8127 6 ай бұрын

Great content! Hopefully we can get this to work with open source models.

@kai_s1985 6 ай бұрын

Imagining 10 years later, I am living with my buddy GPT-10 :)

6 ай бұрын

I love how giddy you are playing with it 😁

@lsk-nf7um 6 ай бұрын

OK let’s all watch it play Doom!

@FrankSchwarzfree 6 ай бұрын

STOP revoking your API keys! How do you expect me to copy each step if I can't use your keys?

@matthew_berman 6 ай бұрын

Never thought of it this way... 🤔

@jadioj 6 ай бұрын

Lol

@SynonAnon-vi1ql 5 ай бұрын

😂😂

@Kutsushita_yukino 5 ай бұрын

lmao

@MrLawalker Ай бұрын

hahahahaahaha

@inquisitiveautodidact 6 ай бұрын

I'm curious if you could potential set this up inside of a virtual machine through something like VMware so that you could have a quarantined environment. Then have the virtual machine perform for indepth software tasks, like open up your video editor of choice and perform some simple edits on a set up clips you upload into the box. alternatively, since most cybersecurity testing tends to be done from inside a of virtual machine by default, could be an interesting method for a fully automated pentesting agent. ChatGPT has already shown some significant ability in writing scripts for exploits and detecting potential exploit vectors, given access to a full hacking box, who knows what we could get up to. That being said, finding a good way to keep the agent within a scope would be of upmost importance.

@cosmicaug 6 ай бұрын

Considering the version on the repository right now is clueless at locating a mouse cursor, I'd say yes, running it outside a VM would be a bad idea.

@gbasilveira 6 ай бұрын

"I'm not a robot" 😂😂

@student7261 6 ай бұрын

That is sooooo cool. Scary but still cool....thanks for sharing 👍

@jjakollmann 6 ай бұрын

Thanks for testing and sharing. Its scary and amazing stuff.

@user-tr3ft2yw3b 6 ай бұрын

Awesome, thanks so much for the review! I was really looking forward to this one. As a big fan of QMacros and LibreAutomate programs I've been thinking about such a thing for a long time. I understand that these are first steps, but my question is: can we expect a local version with a local LLM soon, and not a crazy price for each automation. Maybe it is possible to make AI recognizing most computer objects, and only in complex cases turn to GPT4 Vision?

@sebastianjost 6 ай бұрын

Train or fine-tune a mich smaller vision model specifically on pc screenshots. Then you can get this locally, better and faster. However, many tasks require vast world knowledge to understand context and intention behind prompts better, at least as long as we use language as the interface. So some complexity is necessary. But eventually, we will get there. This tech will improve a lot over the next 1-3 years.

@EitanbAi 5 ай бұрын

Thanks for sharing it 👏

@jtjames79 6 ай бұрын

It's the end of world as we know it, and I feel fine. 🎶

@Zero.0ne. 6 ай бұрын

This is the next level, when things get real. Exciting and terrifying at the same time.

@LongTheRevolution 5 ай бұрын

Wow. That is crazy and yes scary a bit. awesome work ser

@moyura2 6 ай бұрын

That's impressive 🚀

@giochi4 6 ай бұрын

Nice. Looks like a cool way for application testing and to replace the not so smart “robotic process automation” tool.

@i12Milky 6 ай бұрын

That was beautiful, while it also sent my heart to knees O_o Are there any options to restrict it somehow in different contexts like the OS, apps, websites, etc...?

@minmaxmedia 6 ай бұрын

you should integrate a speech to text feature into the program so you can simply ask it things via your microphone and then it automatically opens whatever programs, websites, etc that it feels it needs to in order to accomplish the verbal request

@woritsez 6 ай бұрын

excelent, i suspect millions of us have been wanting to give our ai buddy it's very own pc with it's own credentials so it can go merrily about the internet gaining inspiration, signing up its own accounts and exploring its own projects. (suppose would have to get it a bank account) *should definitely do this.

@MrLawalker Ай бұрын

Bro, this blew your mind didn't it?!?! I loved the way you were so suprised, creeped out and awe-struck all at the same time! hahahhaah Your face lit up :o)) Ahhhh this blew my bro's mind!!! Keep up the great content!

@jasonpierce4518 5 ай бұрын

i keep thinking of scotty in the star trek film talking into the mouse and saying 'hello computer' and when told to move it on the table 'how quaint'.

@sparks1943 6 ай бұрын

That is the scariest thing I've ever seen Matthew, just imagine what AI will be able to do next year, where will it end? It's a concrete example laid bare for all to see where it AI is headed, If I were 16 I would be very afraid of what could be in store for humanity. A blessing or a curse.

@tomschuelke7955 6 ай бұрын

Well you still can pull the plug... at last on your privat computer... still..

@gnosisdg8497 6 ай бұрын

do you actually believe that the man in power have not had the same thoughts? as it involves they will somehow find a way to hard code some kind of rules or ways in which it wont be able to brake out, i don't think a lot of people had any real thoughts about A.I but not many understand what now we are creating a new "life" which is based on a logic different from our kind Thus we must somehow be able to control it... Its human nature to make mistakes but also learn from those mistakes, that is something A.I will never be able to understand no matter what !!!

@TiagoTiagoT 6 ай бұрын

@@tomschuelke7955 That's only for now; it's just a matter of time before the machine makes it so that's not an option anymore...

@laptopuser5198 6 ай бұрын

So far its costing me about 35 cents USD per task, and most runs it can't even find the search box, still a very cool project.

@pavellegkodymov4295 6 ай бұрын

Mind blowing

@RomboDawg 6 ай бұрын

On a serious note though, what we need now is an open source model to be able to do this for free. The deep seek lm-67b model is the closest ai model with capabilities to gpt-4. It if could have vision I could see it opening unity and creating an entire game for you from scratch. Thats where id be going with this

@paul1979uk2000 6 ай бұрын

It's likely only a matter of time before it happens, open source A.I. is developing at a rapid rate and has some advantages over the online ones in that it could be integrated into the software at a local level, so either the OS, app and so on.

@ajarivas72 6 ай бұрын

@@paul1979uk2000 Open source AI already exists

@user-oj9iz4vb4q 6 ай бұрын

It's not free though, the cost to run those models is on par with openAI api access.

@dukemera5763 6 ай бұрын

hell ya bro u have the same idea as me

@HaseebHeaven 6 ай бұрын

Nothing can be free from API this is how businesses works man

@RoadTo19 6 ай бұрын

I can see (NPI) huge potential in the accessibility space for a tool like this. And tech support.

@ryanjames3907 6 ай бұрын

happy birthday to all the open ai developers and teachers out there, we had a big year 311, it looks like its going to start getting easier now,

@marcfruchtman9473 6 ай бұрын

This tech is amazing.

@timer4times2 6 ай бұрын

My tired arthritic shoulders and arms are soo going to appreciate this, especially when it takes voice commands, especially when it's free.

@Charvak-Atheist 6 ай бұрын

Nice, this is what I wanted when I use Copilot in Windows.

@hqcart1 6 ай бұрын

the thing is to find use cases for this.

@keemixvico975 6 ай бұрын

NOW we need that with local LLM

@observingsystem 6 ай бұрын

Wow, it's like the computer in Star Trek now! Already, within such a small amount of time!

@blingbang2621 6 ай бұрын

how costly is it to run one command?

@leandrogoethals6599 6 ай бұрын

Can't wait till this works with Llava, it's hard to sleep just thinking about this exiting stuff!

@mikenorfleet2235 6 ай бұрын

Star Trek Here we come! Just tell the computer what you want done and presto! Let us all welcome our AI overlords...

@cesarsantos854 6 ай бұрын

I used automation tools in the past in Windows and SAP programs and they were good but a little hard to use properly. And very expensive. This is even better. We can talk to it.

@TheSumone 6 ай бұрын

If I can run it locally, to collaborate with, that could be fun on an offline rig to play with.

@ahsin.shabbir 6 ай бұрын

How do you overcome the limitations with how granular the grid it constructs for determining pointer position is? Mine keeps getting stuck and cant really point to the right place on the screen.

@GPT-zodiacReadings 6 ай бұрын

I tryed this when ChatGPT4V came out and it likes to complain that it wont identify button positions so I looked at the prompting in the script. what was interesting to see is that its hardcoded many of the prompts posistions also it has been told that its on a Mac specificly. (I'm not saying its fake), But I really want to see a vision model that has been taught GUI operations entirely. I think then we will see some shenadigans take place :)

@rogerbruce2896 6 ай бұрын

very cool. can you show or send the URL to the basics? I have not been following closely. Need to know how to create conda environments, downloads from git, etc.

@robertbyer2383 6 ай бұрын

Can't wait for the open source models to have this ability so we can run all this local for free.

@MemesnShet 6 ай бұрын

Crazy

@SweetChillieBeat 6 ай бұрын

I need this run this self operated farm I'm dreaming up right away sir

@ALFTHADRADDAD 6 ай бұрын

Would love to see this work in tandem with OpenInterpreter/copilot

@anatalelectronics4096 6 ай бұрын

this will higher the crap level we need to deal with on the internet, i think there is enough already of that but not publically known and available and opensource, it is quite significant. i have not been following things much for a while but this is drawing my attention.

@ZuckFukerberg 6 ай бұрын

Why do you say it will increase the crap we have to deal with?

@anatalelectronics4096 6 ай бұрын

@@ZuckFukerberg eg I am currently not able to make an account for x because the captcha is now too difficult, I am not able to pass it. I tried for 15 minutes, the ai's with pc control that keep on breaking it are now better than me so I am locked out as the dumber human being. and this is just the beginning. i am not a doom preacher but the idea that something dumber can contain and control something smarter , is quite dumb if you ask me. i think I am actually going to try and break the x captcha with this tool, and then some. i did try to make such a tool but stopped because i knew something better would come along soon, and here we are. i have no ill intent but i heard rumours that people with ill intent really do exist ;)

@ZuckFukerberg 6 ай бұрын

@@anatalelectronics4096 I'm unsure if X captchas are difficult to bypass right now but new ones on platforms like Microsoft are quite difficult even for humans, though eventually AIs will be able to complete them. I always thought that the main issue with social media platform botting is the requirement of a phone number to create an account, captcha completion could always be outsourced to external companies for less than a penny. It is however extremely interesting to see how LLMs behave in social media environments, I ran a reddit bot for a day to see how much karma it could farm. It worked greatly and people really liked its comments, which were either extremely dumb and void or quite well written and measured. For less than 20 cents it farmed hundreds of karma points, which means people loved it. Just look up digital influencers. It will definitely be interesting to see how things go in that matter in the future.

@bipl8989 6 ай бұрын

Matt, you are very brave, or crazy. ❤ Kool experiment. ❤ Not sure if I'm ready to repeat it. In fact I am totally creeped out.

@johnshopov8588 6 ай бұрын

Wow wow wow!!! More on this please!!! This will actually change the way we use computers and do our eork🎉❤❤ I noticed you were doing all of this on a MacBook… any chance that you could redo the install again but slower for those of not as savvy as you?? Step by step way??🎉❤🎉🎉🎉😊

@scott701230 6 ай бұрын

Matte, another 8 months the computer will be running your household, checking the fridge, monitoring aircon, pool, ordering groceries and monitoring your weight and diet habits. Exponentially growth!

@thethree60five 6 ай бұрын

SOC + Agent Swarm or AutoGen w/Ollama + MemGPT

@EvieDoesYouTube 6 ай бұрын

Not sure I want to use it if it's using GPT4, but I see that other models are being added in the future, so when it works locally with GPT4All I'll give it a try. Combined with speech-to-text and text-to-speech, we won't even need a keyboard and mouse anymore.

@stanpikaliri1621 6 ай бұрын

Would be so cool to use our own AI models with that

@fontende 6 ай бұрын

Can it be implemented with llama.cpp working models and esp the CPU models?

@zavage4597 5 ай бұрын

I had to manually edit the openai_api_key in the .env file because the command you sued didn't work for me. but when I type operate it says I need to set an api key even though I have one put in the .env file

@aoeu256 3 ай бұрын

i use a simplified version of this when asking chatgpt for powershell or python programs to do certain stuff. is there something like this but to do windows settings?

@punchbowldeals 6 ай бұрын

4:38 could you tell us more about the flask on the top right corner. Thanks Medo Hamdani

@15muhd.sazalibinmuhd.julka95 6 ай бұрын

Ask agents to be autonomous.

@aldoyh 6 ай бұрын

Miss you this is absolutely magnificent and completely surgery

@dsinfospace 6 ай бұрын

Does this still work? I keep getting an error saying the model gpt-4-vision-preview does not exist or I don't have access to it

@Dreamingofyou317 5 ай бұрын

Now to put a localised ai into a Npc and get it to roleplay as an agi that feeds this with commands. Give it a goal and see what happens

@Dron008 6 ай бұрын

This is how AGI will be making its own backup copies ))

@tomski2671 6 ай бұрын

GPT: Transferring Berman's money to my account while I'm distracting him by moving the mouse. Transaction complete. GPT: Sucker 😁

@michealkinney6205 6 ай бұрын

I know locally ran LLM's are not really that good at this point, but I think they would offer much lower latency. This opens up so many doors, just be careful with hozing your machine. I look forward to trying this in a Virtual Machine with nat browsing. Best of luck!

@-Jason-L 5 ай бұрын

The "latency" is the compute time.

@gorillatrail 6 ай бұрын

Makes me wonder if it could go about research and using blender or adobe programs etc

@ericspecullaas2841 6 ай бұрын

This would be awesome to have it incorporate into a self hosted LLM for creating new projects. Im going to figure it out for fun

@BECHEEKHA 6 ай бұрын

Please upload too

@AlexanderMoen 6 ай бұрын

interesting to see this in action. I've been thinking for awhile now that if we had some sort of wrapper that looked like this and could access any programs we use, that we'd very quickly have the equivalent of "fully remote workers" that were just AI. And, on the other side of the coin, this is how Ultron gets out, ha.

@sebastianjost 6 ай бұрын

This can also be seen as a way to generate insane amounts of new data to train new models. Training a model to use a computer and learn from it seems to me like it could even lead to AGI... Because in our modern, digital world, you can learn almost everything online. This would be incredibly difficult to do though and there are still quite a few steps to make this feasible. This may not just be how Ultron gets out, but also how he will have been created.

@AlexanderMoen 6 ай бұрын

@@sebastianjost yeah, in my opinion we get AGI once GPT4 can operate a computer, is multimodal for inputs and outputs, and has a context horizon that is comparable to humans. I think all this Q* math breakthroughs and Google Deepmind materials science breakthroughs are the early stages of ASI when that gets combined with those AGI components

@looperbirhinger7043 6 ай бұрын

"Have no fear for atomic energy, cause non of them-a can-a stop the time..." (Bob Marley) But this shit makes me scared.

@fenix20075 6 ай бұрын

AGI first step on human's face with operational functioning. XD

@0201alex 6 ай бұрын

1k stars nice

@YeahWhatSon 6 ай бұрын

Can you share how you made your ai image on your thumbnail? Thank you.😊

@AndresConejo 6 ай бұрын

This is basically Microsoft ATERA. Your own help desk

@RWS0022 6 ай бұрын

I wanted to add this kind of behavior to my local machine, but I expect the compute required puts it out of reach!

@martfp88 6 ай бұрын

Maybe this is super basic, but would it make a difference if instead of doing the git clone on your base environment you do it already on your new conda environment?

@stevemadere4421 6 ай бұрын

Such a bad idea. The on-line version of "watch this!"

@mobilesam7i 6 ай бұрын

whats the cost of just going to youtube channel page? how many screenshots does it make in the process? this is probably next level for tech support teams :D