Self-Improving Agents are the future, let’s build one

Рет қаралды 60,866

2 ай бұрын

If you're serious about AI, and want to learn how to build Agents, join my community: www.skool.com/new-society
Follow me on Twitter - x.com/DavidOndrej1
Please Subscribe.
Credits: @matthew_berman @maya-akim @AssemblyAI
This video is a deep dive into AI Agents and how to design them with AutoGen Studio.

Пікірлер: 116

@DavidOndrej 2 ай бұрын

💻 All the code, prompts, resources are now available in my community. Join now! www.skool.com/new-society

@dillonscott5864 2 ай бұрын

i am not paying 70 dollars a month for 4 prompts thats insane. If you are gonna paywall us that hard at least show the prompts completely in the video.

@henrismith7472 Ай бұрын

Who's having issues with the shell command in VS Code on Windows 11? Uninstall and reinstall. During the installation process it should allow you to install code command in path so you don't have to do it in VS Studio. Test if it worked by hitting windows button r and typing in code then hitting enter. If VS Studio opens the problem is solved and you don't have to worry.

@MusicAndStopMotion Ай бұрын

Thank you for the tip 🙏

@henrismith7472 Ай бұрын

@@MusicAndStopMotion I've never done programming/coding before. I think I signed up to a class to learn python once but never went. I ran into so many issues throughout this tutorial even though I followed the instructions perfectly. How I got through this was by using the screenshot feature in copilot. I'd upload a screenshot of the problem along with my question and go from there...

@latentspaced 2 ай бұрын

This one is superb, David. It's on my bluetooth speaker in my kitchen and I'm on my laptop in living room with it on yet at different parts. lol. I appreciate that you step by step go through your flow -it's clear precise structured and beautiful .

@chrism3440 2 ай бұрын

David, you are the man. Thank you for the work you're doing.

@derrickwilliams2877 2 ай бұрын

Great video! Leaving in the parts where the AI agents or prompting wasn't working as expected is a great way to show new developers that everything doesn't happen on the first attempt. Often times, new devs get discouraged because they watch vids & see the person making this really hard application in 30 mins & never struggling, but those videos cut out behind the scene struggles. Just a thought, would it possibly be useful to add in a 4th AI agent the code tester & their main function would be to write test cases and attempt api calls & if the code fails it could hand that error back to the planner so the system is iterative & also verifies that it works prior to giving it to end users.

@freddy29228 Ай бұрын

David, your vids are great! This is Vid #3. Very interesting, glad it found the answer. Mt Everest, then Pyramids of Giza.

@bennythe 2 ай бұрын

Thanks David, your first AI Agent worked like a charm, I am going to configure this now!

@DavidOndrej 2 ай бұрын

I'm glad you found it valuable Ben :)

@RBDesignWorks 2 ай бұрын

Excellent demonstration David! Thank you for producing this video on the future of Self-Improving Agents. A video I watched today from Matthew Berman called: AI Pioneer Shows The Power of AI AGENTS - "The Future Is Agentic" has similar commentary as your video from the CEO of AIFund without the demo. I thought it was very informative and timely considering how well your video shows how easy (or not) it is to create an AI agent. I subscribed to the channel and will watch more of Matthews videos hoping that I can someday create Agents. I wonder if this is something I can load into my robot assistant. Sincerely, RB

@maraisdekker2415 2 ай бұрын

Supercool conceptualization to setup an autonomous yet purposefully guided team that gets the job done.

@lovol2 2 ай бұрын

Good effort, nice to see other struggling with this. I found writing my own react framework for looping GPTs calling methods worked really well and while using auto GPT should work, seems somewhat more convoluted.

@kevinclark3333 Ай бұрын

David you are awesome sir, I really enjoy your content!!!

@frankdearr2772 Ай бұрын

great topic, thanks 👍

@ninanordbo 2 ай бұрын

love the content

@iamisobe 2 ай бұрын

do you believe any of these are capable of running an artists social media page and growing them? creating and posting contet?

@hmuller 2 ай бұрын

Thanks for a great tutorial, I got lost in several others, but your were truly great! Can the managers utilize a knowledge base consisting of several documents? And can the agents read throug and review documents providing the managers with information they cross-check with their knowledge base? I am currently trying to develop a workflow that incorparates this as well as letting the agents creating a report of the document reviews. I guess I need an API like ASKYourPDF? Or are there ways of using Open-Ais document information retrieval? If i get ahead of this before GPT4.5/GPT5 I will eliminate 80-90% of the time I spend my current job, that mostly is about reviewing documents and veryyfying that they are 1) legal, 2) acording to what the gouvernment policies 3) acording to the current state of what we know and want within my field of work - all of this can be translated into looking over a vast knowledge base. Can you also make a tutorial on coneting to locally running models thoug LMstudio?

@jonatan01i 2 ай бұрын

At 18:00 you don't have to be ultra descriptive because they'll just not obey. What you can do is to say it only that it shouldn't change anything, and then give an example, like "I give you here an examples: user: 'hey There blahbla.' sending to planner: 'hey There blahbla.'" And it would work much better compared to what you can achieve with trying to be more descriptive.

@mrd6869 2 ай бұрын

Yes self improving agents are the future..But you REALLY want to be a hit? Develop user friendly UI on the front end for non technical people. No nerd speak...drag and drop. Business people dont want to mess with API keys and commands in the terminal. All of this needs to be packaged and streamlined with visuals,so that a non coder can speak & have it built (devin)

@nickbobrowski 2 ай бұрын

On it

@candifemale5118 2 ай бұрын

@@nickbobrowskiLet me know when you do it

@sulyah 2 ай бұрын

@@nickbobrowskiyou need help?

@nickbobrowski 2 ай бұрын

@@sulyah if @@sulyahlet’s talk. @@sulyahat AI in Hand @@sulyah very soon@@sulyah!

@nickbobrowski 2 ай бұрын

@@sulyah we are excited about building reliable AI agents, and looking for contributors as we plan to open-source our AI in Hand Platform and build production-quality applications on top of it.

@mervinedward6985 2 ай бұрын

This is still difficult to follow but even as a novice and absolutely zero coding experience I can see that there is merit to what you're doing, I hope there will be an easier use for normies like me who can create and deploy agents in the near future.

@i_am_a_fungi 2 ай бұрын

This. I’m computer illiterate. I just want an agent to help me do my job.

@JurgendeMooij 2 ай бұрын

I'm in the same camp. I got everything to work but when rebooting I lost everything. I have zero context or understanding of what I'm doing and ChatGPT is clearly not understanding it either. If you have to program a bunch to get an AI to work as a programmer, it kind of defeats the purpose.

@beavis-fu8369 Ай бұрын

Just keep at it. After sometime you'll start to see and recognize certain functions. That happens when you work with certain code a lot.

@jimg8296 Ай бұрын

Interesting nice debugging. As in the long history of new software it's never as easy to make it work vs how simple it looks in the demos. I also liked your concept of self-improving AI.

@adelinadonisa 2 ай бұрын

I will try to remake this on my computer

@truthwillout2371 2 ай бұрын

I think you have to re-create the workflow from scratch if you edit an agent. That's why you were having issues :)

@kevinhoganAI 25 күн бұрын

David, thanks for this video. Which would you recommend CrewAI or Autogen?

@chriskingston1981 2 ай бұрын

ahhh love this video, so helpfull and inspiring to use autogen. Also so much to let autogen create tools for it self aka self improvement system :)

@PrinzMegahertz 2 ай бұрын

Excellent video! I'm totally not a python / visual studio code kind of guy, but thanks to your tutorial, I was able to set up everything within an hour. That being said, is it possible to have a fully autonomous solution, e.g. having you own LLM studio host an LLM and then have AutoGen interact with that local instance?

@Gatrehs 2 ай бұрын

Yes, If you look at models and the TheBloke/zephyr-7B-alpha-AWQ model you'll be able to see a good example of a local model.

@PrinzMegahertz 2 ай бұрын

@@GatrehsThank you. I actually tried to connect AutoGen and LM Studio, but it seems to not really work very well. The conversation usually terminates after three or 4 tokens and every individual token is communicated by a different agent. This seems a bit buggy and will probably work better in future versions. For the time being, I found a good alternative with CrewAI, also you have to have far better Python skills in comparison.

@andrewoods6742 2 ай бұрын

It is absolutely "HEiLL" trying to get VScode to detect Conda on Windows 11..... ABSOLUTE HEiLL

@NahFam13 Ай бұрын

It's easy...to me... but don't take my word on it because I've done it 3 times lol I had to wipe my drive multiple times after playing around with my environmental settings one too many times. The best course of action is to install MiniConda as Administator, and run through the installation, but make sure to check the box that says "Add conda to path" and install the python version it comes with as the default. After that, even after doing that, make sure to do WINDOWS KEY -> ENV -> select EDIT THE SYSTEM ENVIRONMENTAL VARIABLES, then click the "ENVIRONMENTAL VARIABLES" button and in that box you want to focus on the bottom portion (System Variables). You need to add both CONDA AND PYTHON. Scroll down and look for a variable named "Path", and double click it. Click "New", and add your Anaconda installation path. In my case, I decided to make it easier to find so I installed it to a secondary drive You need to add 4 different paths, the root folder, and the root of the scripts which are illustrated below: After you do all of those, restart your pc, and when you need to launch conda make sure you launch your terminal/powershell/cmd in Administrator mode. The easier way to do this is to launch CMD as admin, and run the following /// That -> setx /M PATH "%PATH%;REPLACEME" Make sure to only replace the text and not the semicolon ( ; ) before or the closing quotation ( " ) at the end.

@nickbobrowski 2 ай бұрын

30 minutes of struggle for one simple task… It's still not user-friendly with all the debugging efforts and handling API keys locally. At AI in Hand, we're building a SaaS Platform that manages the configuration seamlessly. We plan to open-source it soon.

@isrcinimatic587 2 ай бұрын

👀

@aaronjaggars321 Ай бұрын

Lmk please

@TheDesmosabie 2 ай бұрын

Been watching a few different people on YT about Agents, what are various examples of abilities within different areas of specific information ? Finance, FinViz screener is free and great example. Current as possible volume spikes, VWAP crosses over 22EMA, RSI hit 25 are others are examples of abilities I want. Searching constantly for these on a list of stocks I keep a list of and alert me to these things happening. Can this be done ?

@tzenmatteo 2 ай бұрын

who here has thought about doing the same, but in healthcare. I believe we will have the ability to substitute doctors (a few specialities at least) in the next few years.

@aoeu256 2 ай бұрын

Dunno you could do this for anything including marketing, making youtube videos, talking to your girlfriend, expanding your business, making breakthroughs. I think there is a hardware bottleneck right now, we someone to build self-replicating robots so that they can double each generation and have them build some kind of hardware factories as well as solar, water, air energy collectors.

@fatherfoxstrongpaw8968 Ай бұрын

when your working with external calls, you could be getting errors because your program needs to wait for a response. add a pause/wait for response.

@lorenzoleongutierrez7927 2 ай бұрын

On this stage, Perhaps would be more illustrative to start with the human in the loopand afterwards take it out from the workflow. Great video, great approach

@dbonneville Ай бұрын

how long does it take after adding credit to stop getting "you exceeded your limit" error in the model test UI in autogen?

@BOBSECURITY 2 ай бұрын

Please I would like you to do video on webhook, we were given project on that to build AI with reports but I don't know how to attempt it

@mordokai597 2 ай бұрын

i think the file writing issues might have to do with the fact when you launched terminal from vscode you put "--appdir folder" instead of a full path to an actual folder or "./" or "$PWD" it doesn't know where in the filetree to start looking for a directory named "folder" xP (In the voice of HAL-9000):"I'm sorry Dave, but i can't do that... Did you try turning me off and on again?" also: when you add a new value to env or manually export it, you have to kill the parent process and relaunch for it to take effect. don't feel bad xD the AI agent Devin had this same problem and couldn't figure out the error. it filed a ticket with it's own devs and asked when it could expect it's "secrets" to update after editing an env file . add a line to manager that says soething like "in the case of an irreconcilable environmental variable error, Request USER terminate and relaunch current running process, and send `TERMINATE` signal"

@jaysonwalsh7666 2 ай бұрын

hey david. which one should i use, autogen or crewai?

@revelari5250 Ай бұрын

use both. eventually it’ll be like choosing logic or protools

@user-kp4zo9hn7e 2 ай бұрын

Try not using an llm for the user proxy. I think that will fix the problem

@DavidOndrej 2 ай бұрын

This is a great suggestion actually! It makes sense not giving it an LLM... I wonder why Microsoft even gives us that option

@tiborsaas 2 ай бұрын

I know this is a demonstration, but you could get this function with zero-shot prompting.

@themax2go Ай бұрын

does this work w/o a local model (thus no api key)?

@yanwesley4181 2 ай бұрын

There is something wrong, i can't create an assistant. After install miniconda, i don't know why but my terminal don't reconize "conda" as a command. And in the visual studio, i can't find the option "Shell command..."

@justaguyonyt869 2 ай бұрын

same issue happening for me. im on windows

@OzzyMoto2K10 2 ай бұрын

@yanwesley4181@@justaguyonyt869 Make sure you are launching the anaconda prompt or anaconda powershell prompt programs (via Start Menu)

@rudolfviljoen2847 2 ай бұрын

Same

@truthwillout2371 2 ай бұрын

Sort out your environment variables in PATH. Google it :)

@kevinrstruck 2 ай бұрын

same

@Kutsushita_yukino 2 ай бұрын

Props to your cat

@zackerymcallister9057 2 ай бұрын

I am having trouble when I download miniconda, it shows that it downloaded but when I type in the commands into the prompt it says it is not recognized. If I join the community would I get help when I run into snags like this?

@henrismith7472 2 ай бұрын

Same issue, i've tried uninstalling and reinstalling miniconda3 and the command prompts aren't recognised. Using windows 11.

@ShaunPrince Ай бұрын

there are some steps to follow to get conda working, that are clearly listed on the site where you download it. If you use windows this will be a big challenge, if no impossible. I woudnt spend $77/month to get help on something so basic like this.

@GlenBland Ай бұрын

do you need a "user_agent" to execute code?

@RiseWith 28 күн бұрын

Isn’t it just the quotes around your keys in the .env ?

@AndyPandy-ni1io 16 күн бұрын

could someone help me do this on windows using AI to try help as suggested but its got me lost.

@faizywinkle42 2 ай бұрын

can you create an agent that could create game in unreal engine?

@deja00 2 ай бұрын

0:05 I like your cat

@user-cb6wo5qg4q 2 ай бұрын

Does anyone have the full system prompts or the ones from microsoft?

@squiddymute 23 күн бұрын

you need 3 agents to just write a function ?? what will it take to develop a small SaS application and how complicated will it be in that interface, let alone debugging it …

@adamjohnson4311 Ай бұрын

There was a kind of 'joke' I heard when I as a kid, or a cynical outlook?(I had to google it and turns out I treated it as it was originally presented.) As a thought experiment, I imagined it was indeed somewhat possible, given a large number(even towards infinity) of monkies with their own type writters, could, given long enough, collaboratively produce the works of Shakespeare (or Sir Francis Bacon, if you drank that koolaide(are you keeping track of the layers of sarcasm? If unsure, turn back now. (And incidentally, if you want a agent to follow instructions precisely, it would be fair if you followed the practice of precise instruction. Wait, semantics, thats it! All we got to do is have these agents 'police' or reward (provide weight) as to the 'end goal' and just lock it all in a room without a way to look inside and see if it and the cat blew up and is dead yet, because we have to be patience about these things, and you probably can tell I'm not much of a cat lover, cause I say check and check often, at least between near infinite computing cycles, measured by its relevance to real world results by the Alpha prime monkey / schrodinger coefficient factorized function value, optimized by the Wolfram computationally irreduction slide ruler, deloyed probably best virtually in Gary's Mod, oddly enough

@adamjohnson4311 Ай бұрын

In conclusion (oh the irony) "here we've bottled up an infinite amount of monkies combined with a time-dissolution mechinism and occlusion pre-formulative traversing algorithm, now go have fun but don't blame me if you end up suddenly with this single sex experiment ending in a theme park of terrifying pre-historic dinosaurs trying to kill you, JUST saying. You know "legal formalities." Can't help but suddenly see a narrative where these tools release for general usage , this "playground" of ai tools is really the next gen "captcha" DON'T YOU SEE?? Soylent AI is made OUT of PEOPLE!! (least with respect to capturing the essence of higher order problem solving data for machine learning.) Perhaps a shortcut to infinity? Let's ask the Owl!

@rraul 2 ай бұрын

Vscode has a bug with environment variable

@itsmedeka 2 ай бұрын

It sobering to realize that the last human software engineer has already been born.

@qAidleX 2 ай бұрын

How about the last living machinist, waiter, functional lawyer... I think they have been born too

@TheExodusLost 2 ай бұрын

Holy shit you guys…🤯wtf…

@revelari5250 Ай бұрын

@@qAidleXwe will still have waiter and bartenders. we love a beautiful woman

@DaffyDubzFPV Ай бұрын

@@revelari5250 until 3d printed AI women are born 😂

@revelari5250 Ай бұрын

@@DaffyDubzFPV true or holographic data 😂😂

@RyanSmith-rb1ch Ай бұрын

I see a cat, I like the video.

@aoeu256 2 ай бұрын

Keep at, if 100,000 people figure out how to improve our productivity by 0.1% we could produce 1.001**100000 = 22015x as much. We need someone to use the ai agents to produce self-replicating robots that create solar energy collectors or GPUs though. All you need to do is make the robot out of 95% legos, that the robot could then mold or print its lego parts.

@adamjohnson4311 Ай бұрын

I can't tell if that makes very close to absolutely no sense, or just makes no sense, absolutely at its very best.

@shimblypibbins 2 ай бұрын

I can tell this is too much for the average user to 'grok' and implement. I would have to block out an entire afternoon to follow along with this with a clear use case in mind already. However, this is still useful for future reference - maybe I'll come back to this.

@shimblypibbins 2 ай бұрын

your comment that the reward is the difficulty that is still in place to execute this workflow was enlightening

@bharath2508 2 ай бұрын

Thumbnail is 😮

@user-wx2fp9cm3i 6 күн бұрын

mapbox api is much better to get coordinates from a address

@hanskraut2018 Ай бұрын

sick now the world just has to work

@RX-8GT 2 ай бұрын

Oh Jesus😂

@efifragin7455 Ай бұрын

why u not installing via homebrew?

@ShaunPrince Ай бұрын

Homebrew is just for wealthy mac people.

@kaael1153 Ай бұрын

bro "aiaiai" is so brazilian lol

@DeepThinker193 2 ай бұрын

Hilarious. It's called a large 'language' model for a reason. Not a large 'coding' or a large 'math' model. It's the same reason why Devin will fail. Came here for a good laugh. Was not disappointed.

@rupertllavore1731 2 ай бұрын

“@DeepThinker193, gotta love the cheer squad, right? 😄 Here’s Devin, coding his heart out, and there you are, turning it into an open mic night! Looking forward to your next hit performance - ‘The Lighthearted Critic: Laughter in the Time of Code’ brought to you by GPT-4.”

@DeepThinker193 2 ай бұрын

Don't get me wrong. Devin will help somewhat. But the irony is that they will be hiring software engineers to help Devin. Just as this guy basically had to figure out the problem through trial and error. There will be many...many errors.@@rupertllavore1731

@rudolfviljoen2847 2 ай бұрын

Without language there would be no code or math

@AdamPippert 2 ай бұрын

@@rudolfviljoen2847exactly, it’s not like we learn algebra from smoke signals.

@truthwillout2371 2 ай бұрын

Why is your username DeepThinker? Is it sociopathy? Garbage take based on nothing.

@sven262 Ай бұрын

A bit too pushy on the "community" membership for $77 / month

@DavidOndrej Ай бұрын

Don't worry, I will increase the price soon.

@fredkzk Ай бұрын

Take it easy boy, don't speak so fast and in such a stressful manner. It feels like you are rushing so you don't miss your bus. Good content but unappealing style. Since you credit master Berman, take inspiration on his speech style please... Thanks for your effort and for sharing those insights.