Llama3 + CrewAI + Groq = Email AI Agent

Рет қаралды 54,475

Күн бұрын

In this video we combine LLama3 70B + CrewAI with the speed of running on Groq to make an AI Agent for replying to emails
Colab: drp.li/JS82H
Groq: console.groq.com
🕵️ Interested in building LLM Agents? Fill out the form below
Building LLM Agents Form: drp.li/dIMes
👨‍💻Github:
github.com/samwit/langchain-t... (updated)
git hub.com/samwit/llm-tutorials
⏱️Time Stamps:
00:00 Intro
00:33 Groq.com
01:00 Code Demo
01:37 Goals for this Demo
04:11 Setting up Groq LLM
04:36 Tools
04:44 Setting up Agents
08:01 Instantiating Agents and Tools
08:37 Building a Crew
08:46 Kicking off the Crew

Пікірлер: 93

@michaeldimattia9015 3 ай бұрын

Great content! This was my first exposure to groq... the potential here is pretty amazing! Thank you for your perspective and candid explanations that really help to grasp at the "ground truth" of these technologies. I love watching the progress in ML and LLMs, as people collectively explore boundaries and breakthroughs!

@kate-pt2ny 3 ай бұрын

Thanks for sharing, looking forward to the follow-up videos with langraph and ollama, thank you for your work

@yellowboat8773 3 ай бұрын

Thanks for the cideo mate, i feel like your of onky a few on youtube that actually dive a little deeper into these tools using different examples, other than the standard copy paste examples we see from everyone else. Appreciate it

@jimofthehill 22 күн бұрын

this is really inspiring. it opens up all sorts of possibilities, in terms of document processing , and combining it with web search

@am0x01 2 ай бұрын

Hi @Sam, I can't appreciate more, I would like to thank you for all the effort you are putting on to share these incredible contents, it helped and is helping me building my project around agents.

@kazoooou 2 ай бұрын

Great job! I'm super excited to try CrewAI. With LLAMA 3, it’s so promising! The future is being written now, friends :)

@caleboleary182 3 ай бұрын

Super cool! Wild to see it run through the whole agent flow in 11 seconds.

@samwitteveenai 3 ай бұрын

yeah I kept thinking it was a previous run when i was testing it, until I realized it is just that quick.

@clray123 3 ай бұрын

Not so wild if you're used to llama.cpp rather than Python crap.

@JJaitley 3 ай бұрын

@samwitteveenai. Excellent video looking forward to the LangGraph video on the same use case.

@watchdog163 2 ай бұрын

@@clray123 I want to do stuff, not read instructions.

@Trashpanda_404 3 ай бұрын

Dude thank you for all of your videos. You’re awesome.

@samwitteveenai 3 ай бұрын

Glad you find them useful.

@ratral 3 ай бұрын

@Sam, thanks. That was excellent help, as always.

@jdallain 3 ай бұрын

Really looking forward to your LangGraph video. I think it’s the best option, but also the hardest to learn

@samwitteveenai 3 ай бұрын

I do feel LangGraph is more stable and it feels more like programming to me. I agree it is more work at the start to learn etc. I want to try and bake in RAG as a tool with the next one too

@ArjunKrishnaUserProfile 3 ай бұрын

Yes, looking forward to langGraph video.

@SamCodeMan 2 ай бұрын

Thanks @Sam Witteveen, it has been very informative, I will start working on my RAG based project now with the help of your colab notebook!

@DonBranson1 3 ай бұрын

An awesome video focusing on effective use of Groq and Llama3 for agentic workflows. Looking forward to the LangGraph video even more. Some minor issues toward the end of the lab, but still teaches key concepts and capabilities. (couldn't find the CSV file) Would be interesting to integrate LangSmith as well to compare answer quality between Mixtral and Llama3.

@samwitteveenai 3 ай бұрын

Glad you liked it. There is no csv file, not sure what you are referring to there.

@thefutureisbright 3 ай бұрын

Hi Sam another excellent tutorial. I've posted it in the crewai discord channel. Thanks Paul

@samwitteveenai 3 ай бұрын

Thanks Paul

@quito96 3 ай бұрын

Llama3 is Amazing for sure, but so is Sam. 😀Thx for sharing

@jayhu6075 3 ай бұрын

What an great explanation! Perhaps next time, you could delve into the topic of constructing an internal knowledge retrieval system (RAG system) for information retrieval, offering an alternative to relying solely on web searches?

@tuneshverma2582 2 ай бұрын

awesome video, really helpfull

@tubingphd 3 ай бұрын

Thank you Sam

@sayanosis 3 ай бұрын

Amazing video ❤

@yazanrisheh5127 3 ай бұрын

Hey Sam. Thanks again for the video and I was wondering if next video with the langraph or video after u could show us hw can we do the internal rag as you mentioned for production lvl apps

@flavorbot 3 ай бұрын

great tutorial thanks a lot

@BradleyKieser 3 ай бұрын

Great video, added points for clever references to The Beatles.

@samwitteveenai 3 ай бұрын

Ringo thanks you very much.

@ravipratapmishra7013 3 ай бұрын

Videos is great as always, but this time the thumbnail is awesome.

@samwitteveenai 3 ай бұрын

truth be told the thumbnail is what convinced me to make the video. 😀

@helix8847 3 ай бұрын

Llama3 is Amazing. I have replaced so many tasks that I used to use ChatGPT for with Llama3 70B.

@samwitteveenai 3 ай бұрын

totally agree 3.5 is looking pretty old now

@Aidev7876 3 ай бұрын

Good video. Can you maybe address sql based RAG with llama3 and crewAI? Such as a recommendation system for a product on an SQL inventory with maybe fall back as internet search?

@bartoszludera2604 3 ай бұрын

Thanks Sam, but there is any option to connect this python code with real gmail or other inbox?

@humzaahmed5608 2 ай бұрын

Thanks for all your uploads Sam your explanations are always amazing! I was wondering if you provide consulting sessions or advice on other AI projects as well, I've got a conversational AI agent that I've been trying to build up for a specific use case in valuations that I would love to talk to you about.

@samwitteveenai 2 ай бұрын

Thanks for the kind words. Best to contact me on Linkedin for any consulting etc.

@drlordbasil 3 ай бұрын

Never really use crewai, but I have an email assistant like this with 2 brains(still upgrading), 1 for tools for researching/leaving notes/reading notes/ect and then another for the response to the email after reviewing the returns from the tool agent. Replies directly to the clients and has RAG via ollama! Although it's all python. I love this type of agentic workflow. I really should learn more about crewAI but Iunno, seems annoying to me still for some odd reason haha. Any other tips that werent in the vid for coders that are hesitant to use crewai?

@Derick99 3 ай бұрын

How can we get it to make a complex wordpress plugins with multiple files without losing progress along the way as it just forgets stuff and leaves placeholders causing us to just do circles if its to complex

@cnclubmember23 3 ай бұрын

Great but is it possible to allow internet search in ollama web UI?

@DigitalBrent 2 ай бұрын

First off, great video - but I have a question and I realize this is a noob question so sorry 'bout it. In the part right around 4:00 you say, "okay here we're setting up our groq api key..." But I'm confused where to put this key. I have the api key already but I can't find where you put the API in the code. Does it go in the os.environ or does it go in the userdata.get? Or both? Or neither? Thanks so much for the video again. If you could help me figure out this part that's the only thing I'm confused about. Thanks.

@yellowboat8773 3 ай бұрын

Anychance of showing how to integrate langchain tools into crewai? Specifically gptresearcher?

@74Gee 3 ай бұрын

This is very impressive indeed, thanks for sharing! I suspect the upcoming Ollama 7b version might not be quite as accurate but this gave me an idea. I was thinking it might work to generate 2 replies for each of say 50 emails from a huge AI, and use the manually chosen replies as examples for the smaller models. It feels like cheating but I think it might give the smaller models an extra accuracy boost they might need.

@samwitteveenai 3 ай бұрын

yeah creating good ICL exemplars works really well. I have done a project recently that makes good use of this with the Haiku model.

@74Gee 3 ай бұрын

@@samwitteveenai Makes sense it wasn't an original idea! Thanks for classifying it for me, now I can look into it further :)

@SaikatDeshmukh 3 ай бұрын

How did you create the thumbnail?

@drlordbasil 3 ай бұрын

damnit I was busy coding >,< lol brb watching

@samwitteveenai 3 ай бұрын

i know the feeling.

@eyadaiman1559 3 ай бұрын

Sam is CrewAI is production ready? it causes a lot of internal server errors in production

@samwitteveenai 3 ай бұрын

I would say it isn't production ready currently. I use it more for trying ideas out quickly and then remaking them in LangGraph or my own little framework

@valentinmarguet575 3 ай бұрын

Nice! Do you know if llama3 powered by groq is usable with autogen instead of crewai?

@samwitteveenai 3 ай бұрын

I haven't tried it but I think it should be.

@valentinmarguet575 3 ай бұрын

Thank you! It would be nice to have the same kind of tutorial with Autogen. I really appreciate the quality of the work you are providing

@hqcart1 3 ай бұрын

I don't understand why do we need to complicate a simple email reply task with crews and agents??? a simple prompt is sufficient to categorize the email and reply based on the prompt.. someone please explain why the complication??

@samwitteveenai 3 ай бұрын

I don't disagree, this could be done with just a chain of prompts. I tried to keep this simple. Where the Agent elements come into their element more is with multiple decision point that the LLM is choosing the flow path.

@landob6393 Ай бұрын

What's the need for crewAI here, or for similar examples? From my understanding, this could be passed into a simple sequential LLM chain and be much simpler. I'm new to AI agents and LLM applications so bear with me, just a genuine question. Any replies would be awesome!

@samwitteveenai Ай бұрын

Its the decision points and parsing (and acting on) those decisions. In many ways I set this up to compare with the LangGraph example that followed it. The CrewAI framework can also be used to decide the next steps itself, though I feel this often isn't reliable.

@ihateorangecat 3 ай бұрын

Agents beginner here 👋 I want to know is there any way that I can use Crewai with javascript??

@samwitteveenai 3 ай бұрын

AFAIK there isn't a version of JS version of CrewAI. You can use LangGraph JS though.

@clray123 3 ай бұрын

And make sure you comply with GDPR when you pass on all that private, confidential, person-related customer information they send you in the email to some external service such as "groq"... You will need to make customer 40 pages beforehand and sign a release before they are allowed to send you email.

@gavinknight8560 2 ай бұрын

And this is why a local LLM makes a lot of sense. The other element is, whatever you do needs to be discoverable. When the law suits start flying, the logging will need to be of high quality.

@clray123 2 ай бұрын

@@gavinknight8560 You can always ask LLM to fake the logs. The reality is that GDPR is only used to extort money and kickbacks from big companies because there is absolutely no way to check compliance (e.g. if I say, I deleted all your data, there is no need - or technical possibility - to prove that I did).

@MattSimmonsSysAdmin 3 ай бұрын

This is a reminder, from a human, to other humans who are using this channel to learn how to implement things like this, to not fully hand your brain to AI. Customer complaint emails might be able to be handled automatically, but the humans running the system can't fix things if they don't know what people are complaining about. Make sure to include some feedback mechanism in what you're doing so that humans can maintain observability of the AI system and the world that the AI system is processing.

@mysticaltech 3 ай бұрын

Yep, super well said. That's how to flywheel of service improvement works, and AI will make it spin way harder.

@zatoichi1 2 ай бұрын

Yet if the humans observe, for example, hallucinations from the AI, there is no way to troubleshoot that due to the black box aspect of high parameter AI. The AI will most likely stop its hallucinations but perhaps after two or three (ten😂?) customers and then the only solution is prompting but there will never be the reliability of programming or finding and fixing broken code.

@zatoichi1 2 ай бұрын

Perhaps the answer is bots watching other bots and fixing their mistakes 😅

@ASchnacky 3 ай бұрын

i just signed up for crew but am waiting in beta, any idea how long?

@samwitteveenai 3 ай бұрын

everything I showed here can be done with just the open source package no need to sign up for anything.

@ASchnacky 3 ай бұрын

@@samwitteveenai oic ty

@alberjumper 2 ай бұрын

This is not an agentic flow, this is just a regular pipeline built with CrewAI.

@landob6393 Ай бұрын

Hey, new to agent space and I was thinking same thing. Can you give me some ideas where frameworks like crewAI would actually be useful and not just an unnecessary layer of abstraction? Current understanding is if you need to implement some sort of loop/cycle into the workflow.

@teprox7690 3 ай бұрын

why the 3rd and 4th have orange sunglases ?

@samwitteveenai 3 ай бұрын

They are the rebels.

@MarcelMilcent 2 ай бұрын

Please, do it with Ollama locally. It would be really nice to have some more multi-agent examples. By the way, as per what you asked in the other video about other languages, llam3 is working ptretty nice in Brazilian Portuguese.Thanks for all!

@samwitteveenai 2 ай бұрын

Interesting to hear it is doing well in Brazilian Portuguese

@pensiveintrovert4318 3 ай бұрын

Llama 3 70b Instruct starts producing junk output once the conversation gets beyond 8k. Pretty unuseable with gpt-pilot, for example.

@alizhadigerov9599 3 ай бұрын

how can you go beyond 8k if context length is maximum 8k?

@pensiveintrovert4318 3 ай бұрын

@@alizhadigerov9599 I guess the only thing to do is employ a sliding window of some kind. Maybe compress old content. There are articles about context size extension. I was using ollama, it may have a problem with how it handles context size.

@choiswimmer 3 ай бұрын

Why are you stuffing all that in there? You can summarize the conversation or do other techniques to manage that. It's just lazy to stuff things in and let the model take care of it

@pensiveintrovert4318 3 ай бұрын

@@choiswimmer when I need to have gpt-pilot agents lectured on being lazy, I am sure to get in touch with you.

@ZacMagee 3 ай бұрын

Bit like the current state of chatGPT

@clray123 3 ай бұрын

419 scammers / scambaiters you better listen up...

@clray123 3 ай бұрын

I suspect that signing machine-generated emails as "Sarah, the resident manager" when there is no "Sarah" is at the very least unethical, and potentially illegal (depending on the context).

@sd5853 3 ай бұрын

I mean when Indian call center guys try to reach out to you do they present themselves as Radesh from Mumbai or as Paul from Missouri ?

@clray123 3 ай бұрын

@@sd5853 You mean Indian scam centers? Yes, scammers usually assume a different identity from their own because it aids their scam. Do you want your company to be perceived as liars and scammers?

@megaimpian986 2 ай бұрын

fuck my brain i cant understand!!

@user-ue9bi2ui2q 2 ай бұрын

There are some issues when you do stuff like this in real life : 1. Groq is not that actually fast when you use it with crew ai 2. In every use case I have tried , just using code is faster and more accurate than using a team of agents . 3. Most LLMs can’t even consistently format the Agent messages properly , resulting in massive waste of tokens as wrongly formatted messages repeat over and over . This makes the utility of this method for writing reliable production code very limited right now. That is the REALITY I am seeing with dirty hands . Please share your own thoughts or experiences with me!

@samwitteveenai 2 ай бұрын

I wouldn't use CrewAI for production at all currently. It is like a idea testing tool/toy. It makes the trade off to get fast and easy creation of agents by giving up full control, custom checking, validations etc.

@user-ue9bi2ui2q 2 ай бұрын

@@samwitteveenai OK , thanks for the reply ! Ya makes sense 👍