How To Connect Llama3 to CrewAI [Groq + Ollama]

  Рет қаралды 21,832

codewithbrandon

codewithbrandon

Күн бұрын

🤖 Download the Source Code Here:
brandonhancock.io/llama3-crewai
Don't forget to Like and Subscribe if you're a fan of free source code 😉
📆 Need help with CrewAI, join our free Skool Community:
skool.com/ai-developer-accele...
This video is perfect for anyone eager to run Llama3 locally on their computer and in the cloud using Grog. We cover what Llama3 is and how it compares to other LLMs. Additionally, we explore how to connect Llama3 to CrewAI. The majority of the video is spent on building an Instagram posting crew that generates image descriptions and text for Instagram using Llama3. By the end of this tutorial, you'll know how to set up, customize, and use Llama3 to automate tasks and enhance your project's capabilities. Get ready to upgrade your tech skills and make your work with AI more productive and innovative. Start now and see how simple it is to bring the power of Llama3 into CrewAI.
📰 Stay updated with my latest projects and insights:
LinkedIn: / brandon-hancock-ai
Twitter: / bhancock_ai
Resources:
- CrewAI Crash Course - • CrewAI Tutorial: Compl...
- Updated CrewAI Tutorial - • CrewAI Tutorial for Be...
- How To Connect Local LLMs to CrewAI [Ollama, Llama2, Mistral] - • How To Connect Local L...
- ollama: ollama.com/
- Llama 3 ai.meta.com/blog/meta-llama-3/
- Configure LLMs for CrewAI - docs.crewai.com/how-to/LLM-Co...
- Instagram crew example: github.com/joaomdmoura/crewAI...
Timestamps:
00:00 Introduction
00:12 Video Overview
02:44 Llama 3 Overview, Comparison, & Testing
07:06 Setup Llama3 Locally with Ollama
12:05 Crew Overview
13:20 Run CrewAI & Llama 3 Locally with Ollama & Crew Deep Dive
22:18 Run CrewAI & Llama 3 with Groq
27:39 Fix Rate Limiting with Groq
29:27 Results
31:01 Outro

Пікірлер: 51
@CodeSnap01
@CodeSnap01 22 күн бұрын
refereshed after short vacation.. hope to see you frequently
@MariodeFelipe
@MariodeFelipe 22 күн бұрын
The quality is 10/10 thanks mate
@bhancock_ai
@bhancock_ai 18 күн бұрын
Thank you Mario!
@GregPeters1
@GregPeters1 23 күн бұрын
Hey Brandon, welcome back after your vacay!
@bhancock_ai
@bhancock_ai 22 күн бұрын
Feels good to be back! I'm recharged and ready to go!
@tapos999
@tapos999 17 күн бұрын
thanks! Your crewai tutorial are top-of-the-shelf stuff. do you have any crewai proejct with streamlit connected to show output on the ui? thanks
@theBookofIsaiah33ad
@theBookofIsaiah33ad 20 күн бұрын
Man, I do not know how to create and write code but you have made a video and I think I can do this! Bless you my friend!
@bhancock_ai
@bhancock_ai 18 күн бұрын
Thank you! I'm confident you can do it! Let me know if you nee help with anything!
@nathankasa6220
@nathankasa6220 22 күн бұрын
Thanks! Is Claude 3 opus still not supported though? How come?
@Omobilo
@Omobilo 17 күн бұрын
Great stuff. Maybe a silly question, but when it was fetching to read data from remote website (the analysis part), does it read it remotely OR does it capture screenshots & download text to feed into its prompt and then clear this cached data or such local cached data needs to be cleaned eventually? Hope it simply reads remotely without too much data saved locally as I plan to use this approach to analyze many websites without flooding my local storage.
@d.d.z.
@d.d.z. 19 күн бұрын
Friendly commment: You look better with glasses, more professional. Great content.
@bhancock_ai
@bhancock_ai 18 күн бұрын
Hey D! Thanks! I love wearing glasses and hate my contacts so I think I might need to go full glasses mode 🤓
@reidelliot1972
@reidelliot1972 21 күн бұрын
Great content as always! Do you know if it's sustainable to use a single groqcloud API key to host LLM access for a multi-user app? Or would a service like AWS Sagemaker be better for simultaneous users? Cheers!
@am0x01
@am0x01 15 күн бұрын
Appreciate your support (with those contents), the only drawback, was the need to subscribe to get access to a project that isn't yours. 😞
@ag36015
@ag36015 18 күн бұрын
What would you say are the minimum hardware requirements to make it run smoothly?
@thefutureisbright
@thefutureisbright 23 күн бұрын
Brandon excellent tutorial 👍
@bhancock_ai
@bhancock_ai 22 күн бұрын
Thanks man! I really appreciate it!
@clinton2312
@clinton2312 21 күн бұрын
Thank you :)
@protovici1476
@protovici1476 22 күн бұрын
Excellent video! Would be interesting to see these frameworks, but within LightningAI Studios. Also, I saw CrewAI will be having a more golden standard approach to their code structuring in the near future.
@bhancock_ai
@bhancock_ai 22 күн бұрын
Thank you! And you're definitely right about CrewAI moving towards YAML. When CrewAI+ drops, I plan on making a lot more content around this new format for you guys! And, I haven't tried out LightningAI Studio yet so I'll definitely have to try it out this weekend. Thanks for the suggestion!
@protovici1476
@protovici1476 22 күн бұрын
@@bhancock_ai Great! I like the YAML approach. William Falcon that started LightningAI (PyTorch Lightning) likes my posts on LinkedIn as I'm a huge fan in developing with it when I mention them. Will be studying your approach with the latest updates and hopefully with their Studio. Thanks!!
@bennie_pie
@bennie_pie 19 күн бұрын
Thank you for this and for the code.. How does Llama 3 compare to Dolphin-Mistral 2.8 running locally as the more junior agents do you know? Dolphin-Mistral with its extra conversatuon/coding training and bigger 32k context window appeals! Ive had agents go round in circles creating nonsense with other frameworks as they dont remember what they are supposed to do! A big context window defo could bring some benefits! I try and avoid using GPT3.5 or 4 for coding preferring for this reason. Id then like to use Claude 3 Opus with his 200k context window and extra capability for the heavy liftin and oversight!
@Ryan.Youtube
@Ryan.Youtube 22 күн бұрын
This is awesome! 😎
@bhancock_ai
@bhancock_ai 22 күн бұрын
Thanks! 😁
@mikesara7032
@mikesara7032 23 күн бұрын
your awesome, thank you!
@bhancock_ai
@bhancock_ai 23 күн бұрын
Thanks Mike! You're awesome too!
@jarad4621
@jarad4621 21 күн бұрын
Hi Brandon, the groq rate limit is a big issue for my use case, can i use this same method to use another similar hosted llama 3 70b with crewai like openrouter api or can any api be used instead of groq with your method?
@jarad4621
@jarad4621 21 күн бұрын
Oh i see it has to be an api alreadyy supported by langchain correct or it wont work?
@rauljauregi6615
@rauljauregi6615 22 күн бұрын
Nice! 😃
@jarad4621
@jarad4621 17 күн бұрын
Please for the love of god somebody explain to me why we are using Ollama to download local models and then using Groq anyway to run the model in the cloud. Why can't we just skip the ollama part? I beg you i see all the videos using Ollama with Groq and i don't understand the aspect! thank you. Does ollama do something special to make it work better for crewai then a direct Groq connect?
@markdkberry
@markdkberry 11 күн бұрын
perfect run with Groq. I get great speeds on my PC with Local LLAMA3, but nothing I can do stops it throwing errors. I find that before with other projects thgat a lot of local LLMs have odd issues that change each run from failing to run functions as they change the name to just saying they cant pass tool info. Must be something in the local limitations either with Ollama or CrewAi.
@ashwinsveta
@ashwinsveta 13 күн бұрын
I get a error, then I noticed that we need search api, I added that but still get the error pydantic_core._pydantic_core.ValidationError: 1 validation error for Task expected_output Field required [type=missing, input_value={'description': "Analyze ...e business landscapes.)}, input_type=dict]
@ryana2952
@ryana2952 6 күн бұрын
Is there an easy way to build No Code AI Assistants or Agents with Groq? I know zero code
@shuntera
@shuntera 21 күн бұрын
That is using a very old version of CrewAI - if you run it with the current version of CrewAI it fails because of lack of expected_output parameter in the Tasks
@jalapenos12
@jalapenos12 18 күн бұрын
Just curious why VSCode doesn't display file types on Mac. I'm going bonkers trying to figure out what to save the Modelfile as.
@bhancock_ai
@bhancock_ai 18 күн бұрын
Hey! There actually isn't a file type for that file. You can just leave it how it is. Hope that helps!
@jalapenos12
@jalapenos12 13 күн бұрын
@@bhancock_ai Thanks for the quick response. I figured out that ".txt" works for those of us in other operating systems.
@shuntera
@shuntera 22 күн бұрын
With both the Groq 8b and 70b with crew max_rpm set at both 1 or 2 I do get it halting for a while with: [INFO]: Max RPM reached, waiting for next minute to start.
@bhancock_ai
@bhancock_ai 22 күн бұрын
The problem is that Groq is so fast that it ends up processing too many tokens so it ends up hitting a rate limit and failing. To get around that, we have to slow down our crew by setting the max RPM. Feel free to bump it up to get your crew to move faster!
@magnuscarlsson5067
@magnuscarlsson5067 22 күн бұрын
What graphic card do you use on your computer when running local with Ollama?
@krysc4d
@krysc4d 21 күн бұрын
The key is VRAM. I can run smoothly llama 3 70b on RTX3090 hiting about 16GB of VRam (if remember correctly)
@pratyushsrivastava6646
@pratyushsrivastava6646 22 күн бұрын
Hello sir Nice content
@bhancock_ai
@bhancock_ai 22 күн бұрын
Thanks Pratyush! I really appreciate it!
@pratyushsrivastava6646
@pratyushsrivastava6646 22 күн бұрын
@@bhancock_ai how can I connect with you
@deadbody408
@deadbody408 22 күн бұрын
might want to revoke those keys you revealed if you haven't
@ZombieGamerRealm
@ZombieGamerRealm 11 күн бұрын
gyus do u know any way to run crewai and\or llama on gpu? only CPU is soooooooooooooooooooooooo sloooooooooooooow
@ArseniyPotapov
@ArseniyPotapov 7 күн бұрын
llama_cpp (what ollama is based on) or vllm
@miaohf
@miaohf 20 күн бұрын
Very good video demonstration. I noticed that you chose to use serper search in the video. I would like to know the difference between serper and duckduckgo search and how to choose between them. If you know, please introduce it to me. Thank you.
@raghuls1469
@raghuls1469 15 күн бұрын
Hello Brandon thanks for the awesome video, I was trying to do the same setup with crew AI, but I am getting an error while running, I added the error message below Traceback (most recent call last): File "D:\crew_ai\crew.py", line 114, in result = crew.kickoff() ^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\crew.py", line 252, in kickoff result = self._run_sequential_process() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\crew.py", line 293, in _run_sequential_process output = task.execute(context=task_output) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\task.py", line 173, in execute result = self._execute( ^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\task.py", line 182, in _execute result = agent.execute_task( ^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\agent.py", line 207, in execute_task memory = contextual_memory.build_context_for_task(task, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\contextual\contextual_memory.py", line 22, in build_context_for_task context.append(self._fetch_stm_context(query)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\contextual\contextual_memory.py", line 31, in _fetch_stm_context stm_results = self.stm.search(query) ^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\short_term\short_term_memory.py", line 23, in search return self.storage.search(query=query, score_threshold=score_threshold) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\storage ag_storage.py", line 90, in search else self.app.search(query, limit) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\embedchain\embedchain.py", line 635, in search return [{"context": c[0], "metadata": c[1]} for c in self.db.query(**params)] ^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\embedchain\vectordb\chroma.py", line 220, in query result = self.collection.query( ^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\models\Collection.py", line 327, in query valid_query_embeddings = self._embed(input=valid_query_texts) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\models\Collection.py", line 633, in _embed return self._embedding_function(input=input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\types.py", line 193, in __call__ result = call(self, input) ^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\utils\embedding_functions.py", line 188, in __call__ embeddings = self._client.create( ^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai esources\embeddings.py", line 113, in create return self._post( ^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 1232, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 921, in request return self._request( ^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 1012, in _request raise self._make_status_error_from_response(err.response) from None openai.NotFoundError: 404 page not found
How To Connect Local LLMs to CrewAI [Ollama, Llama2, Mistral]
25:07
codewithbrandon
Рет қаралды 52 М.
FOUND MONEY 😱 #shorts
00:31
dednahype
Рет қаралды 5 МЛН
[Vowel]물고기는 물에서 살아야 해🐟🤣Fish have to live in the water #funny
00:53
ISSEI funny story😂😂😂Strange World | Pink with inoCat
00:36
ISSEI / いっせい
Рет қаралды 29 МЛН
I Automated My YouTube Channel With CrewAI [Free Source Code Included]
58:15
CrewAI Tutorial: Complete Crash Course for Beginners
1:05:43
codewithbrandon
Рет қаралды 122 М.
Using Llama3 with Ollama & LlamaIndex | Generative AI Tools
6:19
The Code Cruise
Рет қаралды 1 М.
"okay, but I want Llama 3 for my specific use case" - Here's how
24:20
Unlimited AI Agents running locally with Ollama & AnythingLLM
15:21
Powerful AI Agents using CrewAI, GPT, Groq, Ollama, LLama3
36:42
Python RAG Tutorial (with Local LLMs): AI For Your PDFs
21:33
Creating J.A.R.V.I.S. powered by GROQ and Python
14:47
AI FOR DEVS
Рет қаралды 132 М.
Llama 3 RAG: Create Chat with PDF App using PhiData, Here is how..
7:35
Он Отказался от БЕСПЛАТНОЙ видеокарты
0:40
ЖЕЛЕЗНЫЙ КОРОЛЬ
Рет қаралды 1,9 МЛН