Playing with Llama 3 on Ollama and it is bonkers good

thank you for the break down it will help us to create our own specialized agents

Impressive, very nice. And speed of this is insane.

Great video I am super excited about being able to use function calling with a smaller model.

Thank you for sharing! Awesome! 🔝🔝🔝

Great video! Thank you very much for the explanation.

Useful information. Thanks!👍👍👍

Something that is not clear for me is why we should use the open ia API in the agent, was it not possible to only run the entire model locally ?

I love the content. I have an idea for a video and this is something a lot of people could use. I’m terrible at taking meeting notes. A local model that could listen using the laptop mic and summarize in person meetings would be amazing.

one of the best videos ive seen on this topic maybe the best! keep it up broski!

Thank you 🙏 great breakdown

Is the model fine tuned in anyway to use the function calling convention (tags + json), or is it solely based off the system prompt? If the later, I am mighty impressed for a 8B model.

Amazing content. What of your videos can I watch prior to these steps to understand how to get Ollama?. new Argy sub.

That's amazing! What is the difference between using LangChain and not using it? Thanks!

Great video😮😮

Nice video!

How does it compare to the likes of Dolphin-Mistral which has the bigger context window/code training?

Thanks for putting this together! What would you recommend as a good budget/starter setup for training Llama 3 8B? a RTX 3070 or better? a decent 8-core cpu like 13600kf ? 64gb ram i assume? I'm not in any sort of rush so I'd prefer not to rent the processing and purchase my own equipment over time.

I cant host 70b locally or even 8b, i need to use an llama3 hosted api with agents that crawl many urls, can you suggest a setup that would work for that

This is amazing, thanks for sharing.

Where is the code for this Vid?

Awesome. Are you sharing the github repo anywhere?

I've just joined the community. I'd like to use LM Studio instead of ollama as ollama doesn't like windows. how can I adapt the local rag with LM studio, and I would like to create some agents. Thank you.

what quant you using?

Hi, I just joined, how can I get the git for this code?

Mr. Ai, please create prompt installation

Thanks for your video! Any background music is very annoying in tutorial videos.

Do you know hardware requirements for 70b model (locally)?

Have tested llama 3 70b Instruct with gpt-pilot. It does not work. The model is too stupid to work like GPT-4-TURBO-PREVIEW.

How is Llama 3 compared with Mistral. Isn't this a risk to be dependent on Facebook one more time ?

When I use llama 3 8B on ollama or LM Studio, it is much dumber than on OpenRouter. Even after resetting all parameters to factory and loading the llama 3 preset. Even with the full non-quantized 8-bit version on LM studio.

The word "so" was used 426 times in this video! 25 "so" per minute! So, its look like Chris very loves the word "so". Hahaha)))

cant help thinking if you could do a video to run Microsoft UFO on local computer using an open-source model with vision? That will be quite promising and one step closer to AI PC

Use a hacker thief's AI that has no inventor as an "agent" slave to do your bidding... what could possibly go wrong.

Ricky Bobby.

Great video, how to download the code?

🤖 (00:00) **Intro to Llama 3 8B and its capabilities** 🧠 - Demonstrates the agent's ability to search Google, extract information, and send emails. 📧 - Highlights the impressive instruction-following abilities of the model. 🛠 (02:00) **Code Deep Dive - Setting Up Functions** - Explains the functions used in the system, including search Google, send email, and check context. - Shows how to add custom functions, like write to notes. 📝 🧠 (04:21) **Understanding Function Calls** 🤔 - Breaks down the process of how the AI agent interprets user input and triggers function calls. - Explains the role of secret instruction notes and the parse function call function. 🕵‍♀ (07:18) **Surveillance System** 👀 - Discusses the surveillance part of the system that monitors the AI's output for function call tags. 💻 (11:21) **Showcasing the System** - Demonstrates the system in action by performing tasks such as searching Google for AMA models and extracting information from context. 📝 (13:33) **Adding a Custom Function (Write to Notes)** - Walks through the steps of creating and integrating a new function that writes content to a text file. ✅ **Benefits** ✅ Shows the potential of local AI agents with Llama 3 8B. ✅ Provides a detailed explanation of the code and logic behind the system. ✅ Offers a tutorial on how to set up and customize your own functions. ❌ **Cons** ❌ The video is quite long and technical. ❌ Assumes some prior knowledge of AI and coding concepts. ❌ The explanation can be a bit verbose at times. **Key Takeaways** 🔑 - Llama 3 8B is a powerful language model that can be used to build local AI agents. - The system uses a combination of functions and instructions to perform tasks. - Users can customize the system by adding their own functions. **Summary** The video provides a comprehensive tutorial on building a local AI agent using Llama 3 8B. It covers the code, logic, and functionality of the system, as well as how to add custom functions. While the video is technical and assumes some prior knowledge, it offers valuable insights into the potential of local AI agents. **Verdict: Like 👍** (Despite the cons, the video's educational value and insights make it worth a like.)

This is what always gets me about function calling, unless you intend to make the endpoint available to the public, it would be far easier and faster to just expose the function call. So instead of asking the AI to search, you just call the search command directly. Using AI to figureout what function to call and the needed parameters, is like using a flame thrower to light a candle. It only makes sense if the user doesn't understand function calls.

Any way to get the code in git?

Full tutorial but code is pay walled, reminds me of "open"ai

They sell the idea that these models are extraordinary, with exceptional performance, capable of answering various questions, etc. Then you download the model and when you ask it to perform a ridiculous task, such as passing a set of a few sentences and asking it to return the one that most relates to what you said, then the model either hallucinates or gives a wrong answer. How can you trust something like this to perform everyday tasks? Or how can you give it freedom to execute actions on your operating system? Unfortunately, we are still dependent on large companies with models trained with billion-dollar investments.