Unlimited AI Agents running locally with Ollama & AnythingLLM

Рет қаралды 47,579

Күн бұрын

Hey everyone,
Recently in AnythingLLM Desktop, we merged in AI Agents. AI Agents are basically LLMs that do something instead of just replying. We support both tool-call-enabled models like OpenAI but have even now have a no-code way to bring AI agents to every open-source LLMs like with Ollama or LMStudio.
Now, with no code required, you can take any LLM and get automatic web scraping, web-browsing, chart generation, RAG memory, and summarization all autonomously and running locally.
If the future of AI is agents, AnythingLLM is where it is going to happen!.
Download AnythingLLM: useanything.com/download
Star on Github: github.com/Mintplex-Labs/anyt...
Chapters:
0:00 Introduction to adding agents to Ollama
0:45 What is Ollama?
1:08 What is LLM Quantization?
1:28 What is an AI Agent?
2:54 How to pick the right LLM on Ollama
5:11 Pulling Ollama models and running the server
5:45 Downloading AnythingLLM Desktop
6:17 AnythingLLM - Initial setup
7:21 Sending our first chat - no RAG
8:22 Uploading a document privately
8:43 Sending a chat again but with RAG
9:10 How to add agent capabilities to Ollama
10:45 Add live web-searching to Ollama LLMs (Free)
11:41 Using agents in AnythingLLM demonstration
13:24 Agent document summarization and long-term memory
14:35 Why you should use AnythingLLM x Ollama
15:00 Star on Github, please!
15:06 Thank you

Пікірлер: 232

@sergiofigueiredo1987 22 күн бұрын

@TimCarambat I had to pause the video just to leave a comment! I'm deeply impressed by the excellence and simplicity of the content presented here. It's truly remarkable to have access to such tools, created by a team that clearly demonstrates passion and a keen ear for what we all think and wish would be great to have, and at every update, distilling all p of these wishes into a few simple clicks within this amazing piece of technology! I'm immensely grateful for the opportunity to experienceh the brilliance of software engineering and development of Anything LLM, especially within the context of open-source communities. Participating in the advancement of genuine and incredible open tools is a privilege. Thank you Tim! I will be promoting this project to the moon and back, because this deserves to be known.

@TimCarambat 22 күн бұрын

This is so incredibly kind. Sharing with team!

@THOOOMEME 8 күн бұрын

haha I was just about to leave a comment when I read yours. I feel the same. What a champion Tim is. I do not know if I will ever install AnythingLLM but I think I will donate to Tim regardless.

@ts757arse 5 күн бұрын

Aye, I was interested in anythingLLM a while back but chose another project for my inference server. I've found getting half decent agent capabilities to be a huge time sink for someone with my skill set (I'm a physical security guy, not a programmer) and the results just weren't worth the time invested. Even basic agent capabilities with RAG, memory and so on in a package that I can just plug into ollama sounds awesome. Prepping the server now. Here's hoping.

@surfkid1111 22 күн бұрын

You built an amazing piece of software. Thank god that I stumbled across this video.

@flusyrom 9 сағат бұрын

Funny ! I heard yesterday for the first time about AnythingLLM during an AI-info event.... and discarded the idea of giving it more attention because it was presented as "just another local RAG support". And now I stumble across this video by chance - and the additional agent functionality changes everything ! BTW, very well presented , this feature ! My immediate idea & feedback: if there was ANY chance to model custom agents in Flowise and re-import the JSON exports of this Flowise flow as input for an AnythingLLM custom agent, you'd save yourself the trouble of designing your own agent editor AND would start with a comparably large installed base. OK, maybe that's just wishful thinking..... but maybe I'm also not the only one with this wish to facilitate local agent building ;-)

@spacetimepotato 2 сағат бұрын

There were some concepts I didn't quite understand; for example, tunneling from the Windows PC to the Mac (if it's on your local network, why work with VPN protocols rather than client/server - due to needing a stateful connection vs. 200 response code or something?). But the interface itself is brilliant! And I think that when it becomes agent-swarm-capable it's going to be a much better option for me than Crew AI, as it feels more intuitive, I am just going to need multiple agents working together. I have never installed a local LLM, but you have inspired me to give it a try. Thanks!

@fxstation1329 13 күн бұрын

What I love about your tutorials is that you succinctly explain all the things that come across during the tutorial. Thanks!

@SiliconSouthShow 23 күн бұрын

Fantastic Tim! Mine doesnt have agent config, guess i need to delete and udate, ill try that, looks great! keep up good work, i love anythingllm i really do!

@liviuspinu11 19 күн бұрын

Thank you for explaining quantisation in details for niebiews.

@akikuro1725 23 күн бұрын

Awesome! thank you for this. looking forward to more information/details/examples on using agents w/AnythingLLM!

@TheDrMusician 22 күн бұрын

This is by far the easiest and most powerful way to use LLMs locally, full support, like and sub. And many thanks for the amazing work, especially being open source.

@TimCarambat 22 күн бұрын

🫡

@kangoclap 14 күн бұрын

looking forward to utilizing AnythingLLM, it looks really awesome! congrats on creating such an impressive application! thank you!

@SiliconSouthShow 23 күн бұрын

@TimCarambat I'm excited to see the features you talked about work with the ollama like in the video for the agent, as of now, its same as before I updated, but it's exciting to think of the future.

@ilanlee3025 2 күн бұрын

Good stuff, will try it out. Subscribed. Looking forwards to seeing how this develops.

@jonathan58475 11 күн бұрын

Tim, thank you for making the world a better place with this awesome tool! :)

@quinnlintott406 8 күн бұрын

I had no idea you had a channel talking about your software. Im a big fan of your work!

@jimg8296 21 күн бұрын

Anythingllm is awesome. Glad to hear custom agents are on the roadmap. It's the big hole in capability. Also need config to change agent promt. I scan a lot of code and the @ is used often to define decorators.

@yasin6904 6 күн бұрын

Im a chronic video skipper but watched this back to back. Great explanations and can't wait to try this out! Would love to see more videos, tutorials or even lectures from you. You really have a knack for explaining things!😊

@yasin6904 5 күн бұрын

PS I've starred on Github!

@yusufaliyu9759 23 күн бұрын

Great this will make LLM more understandable for many ppl.

@MartinBlaha 22 күн бұрын

Thank you! Will test it for sure. I think you guys are on the exact right path 😎👍

@SamBeera Күн бұрын

Hi Tim, thank you very much for the great video showcasing open source llms, and tools like anythingllm to create agents. I followed your video and successfully was able to do everything in your video. Are there other agentic videos for other usecases you made, look forward to see them. Cheers

@OpenAITutor 12 күн бұрын

Amazing Tim. Keep up the good work.

@michaelklimpel3020 9 күн бұрын

Big thanks man. This video helps alot for me as an beginner to understand how good a local llm is and which Usecases we have. Thumbs up for this great video.

@tunoajohnson256 15 күн бұрын

Awesome vid! Really impressed with how you presented the information. 🙏 thank you

@figs3284 21 күн бұрын

Incredible.. gonna make building tools so much easier. Cant wait to see more agent abilities added!

@stanTrX 22 күн бұрын

This is the easiest all-in-one platform. Thanks. More videos please ❤

@rockon-wbfqlkjqhsydic72683 3 күн бұрын

Great job! This is wonderful! I will be responding after using to let you know my thoughts if you care to see them :)

@johnbramich Күн бұрын

Can't wait to use this. Thank you!

@MaliciousCode-gw5tq 18 күн бұрын

Damm,... finally found the tools that i been looking for..MAN you save my day, i have been crazy stuck finding webui for my ollama remote server..your a gift from heaven keep it up your helping alot of people like us..thank you so much..❤❤❤😂😅😊😊

@mehmetnaciakkk3983 4 күн бұрын

A fantastic beginning! When do you think we willbe able to create our own agents?

@vulcan4d 22 күн бұрын

This is awesome work. I looked at the other simple to install Windows front ends and stumbled on this. Pretty cool stuff and I love how you can add documents and external websites to feed it information. An offline LLM is soooooo much more preferred. The only item I don't understand is why you could just ask a regular question once you provided the document, but used @agent when asking to summarize a document.

@TimCarambat 22 күн бұрын

IMO, i find having a local LLM that even is **only** like 75% as good as on online alternative is just much more rewarding. Like i can be on an airplane, open my laptop, and start brainstorming with an AI. Pretty neat. Next evolution would be a local AI on your phone but i dont think we have that tech _yet_

@jakeparker918 8 күн бұрын

This is so dope. Great no-code solution and it's awesome that it's open source.

@d.d.z. 21 күн бұрын

You are amazing. Thank you 🎉

@zirize 22 күн бұрын

I think it's a very good application, easy to use, and after testing it for a day or so, I have some wishes. 1. direct commands Bypass Agent LLM in Agent mode. It takes time for the agent to understand the sentence and convert it into internal command, and url parsing sometimes fails depending on the agent. For example, a command that scrapes the specified URL and shows the result, or a command that lists the currently registered documents with numbering. And a command that summarizes the document by this number instead of its full name. 2. I wish there was a way to pre-test the settings in the options window to make sure they are correct, such as specifying LLM or search engine. I hope this application is widely known and loved by many people.

@mrinalraj4801 19 күн бұрын

Great work. Thanks a lot 🙏

@FlynnTheRedhead 23 күн бұрын

So training/finetuning is coming up as well? Loving the progress and process updates, keep up the great work Tim!

@TimCarambat 23 күн бұрын

how'd you know!? We will likely make some kind of external supplemental process for fine-tuning, but at least make the tuning process easy to integrate with AnythingLLM. RAG + Fine-tune + agents = very powerful without question

@FlynnTheRedhead 22 күн бұрын

@@TimCarambat That's awesome to hear!! I created an agent to get insider info, that's how I know of course!

@TimCarambat 22 күн бұрын

@@FlynnTheRedhead !!!!! I thought i was hearing clicks during my phone calls!!!

@sashkovarha 23 күн бұрын

This explained the rag and agents parts I couldn't set up. Great educational content for those who are not programmers. Appreciate your explanations being without that much of "pre-supposed" know-how, that coders have - which is most tutorials on youtube... I still didn't get why there's a difference between @agent commands and just regular chat

@TimCarambat 23 күн бұрын

In a perfect world, they are the same. AnythingLLM originally was only rag. In the near future @agent won't be needed and agent commands will work seamlessly in the chat. So @agent is temporary for now so you know for sure you want to possibly use some kind of tool for your prompt. Otherwise, it's just simple rag

@gillopez8660 22 күн бұрын

Wow this is amazing... I'm gonna go star you!

@star95 18 күн бұрын

Great video! I also want to know how well the RAG function of AnythingLLM performs. It's important that text, images, and papers are handled properly and meaningful chunking are achieved

@JacquesvanWyk 11 күн бұрын

Really awesome demonstration. I am excited about agents. Would be nice to be able to build custom tools in python for agents to use.

@TokyoNeko8 20 күн бұрын

Debug mode would be ideal. Agent to scrape the web just exits without any error even though I do have search engine api defined

@CotisoHanganu 14 күн бұрын

Great things shown. Tx for all the work and commitment. 🎉 Here is a kind of dedicated use case I am interested to get acces: I am a mind mapping addict. I use Mind Manager, that stores the mm in .mmap format. I would like to ask ANYTHINGLLM to help me scan all folders for mind maps on different subjects and Rag & summarize on them, without having to export all mmap files in another format. Is this doable at this stage? What else should have or have created?

@aimademerich 23 күн бұрын

Would love to see this run stable diffusion and comfy ui workflows

@red_onex--x808 20 күн бұрын

Awesome info……thx

@GoranMarkovic85 10 күн бұрын

Amazing work 👏

@johnbrewer1430 12 күн бұрын

@sergiofigueiredo1987, @TimCarambat, I agree with Sergio. Wow! I have Ollama installed locally on a Windows machine in WSL. (I was leery of the Windows preview, but I may switch because NATing the Docker container is a pain.) I also pondered how to build a vector DB on my machine and integrate agents. You guys have already done it!

@madhudson1 22 күн бұрын

Been struggling to get custom agents to integrate reliably with external tooling, using frameworks like crewui with local LLMs. Would love a video guide explaining best practices for this

@EddieAdolf 16 күн бұрын

I've been using it for months. Love it! Will you enable voice to voice soon?

@TimCarambat 14 күн бұрын

We just did in our most recent update. TTS is live for all, STT is only live for the docker version. There are some restrictions and limitations we need to work around to get STT to fully function cross-platform. It will be solved soon

@sharankumar31 12 күн бұрын

this is seriously very neat tool👏👏👏 Pls add some feature to custom develop agents with function calls. It will be helpful for our local automations.

@TimCarambat 12 күн бұрын

This is shown in the UI that we will be supporting custom agents soon!

@finessejones3109 11 күн бұрын

I'm so happy I came across your video. Thank you. I am having trouble on where you to get the base link that you pasted in @6:36 mark to install the ollama3

@finessejones3109 11 күн бұрын

I was able to follow along from your other video to install it. Thank you I'm now a new sub.

@themax2go 16 күн бұрын

very cool!!! subbed!

@user-mz2ei2nx2p 5 күн бұрын

Great video! however, i followed every step you described in every detail, but i could not make the agents communicate with outside world. in any ''search'' or 'webscrape'' request, the model is hallusinating, and presents data that are already to its knowledge insted of real time data (i.e. current gold price ). i used llama3 Q8, i inserted google api and id code, i also tried the other search engine.. nothing. the logs show that it really creates json commands, but nothing comes in from the internet.... any help ?

@DaveEtchells 14 күн бұрын

Wow, this looks *_amazing!_* I’m just starting to experiment with local LLMs and wanting to play with agents; this looks SO easy! I’m going to download and set it up right away. I’m also interested in Open Interpreter for having an AI assistant do things on my local machine. Can this interface with that, or is it really meant as a substitute/enhancement to it? (Also, how can I support your project? I gather your biz model is selling the cloud service, but my usage will be purely local. Anywhere I could send a token few bucks?)

@vishalchouhan07 15 күн бұрын

Hi Tim.. I am absolutely impressed with the capabilities of AnythingLLM. Just a small query..how can I deploy it on a cloud machine and serve it as a chat agent on my website? I actually want to add few learning resources as pdf for the rag document of this llm so that my users can chat with the content of those pdfs on my website. I also want to understand how many such parallel instances of similar scenario but with different set of pdf is possible? For instance, if I am selling ebooks as digital product to my users, can I have unique instances autogenerated for each user based on their purchase?

@TimCarambat 14 күн бұрын

We offer a standalone docker image that is a multi-user version of the desktop app. It has a public chat embed that is basically a publicly accessible workspace chat window. You can deploy a lot of places depending on what you want to accomplish: github.com/Mintplex-Labs/anything-llm?tab=readme-ov-file#-self-hosting For this, you could do one AnythingLLM instance, multiple workspace where each has its own set of documents, and then a chat widget for each. This would give you the end result you are looking for

@marinetradeapp 11 күн бұрын

Great work - thanks for sharing - Question - how can we send data to the agent via webhooks - is this a possibility?

@emil8367 13 күн бұрын

Many thanks for nice introduction ! Is there a way to configure this LanceDB ? Is there a doc how it's integrated with the AnythingLLM ?

@TimCarambat 13 күн бұрын

There is nothing to configure, it is preinstalled and saves to the same location as the application's main storage folder!

@UrbanCha0s 22 күн бұрын

Looks really good and simple. I tried PrivateGPT using conda/Poetry and could never get it to work, so jumped into WSL for Windows connecting to Ubuntu running ollama, via WEBUI. Works great, but this just looks so much easier. Will have to give it a try. What I do like with the WEBUI I have is I can select different model, and even use multiple models at the same time.

@TimCarambat 22 күн бұрын

Yeah, we didnt want to "rebuild" what is already built and amazing like text-web-gen. No reason why we cant wrap around your existing efforts on those tools and just elevate that experience with additional tools like RAG, agents, etc

@mouradlaraba 8 күн бұрын

thanks a lot for your video, this the first video that i see and it's really simple to understand, i have a question, if for example the model that i use know that the capital of france is paris, how can i change that information and make the answer different from paris? best regards

@Alex29196 19 күн бұрын

Hi Tim, thank you for your dedication and effort in teaching us about local LLMs. I have a medium-spec computer with 4GB VRAM and 16GB RAM. The last time I installed ALLM, the inference speed was a bit slower compared to other alternatives. How does it perform with the new version? Thanks again.

@TimCarambat 14 күн бұрын

Unfortunately, i doubt much would change on the inference side. When you say alternatives, what were you using? You might get slower responses in AnythingLLM vs just chatting via CLI in ollama, but that is because we are adding that valuable context to the prompt. More tokens = more work on the LLM to respond!

@ImSlo7yHD 11 күн бұрын

This is perfect it just needs more tools and agent customization like crew ai and it is going to be an absolute killer for the ai industry.

@TimCarambat 11 күн бұрын

Will be coming soon! Just carving out how agents should work within the context of AnythingLLM and should be good. Also, it would be nice to be able to just import your current CrewAI and use it in AnythingLLM - save you the work you have done so far

@betterlifeexe4378 9 күн бұрын

I know it's a huge ask, but it would be great if it could listen to a inputs and active windows. it could be really cool if it could capture and describe my workflow, i could analyze what i am doing, and than generate macros for me.

@carloscms23 21 күн бұрын

Great Work :)

@Great_Muzik 14 күн бұрын

Awesome tutorial Tim! Can this extract specific data from PDF files and save it to an Excel file?

@marius2591 9 күн бұрын

Hi, How does quantization type affects the system resources needed to properly run that model? Great video by the way!

@TimCarambat 9 күн бұрын

It mostly impacts the RAM and overall storage side of the GGUF modelfile. It's tricky to determine the exact requirement decrease because it has to do with the specific model parameters and other factors. Im not aware of a simple equation or expression that is a direct calculation for all models. In general, lower quant -> Smaller file size and memory footprint when loaded, but much worse output performance

@rogerunderhill4267 7 күн бұрын

Brilliant! Could it use my own computer as a data source for the agents? Can I scrape my mac?

@redbaron3555 20 күн бұрын

Amazing software!! Congratulations and thank you! Very similar to MemGPT server but seems easier to set up and use. I wonder whether you can save a whole company database (i.e. ERP data: products, materials etc.) in it and being g able to ask questions about it? Also can you instigate more than one agent simultaneously?

@TimCarambat 20 күн бұрын

In theory, this would be better delegated by some purpose-built agent that can traverse the data. Currently, we only have one-agent conversations but the code _does_ support multi-agent. We just find it to be really messy and cumbersome when many agents are once are trying to do something and your Ollama instance is already at max use generating tokens!

@Oliver-zy8sq 5 күн бұрын

Hey, thank you for putting out anytingllm. I have two questions: 1. When I ask the llm to remember something, is that long term memory stored on my pc on a server? 2. is the summary part of the long term memory necessary? And I have a feature request for an automatic long term memory. Meaning that I don't have to say specifically what to remember but that the llm will be able to recall the entire chat history - eveything i have ever said in that thread. Is that in the picture?

@mrgyani 15 күн бұрын

This is incredible..

@SiliconSouthShow 23 күн бұрын

@TimCarambat Hey Tim it wont let me select anything under Workspace Agent LLM Provider even though everything is setup and working, obviously ollama is running and everything else in anything is using ollama fine in the app, but this selection option doesn't show like yours does.

@elu1 16 күн бұрын

really nice!

@DanRegalia 12 күн бұрын

Hey, just found you on a random youtube video suggestion. Love this concept.. A few questions, how deep into a website can this scrape? Can it read a sitemap or robots.txt and download all the data, summarize, etc? Can I hook it into different LLMs? For instance, assign agents to different LLMs? Most importantly, if we're using a vector database, can I feed it rows and rows of data to remember forever?

@TimCarambat 11 күн бұрын

The one in the document uploader is a single site, but we have a deep website scraper as you mentioned. You can use a different LLM per workspace and also per workspace-agent. So yes. The vector database we use runs locally and is built in. It works like any other and yes does persist information - so yes to the last point as well

@foxnyoki5727 20 күн бұрын

Does Internet Search Work for You ? I configured the agent to use Google Custom Search Engine but search does not return any results.

@TimCarambat 20 күн бұрын

With some models you _might_ have to word a prompt more directly. Like even explicitly asking it to call `web-browsing` and run this search. Which i know breaks the "fluidity" of conversation, but this is just a facet of the non-determinisic non-steerable nature of LLMs and trying to get them to listen. Mostly, its the model that needs to be better so it can follow prompts more closely, but its also not always that simple!

@HarpaAI 10 күн бұрын

🎯 Key Takeaways for quick navigation: 00:00 *🤖 Introduction to Ollama & AnythingLLM and AMA* - Introduction to Ollama and AnythingLLM - Explanation of AMA application for running LLMs on local devices - Overview of quantization process and agent capabilities in LLMs 02:30 *🧠 Understanding Model Quantization and Selection* - Importance of selecting the right quantization level for LLMs - Differences between various quantization levels like Q1 and Q8 - How quantization impacts model performance and reliability 06:07 *🛠 Setting up AnythingLM with Q8 Model and AMA* - Instructions for setting up AMA with Q8 LLW model - Steps to download and run AnythingLM on local devices - Connecting to AMA server and configuring privacy settings 08:27 *💬 Enhancing Model Knowledge Using RAG and Workspace* - Uploading documents for model referencing in workspace - Improving model responses by utilizing documents in the workspace - Configuring workspace settings for better model performance 11:41 *🌐 Using Agents for Advanced Functionality in AnythingLLM* - Utilizing agents to enhance LLMs capabilities beyond basic text responses - Enabling web scraping, file generation, summarization, and memory functions - Integrating external services like Google for web browsing functionalities Made with HARPA AI

@LakerTriangle 23 күн бұрын

Literally sitting here wondering this when you dropped the video

@jackiekerouac2090 2 күн бұрын

@Tim: I am a professional translator (English to French), and I've just discovered AnythingLLM. Sometimes I have to translate confidential documents that cannot be shared on the cloud. They need to remain locally on my own computer. Once the translation is done, they have to be encrypted to be sent to clients. Could I use AnythingLLM to help me with the translation process? Could I use it with my actual Lexicum, glossaries and personal dictionaries? Most are PDF or DOCX files. How would I do that? What are the first steps? Many thanks if you can give me some hints on how to proceed. I'm now a new subscriber! 😊

@SiliconSouthShow 23 күн бұрын

wOOHOO I GOT IT NOW! ID LOVE A UPDATE BUTTON LOL!

@TimCarambat 23 күн бұрын

It probably just was not refreshed yet. I think we have it on a 1 hour expiration to check so it may have been in between checks

@caleb.miller 17 күн бұрын

Thanks for the tutorial Tim. For some reason I am not able to get web search working. I am using the same setting you showed in the video. Can you do another video with more detail on setting up the google search engine for this purpose?

@ChristianIsai 15 күн бұрын

I have the same issues, the agent will answer that it doesn't need to use any function and will answer its alucinarions, if I give it the direct order to scrape it will trow a lack of openid key, I think is a work in progress still

@TimCarambat 14 күн бұрын

Is the model just refusing to call the tool at all or when it does call the tool it says it failed?

@ChristianIsai 14 күн бұрын

@@TimCarambat the model will tell me no need for using any tool I got this and then hallucinate

@SebastianMuller-pz9xl 22 күн бұрын

Amazing ⭐⭐⭐⭐⭐

@aimademerich 23 күн бұрын

Phenomenal

@user-ld8sy9xu2v 18 күн бұрын

Hey Tim,what is actual folder that Anything LLM use to store models? I have all models downloaded using it on other apps so i would rather just put the model in the right folder then download it again. Thanks in Advance!

@TimCarambat 14 күн бұрын

on Mac: /Library/Application Support/anythingllm-desktop/storage/models On window: /Users/user/AppData/Roaming/anythingllm-desktop/storage/models

@user-ld8sy9xu2v 14 күн бұрын

@@TimCarambat thanks!

@AndyBerman 23 күн бұрын

@TimCarambat Can this run on an old slow server and connect to ollama on a fast server, or does AnythingLLM use a lot of local CPU when invoked?

@TimCarambat 23 күн бұрын

Actually, this is a perfect combination. AnythingLLM using an external LLM and embedder is no more overhead than just running an HTML page - seriously. The only demanding process is if you use the built-in embedder, and that is really only when you are embedding documents. Depending on the size of your documents you could crash the server with the built-in embedder. For reference, our hosted starter tier is 2vCPU and 2GB RAM and we squeak by. If it's more than that, you are golden. The vector database is so lightweight and fast it is legitimately a non-issue.

@Nicola-cc2di 15 күн бұрын

@TimCarambat can you please let me know wich model is anythingLLM using to generate embedding and if it is possible to choose another one? thanks

@TimCarambat 14 күн бұрын

We use the huggingface.co/sentence-transformers/all-MiniLM-L6-v2 by default, 384 dimension

@flb5078 23 күн бұрын

So it works only with Ollama or also with LM Studio which is my LLM provider, as for many people Ollama does not work on windows?

@TimCarambat 23 күн бұрын

I didnt go over every provider in the window, but lmstudio is supported as well and I was going to make a video showcasing that provider because there are many more models to choose from

@amulbhatia-te9jl 10 күн бұрын

Would it be possible to see a vide of setting up your Ollama models on Anything LLM, I followed these instructions but my ollama models never load.

@leninmariyajoseph352 18 күн бұрын

Great!!!...

@septemberstranger 19 күн бұрын

Hello! Thanks for uploading this...very helpful. I'm stuck on something though. When I try to setup agents for Ollama, it says that agents only work with OpenAI currently. When I try to scrape sites like you do in the video using Ollama, the AI tells me that it can't. Am I missing something?

@gammingtoch259 16 күн бұрын

I have the same issue, but i am using lmstrudio as backend

@TimCarambat 14 күн бұрын

You are able to use Ollama as you agent correct? If that is the case, are you using a small quantized model? Sometimes models have issues calling tools when they were built for that. Our system we implement works well, but we dont "force" the model to call a tool, it still has to generate a valid response to call it.

@RhythmRiftsDataDreams 21 күн бұрын

What is the chunking method you use to create the vectors? Is there a way that the user can control the method of chunking? Say : Short, Token Size, Semantic, Long etc...

@TimCarambat 20 күн бұрын

We currently use a static recursive chunk splitter. So basically just character counts. You can modify those chunking settings in the settings when you go to "embedder preference". So you can define max length and overlap

@TheShawn2880 20 күн бұрын

Your the best

@pradeepjain2872 19 күн бұрын

Hello. I was just playing with RAG. It seams that the acuracy and results are very poor. I tried with laama 3, wizardlm etc. LLM is unclear of my questions. Is the context windows too short? LLM is giving answeres in a hindsight

@4AlexeyR 9 күн бұрын

Hi, Tim. Great work. I'm trying to use Google. But... it is free for 100 queries per day. How I can control it or limit it? Other options are payable :)

@user-go7xt5jd2b 17 күн бұрын

Hi could you explain how to add the Google Programmatic Access API Key and search engine ID?

@TimCarambat 14 күн бұрын

I see this question a ton, i didnt think it would be so confusing but Googles doc that we link to probably can be a bit confusing for those who are unfamiliar with how Google/GCP works

@davidgalea430 22 күн бұрын

Will not load models in the linux version when I select local Ollama

@wyohost 10 күн бұрын

Just went through this whole setup and for some odd reason It keeps telling me it can't search the internet. I've tried local LLMs and OpenAI API with GPT-4o. Also have both Google Search API and Serper API. Neither seems to be able to 'reach' the internet. What in the heck am I missing? I understand this stuff pretty well and I just can't get it to search the web.

@deylightmedia3266 14 күн бұрын

@TimCarambat sir kindly have a for loop so that multiple agents can talk to each other in a chatroom style conversation

@nagisupercell 13 күн бұрын

Can I edit my question and regenerate the result in AnythingLLM? I use OpenAI GPT-4o api, but I don't find the edit button in AnythingLLM UI.

@sashkovarha 23 күн бұрын

Also, will there be a text to speech and speech to text option?

@TimCarambat 23 күн бұрын

It is a pending issue at this time, yes

@lhxperimental 10 күн бұрын

13:00 Funny how in American English a question is actually a command. An LLM dve;oped by another culture would just say yes but not store it till you command it to.

@lucygelz 20 күн бұрын

Any plans or methods to integrate text to speech or just a way for the output to be spoken to you

@TimCarambat 14 күн бұрын

We just merged in TTS for all platforms for v.1.5.5, STT is live for docker only but we should hopefully have STT live for desktop soon. Just some technical details blocking that

@user-tz1hj8em7e 13 күн бұрын

can you upload a video showing how to embed a chat widget onto a website using the llm ran locally on ollama?

@tonyppe 2 күн бұрын

i tried anything LLM and RAG sort of works but I can never pull anything factually from my uploaded text files which are configuration files. Is this a model issue? I was using Llama3 Q8 via ollama and llm studio.

@ZeerakImran 23 күн бұрын

Hi Tim. Thank you for the video. One small suggestion. Can you please make the dock icon on macos the correct size please. I won't add it to the dock because all other icons are the same size whereas anything-llm's icon is oversized. Thanks.

@TimCarambat 23 күн бұрын

It is the exact dimensions the Apple guidelines specify with a 100px padding for 1024x1024. I literally got the layout from their published figma file!

@TimCarambat 23 күн бұрын

In older versions it was indeed the wrong size, it should be good now as of 1.5.3

@ZeerakImran 12 күн бұрын

@@TimCarambat hi Tim. Sorry for the late response. You're right. After seeing your message, I tried to see if I could get the program to check for an update but I wasn't able to find an option for that. So I deleted the app and downloaded the latest version which does have the icon as the right size in the dock. The icon also looks much nicer now. The app recently also showed a nice indicator for a newer version being available in the top right. That's nice too. I would quite like a white (non-grey) mode or light grey mode but that's not a top priority feature but if the app continues to get developed and all goes well, that would be lovely.

@TimCarambat 12 күн бұрын

@@ZeerakImran Ah this must have been a really old version! The version has been showing in the Ui for update alerts for a while so that explains why the icon was so ugly as well! We will be adding a light mode now. So many haters on the dark mode only, I personally dont get it but who am i to say!