Completely LOCAL and OFFLINE RAG (Retrieval Augmented Generation) Using LM Studio and LangChain!

Рет қаралды 6,676

Sacred Scriptorium

4 ай бұрын

This is my implementation of Local RAG for local document querying. Here is the script! github.com/ruddythor/localrag

Пікірлер: 21

@aufkeinsten7883 3 ай бұрын

Hey man, appreciate you sharing this. Quick question; If I enter the prompts not in the command line but in LMStudio, the RAG-Pipeline will not be used, correct? Is there a way to write the prompts in LMStudio using this RAG-Pipeline or would I need to build my own UI (like a Web-UI or a small custom application) if I wanted to write prompts in a bit more user-friendly way?

@scriptoriumscribe 3 ай бұрын

I don’t think there’s a way to use rag with on studio yet unfortunately. But you can join their discord and request it as a feature! I think there is demand for that. Gpt4all has a built in rag solution but it is only okay

@aufkeinsten7883 3 ай бұрын

@@scriptoriumscribe Thanks for the quick reply! Will see whether a self-built RAG-pipeline can be integrated with ollama and open-webUI, will definitely check out gpt4all, appreciate the hint :) Do you know if the gpt4all-rag will be outperformed by something like shown in the video? Do you have an idea what this performance would depend on?

@petec737 4 ай бұрын

Looks great, let us know when you can post the links, would love to give it a test. Out of curiosity, have you tested how much text you can give it as input?

@scriptoriumscribe 4 ай бұрын

done! github.com/ruddythor/localrag

@scriptoriumscribe 4 ай бұрын

also, i haven't tested how much i can give it as input, but that is next on my list to try when i get back around to rag!

@sharduljadhav2269 2 ай бұрын

Great Job man! This cleared alot of my doubts regarding this topic. how can we use this in case of ollama? I want my document to be excel , so i used pandas for it, but it eventually gave me alot of errors. can you take input as an excel file ?

@scriptoriumscribe 2 ай бұрын

I’m on holiday in Asia now and will try to do another video on this when I get back. But yes you should be able to use an excel file, though you would likely change the data loader. Thanks for the idea on the next video!

@mple2836 4 ай бұрын

how do we make it work ? I have lm studio, trying to do something similar to use my obsidan notes as reference for the llm.

@scriptoriumscribe 4 ай бұрын

Hmm. Maybe I’ll put some better readme stuff in the repo. Let me do that tonight. Thanks for the comment!

@mple2836 4 ай бұрын

@@scriptoriumscribe thks man

@LIGTH-BIT Ай бұрын

Is it necessary to use the call you use?

@Alpha1200 2 ай бұрын

Genuine question because I have absolutely no idea what I'm doing: I've downloaded LM Studio. I downloaded LLama2 13B Chat. I have a very large document (300.000 words) that I would like to use this RAG for. What do I do now, exactly? Like you linked the github, but what do I do with it? Do I just copy-paste the script to somewhere? To where?

@scriptoriumscribe Ай бұрын

oh no, i've just now seen this! let me see if i can make some kind of blog post. but there's a few things you'd have to do to get something similar working. i'll see if i can make either a video or document explaining how to do this better. thanks for the comment! tldr, i think you'd have to download/clone/copy the script, then get your python environment set up for your computer, then install the python dependencies (pip install), then update the directory to point toward the correct directory. also worth noting you'd have to download LM studio and a couple ai models too. does this help some?

@lovelynxo54 13 күн бұрын

@@scriptoriumscribe did u ever make a post for this? I'm a beginner and lost on how to set everything up as well? I appreciate it.