Private Chat with your Documents with Ollama and PrivateGPT | Use Case

Private Chat with your Documents with Ollama and PrivateGPT | Use Case | Easy Set up

Рет қаралды 76,800

Күн бұрын

Important: I forgot to mention in the video . Please delete the db and _cache_ folder before putting in your document. Otherwise it will answer from my sample document.
In this video, we are going to chat with your documents using privateGPT powered by Local LLMs from Ollama.
PrivateGPT: Interact with your documents using the power of GPT, 100% privately, no data leaks
Learn step by step how to set this up on your own system.
I have set up a public github repository for all the Ollama Open Source models.
You can clone the repo and get started with any particular folder for testing out any feature of Ollama.
The code can be accesed from this link:
www.patreon.com/posts/private...
Install Ollama: ollama.ai/
Let’s do this!
Join the AI Revolution!
#ai #privateGPT #ollama #webui #github #python #llm #largelanguagemodels
CHANNEL LINKS:
☕ Buy me a coffee: ko-fi.com/promptengineer
🧛‍♂️ Join my Patreon: / promptengineer975
❤️ Subscribe: / @promptengineer48
💀 GitHub Profile: github.com/PromptEngineer48
🔖 Twitter Profile: / prompt48
🤠Join this channel to get access to perks:
/ @promptengineer48
TIME STAMPS:
0:00 Intro
1:28 Install Ollama
2:23 Test Ollama
3:30 My GitHub Repos
5:21 Starting VS Code Editor
7:16 Setting up Virtual Environment
8:42 Install the requirements
10:18 Files for Chatting
10:58 Ingest the files
11:54 Run privateGPT
14:21 Summarize
15:24 Conclusion
15:40 Join Me
🎁Subscribe to my channel: / @promptengineer48
If you have any questions, comments or suggestions, feel free to comment below.
🔔 Don't forget to hit the bell icon to stay updated on our latest innovations and exciting developments in the world of AI!

Пікірлер: 435

@enceladus96 6 ай бұрын

You've saved me from going down my RAG rabbit hole. The code is extremely detailed, clean, and easy to understand too. God bless.

@PromptEngineer48 6 ай бұрын

Thanks for tuning in.

@lukasknowledge Ай бұрын

i get an error = FileNotFoundError: [WinError 2] The system cannot find the file specified: 'db'

@Paul-gg3cr 7 ай бұрын

I've been looking for this for months. Thank you alot, dude! Subscribed :)

@PromptEngineer48 7 ай бұрын

Thanks for the sub!

@mjackstewart 3 ай бұрын

Dude … This is AMAZING! I was just looking for pushes in the right direction, but this actually does exactly what I was attempting to do! Thank you!

@PromptEngineer48 3 ай бұрын

Glad I could help!

@justhuman9551 7 ай бұрын

Just found out your channel and subbed after watching this video. Very good quality video! Keep up the great content creation! I am impressed with your motivation to answer the questions from your comment section. Not every youtube channel cares about answering subscriber questions and doing content around what people comment , so very good job!

@PromptEngineer48 7 ай бұрын

Thank you. It's my pleasure to be talking with my viewers..

@batzizou Ай бұрын

Good work! I found your video well done!

@sergeaudenaert 3 ай бұрын

Thank you! Very clear

@PromptEngineer48 3 ай бұрын

Welcome

@AbhishekThakur-yh9bd Ай бұрын

This is awesome, but still not bale to chnage the base url for ollama, is there any way i can change that?

@rgm4646 7 ай бұрын

great video!

@PromptEngineer48 7 ай бұрын

Thanks

@salvatorespampinato3788 5 ай бұрын

Thank you very much, great video.

@PromptEngineer48 5 ай бұрын

You are welcome!

@AlexanderDeplov-sd5pg 5 ай бұрын

Great tutorial, one of the best on the web!! Thanks for your time and effort! Upvoted 👍

@PromptEngineer48 5 ай бұрын

Thanks so much.

@SorobanWorld 7 ай бұрын

Amazing!

@PromptEngineer48 7 ай бұрын

Thanks!

@charlesbiggs7735 3 ай бұрын

Awesome effort!! Your code worked right off the bat. Thanks for saving me a LOT of time.

@PromptEngineer48 3 ай бұрын

Thanks

@BeyondTheSide 2 ай бұрын

I will like to say that this works just as well even today. Much thanks to the prompt engineer, you have made my life and others a lot easier.

@PromptEngineer48 2 ай бұрын

Thanks for watching

@BexNoel Ай бұрын

My brain is exploding because I was just using your code repository earlier today and then closed everything out. I'm trying to run it again and I'm now getting a bunch of depreciation LangChain warnings. I still get a response but the response is no longer referencing or citing the documents I placed in the folder. Any idea why this would happen?

@PromptEngineer48 25 күн бұрын

my bad. i should have freezed the dependencies. Fear not. I will create another video with fresh codes.

@dreadmadseen 2 ай бұрын

Success!! Thank you!!

@PromptEngineer48 2 ай бұрын

Welcome

@MA_808 4 ай бұрын

Thanks!

@PromptEngineer48 4 ай бұрын

Thank You so much !!

@davidpe76 8 ай бұрын

Great video, took me a few tries getting Ubuntu configured (using wsl under windows) and updated before it would build the scripts, but I am very impressed. Thanks for all the effort you put into these videos 😁

@PromptEngineer48 8 ай бұрын

🤗 welcome. Trying to bring the best.

@waynesbigw2305 4 ай бұрын

what is "wsl under windows?" I'm running Linux. No Windows here. His instructions in the video don't work for my system at all.

@donniealfonso7100 5 ай бұрын

Followed your instructions here and installed on Raspberry Pi4. Works but of course painfully slow and chip approach 145 degrees F which slows things down as well. But it works and may try on Pi5. Was using a pdf manual for Viking drill press for document. Have to try something with just text.

@PromptEngineer48 5 ай бұрын

Hmm. I see.

@bx1803 3 ай бұрын

I want to try to give it some ability to troubleshoot for me, like conduct pings and traceroutes.

@PromptEngineer48 3 ай бұрын

looks good

@thoamslau14 3 ай бұрын

I'm having trouble activate private1. It says I need to run conda init first but then I still couldn't activate private1

@PromptEngineer48 3 ай бұрын

please install anaconda first. www.anaconda.com/download

@ganeshnayak9459 Ай бұрын

Great Tutorial, Why is it saying loading 235 new documents when it has only one in the source_documents folder. I had 2 in mine and it said 8, wondering why.

@PromptEngineer48 Ай бұрын

It's because of the chunking.. I had put only one document but it was chunked into many pieces.

@MerguVinay 2 ай бұрын

i am facing thiis error conda : The term 'conda' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + conda create -n private1 python=3.11 + ~~~~~ + CategoryInfo : ObjectNotFound: (conda:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException

@PromptEngineer48 2 ай бұрын

docs.anaconda.com/free/anaconda/install/windows/ install the anaconda

@user-nd2zy9so8k 4 ай бұрын

i use "pip install tqdm ",The system prompts me: ModuleNotFoundError: No module named 'tqdm'

@natehedgeman 3 ай бұрын

Make sure you have installed all the frameworks listed in the requirements.txt file. Rewind the video, he explains how to do it all at once. pip install -r requirements.txt If you have done that, make sure you are working in the correct environment. The same environment you installed the python frameworks in. He explains that as well.

@raminderpalsingh123 9 күн бұрын

I get the same error. Everything installed successfully, and I'm in the same environment ... :0)

@darshanpatil1663 3 ай бұрын

I am getting the sqlite3 error of using a unspported version, even the link specified does not solve my error can, I get the error when I run the ingest.py file

@PromptEngineer48 3 ай бұрын

Were u able to solve?

@RealBassPhat 2 ай бұрын

Very interesting, easy to follow. I tested this with a music instrument manual, and it wasn't giving accurate answers at all. Any ideas on how to improve this? It's unusable for this type of document. Makes me wonder how accurate it would be with any content. Thank you!

@PromptEngineer48 2 ай бұрын

That was pretty old stuff. Please watch the recent videos on my channel.

@MatiasFedericoWolters 2 ай бұрын

hi, ask a question. How can I change the model for example to llama3 with model 8b-instruct-q6_K?? please

@PromptEngineer48 2 ай бұрын

go to line 12 of the privateGPT.py file and change the mistral to whatever model your heart desire.

@TheArchitect101 3 ай бұрын

The response speed is slow on MacBook Air

@PromptEngineer48 3 ай бұрын

okay. Try switching the model

@ryanwales9399 3 ай бұрын

Keep getting a error when doing python3 ingest.py says line 8 no module named langchain

@PromptEngineer48 3 ай бұрын

pip install langchain langchain_community

@jameswillis3848 3 ай бұрын

it works on windows now

@PromptEngineer48 3 ай бұрын

Yes.

@jameswillis3848 3 ай бұрын

@@PromptEngineer48 thats really cool, ive only played with dolphin mixtral so far which can be 'pursuaded' to learn pdf's from a web link, but really want to give this a go with some local pdf's that I have on my pc, like nautical almanac and see how well it does at understanding the information in there.

@tier1recon836 8 ай бұрын

Would like to see ollama with openai assistant or similar that can use a file and have assistants do action to the file such as execute code or clean up data etc.

@PromptEngineer48 8 ай бұрын

Got it. Let me do some research.

@odjebiuskokovima 5 ай бұрын

It constantly hallucinated about Napoleon Hill - Think and grow rich!?

@PromptEngineer48 5 ай бұрын

Sorry, need to delete the db and cache folder please.

@r0ntuber 2 ай бұрын

Thanks for doing this: It seems like when one clones the repository, you need to delete everything in the db folder or it will mess up the results of the information you are trying input yourself.

@ibbobud 8 ай бұрын

Now we need a webui

@PromptEngineer48 8 ай бұрын

Okay. On it

@ibbobud 8 ай бұрын

actually, i think it comes with a gradio webui now anyways, i was too quick to respond haha@@PromptEngineer48

@PromptEngineer48 8 ай бұрын

I guess there is no requirement of video from my end now.. 🤣

@ibbobud 8 ай бұрын

Do it ! , might as well a video with the full package :) but don't have to make it a priority.@@PromptEngineer48

@user-xi4mw9rx1l 3 ай бұрын

finally finally windows version of came, because i don't notice it until i this video and rember it.

@PromptEngineer48 3 ай бұрын

yes. bro

@fabriziocasula 8 ай бұрын

wow 🙂

@PromptEngineer48 8 ай бұрын

Thanks

@mohamedsabirudeen9249 4 ай бұрын

If I give ollama pull mistral I'm getting error that says could not connect to ollama app, is it running ? Please give me a solution for it

@PromptEngineer48 4 ай бұрын

simple, before typing ollama pull mistral, write ollama serve..

@mohamedsabirudeen9249 4 ай бұрын

Actually using it in linux platform but still I gave the command ollama serve, I'm getting no GPU detected, Please give solution for this ! @@PromptEngineer48

@JohnDo-ntchaknow Ай бұрын

If my company has a pre-existing Data Dictionary, is there a way to allow Ollama to integrate it so that it better understands the data I am working with?

@PromptEngineer48 Ай бұрын

yes. that could be included technically.

@heinzpeterklein9383 8 ай бұрын

Awesome idea. Now use Streamlit or Flask as GUI and the solution is perfect. Thanks for the inspiration. Questions: 1. which OS are you using? 2. python version? 3. do you rather use CPU or GPU? Would an M3 with 128 GB also be sufficient for a quick training / fine tuning of hugging face models up to 20B? Thanks for the answer in advance. Hp

@PromptEngineer48 8 ай бұрын

1. Mac OS. Pretty basic. Macbook Air M2, 8 GB 2. python version 3.9 ++ version 3. M3 with 128 GB.. Hmm when you say fine tuning it depends on the model. But here is a rough calculation. If you have a 20B parameter model in 32 bits, then you need 20x32/8 = 80 GB GPU. your system should be able to do the fine tuning. Else please go for 4 bit quantization, then the requirement will reduce by 8 times, now requiring only 80/8 = 10 GB of GPU.

@user-pu7qx9lo3u 26 күн бұрын

Thanks for your tutorial. I am trying this on Windows pc. I am in anaconda prompt. I am stuck at this error "Error: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based project". I tried installing installing visual studio build tools but the error still persists. Any idea?

@PromptEngineer48 25 күн бұрын

I believe we need .NET SDK, .NET Framework and other tools using the build installer

@user-pu7qx9lo3u 23 күн бұрын

@@PromptEngineer48 all installed but still wont run.

@oskar6065 3 ай бұрын

How to remove old documents, i have a lot of data in the db

@PromptEngineer48 3 ай бұрын

Delete the cache and the db folder

@oskar6065 3 ай бұрын

@@PromptEngineer48 thank you , and thanks for the guide. Really easy to follow along.

@harishhari605 3 ай бұрын

Could you provide suggestions on how to enhance the conversational capabilities of this bot?

@PromptEngineer48 3 ай бұрын

Prompts and proper chains ⛓️

@frankbradford2869 4 ай бұрын

How do I remove the embedded Think and grow rich pdf file. I say this because when I add other file the query goes back to this pdf file and quotes it

@PromptEngineer48 4 ай бұрын

delete the db and cache folders

@frankbradford2869 4 ай бұрын

Thank you, forgive me for sounding slow. You do mean remove or delete the chroma.sqlite3 file?@@PromptEngineer48

@frankbradford2869 4 ай бұрын

If I delete the cache folder name db , will this effect how the program will ingest the files I supply or will create a new db and then ingest the new files?

@iclonethefirst 5 ай бұрын

Thank you a lot for your effort to create an easier way to get up and running. Could you make a video explaining what all of the parts of "requirements.txt" are needed for? I would like to understand how everything works in detail

@PromptEngineer48 5 ай бұрын

Sure thing!

@_Rithika-xh7hn Ай бұрын

I am getting answers from out of pdf also.How to restrict that to only pdf specific?

@PromptEngineer48 Ай бұрын

Ingest only the pdf.

@_Rithika-xh7hn Ай бұрын

@@PromptEngineer48 even after ingesting the PDFs,I am getting answers for some questions that are not in pdf.Is it because of the already trained model?

@TuralNaghiyev 17 күн бұрын

cannot exit from query ))

@PromptEngineer48 13 күн бұрын

for kill. Control + C if in VS Code editor

@Clammer999 2 ай бұрын

I couldn’t get Conda to work after installing Conda . The installed files are in /opt/miniconda3 but whenever I run Conda, it’s says Command Not Found

@PromptEngineer48 2 ай бұрын

www.anaconda.com/download You need to install anaconda. however, you could use the .venv instead of conda. We just need a virtual environment.

@Clammer999 2 ай бұрын

@@PromptEngineer48ok managed to get Conda working. However when I run the python3 ingest.py, I got an error: line6, in from tqdm import tqdm. ModuleNotFoundError: No module named ‘tqdm’

@Clammer999 2 ай бұрын

Ok made more progress but now stuck in pymupd. Tried installing it but keep getting message Requirement already satisfied: /opt/anaconda3/envs/privategpt/lib/python3.11/site-packages

@PromptEngineer48 2 ай бұрын

pip install tqdm should have worked

@frankbradford2869 4 ай бұрын

This work very well but it has issues ingesting docx, pptx and ods files without a pip python install

@peregrino2002gil 2 ай бұрын

Thanks for the video, but both of python files do not work

@PromptEngineer48 2 ай бұрын

Let me check

@LumpBrady0 13 күн бұрын

Hello, I am trying to follow your youtube video about "Private Chat with your Documents with Ollama and PrivateGPT" but when I type in my query after running the 'python privateGPT.py' part. I get the following error message "ValueError: Ollama call failed with status code 404. Details: model '7560' not found, try pulling it first". I'm not sure what this is talking about as I do the 'Ollama pull mistral' command before running the rest of the code. Any idea how to fix this?

@PromptEngineer48 13 күн бұрын

try this ollama run mistral:latest

@LumpBrady0 12 күн бұрын

@@PromptEngineer48 It let's me run mistral:latest but how will this fix my error above? Do I have to add that to the python code somewhere?

@randomscandinavian6094 4 ай бұрын

I'm getting CondaError: Run 'conda init' before 'conda activate' during my installation. I did try conda init but then it says "no action taken". As usual I can't get a step-by-step tutorial to work.

@PromptEngineer48 4 ай бұрын

So you were able to create a conda environment?? Using conda create -n your-name python=3.11??

@randomscandinavian6094 4 ай бұрын

Yes. Followed everything up until the activation part

@randomscandinavian6094 4 ай бұрын

Although I don't get the (base) in front of my path like you did after Preparing transaction: done Verifying transaction: done Executing transaction: done

@BetterEveryDay947 2 ай бұрын

Can you tell, how to use other models like llama3, without using mistral?

@harishhari605 2 ай бұрын

Hi, can you create a video on how to clean our own data which is in my CSV file which is best to answer our query very effectively?

@PromptEngineer48 2 ай бұрын

Yes I can. but to be more clear.. you want to use a data cleaner llm which will give clean you csv file?

@harishhari605 2 ай бұрын

@@PromptEngineer48 Okay pls go ahead

@pankajagarwal1980 3 ай бұрын

Well explained. Can you suggest if we want to pass a onenote how we can pass it.

@evelbsstudio 3 ай бұрын

How do you turn sources off? Just get the answer?

@PromptEngineer48 3 ай бұрын

python privateGPT.py --hide-source Try this

@renierdelacruz4652 7 ай бұрын

Great video, thanks for sharing it. Can you create a Video content talking about making work Cheshire cat on ollama ? Must be interesting.

@PromptEngineer48 7 ай бұрын

Thanks for the idea! On it.

@joseffb7821 5 ай бұрын

Can you still use the ollama API to search your documents? or does it need to be via the console?

@PromptEngineer48 5 ай бұрын

API can do that

@drkvaladao776 2 ай бұрын

Hi, while setting up Virtual environment I'm getting an error, 7:39 what programs do I need? I have installed Miniconda and it's still no running the line. Thks

@PromptEngineer48 2 ай бұрын

then try with Ananconda

@weihe2047 4 ай бұрын

Thank you very much, all the videos you have made are great! However, when I was building the local llm, I found that there are multiple frameworks, such as privateGPT, localGPT, langchain, etc. Similarly, I found that there are very many choices for the llm as well as the vector database (e.g., hugging face vs. ollama), which gave me a big headache, and I was wondering if you could make a I was wondering if you could make a video that explains your recommendations for each part of the process of building a rag-based personal local document chat llm?

@PromptEngineer48 4 ай бұрын

That would be a very good video but very less relevant as everyday we have so many updates. But this is something I can create. thanks for the idea. Definelty will work on that

@weihe2047 4 ай бұрын

@@PromptEngineer48 Thank you for your response! It's true that, as you say, the various programs are moving so fast. Since I'm hoping to be able to build something myself via langchain, I'm starting to work based on your github project, and some of the other out-of-the-box projects (e.g. open-webui, privateGPT, etc.) are just too heavy for me to get into and modify.

@PromptEngineer48 4 ай бұрын

Cool.

@gabrielalejandroverapinto1974 3 ай бұрын

This is great, can you add or show how we could add gpu integration even better if it is over a GUI with privategpt 2.0?

@PromptEngineer48 3 ай бұрын

Alright

@varun_tech7 3 ай бұрын

is there a way to view the actual embeddings value from ChromaDB?

@PromptEngineer48 3 ай бұрын

Sure it is

@varun_tech7 3 ай бұрын

@@PromptEngineer48 How exactly can I view them ?

@panfeng2879 2 ай бұрын

Is there a limitation on the max number of personal documents that I can upload to PrivateGPT?

@PromptEngineer48 2 ай бұрын

No. but then the vectorstore gets confused and not able to get the relevant chunks.

@ahmadsiddiqui7998 3 ай бұрын

@promtEngineer, can you host it with basic UI interface, like people could upload their docs and ask questions, without doing all of this hardwork🙈, and also dont keep anyones personal documents with you

@PromptEngineer48 3 ай бұрын

@davidaliaga4708 Ай бұрын

Do you have the pdf document you tried? Would like to try it myself

@PromptEngineer48 Ай бұрын

It's the think and grow rich book. Just search for the book on the internet

@kusumahaja Ай бұрын

Hello @PromptEngineer48 , I new to python and want to learn this. I followed your instruction in your great video, but had many errors when installing modules in requirements.txt. Any update?

@PromptEngineer48 Ай бұрын

why dont i come up with an updated video.. please give me like a week or so.

@kusumahaja Ай бұрын

@@PromptEngineer48 very nice... thank you sooo much....

@jfranz8491 7 ай бұрын

Inspirational video, great work, subscribed. Could you look into integrating a web gui for requests and responses? I'm looking for a local LLM + Documents + URL for tailored and personalized chatbot knowledge store. I use perplexity right now but want to curate my own documents for tailored content. Keep up the amazing videos!

@PromptEngineer48 7 ай бұрын

For this you can use the cat kzfaq.info/get/bejne/f9l3ibaLq8ezoHk.html

@hpsfresh 3 ай бұрын

How does it knows it should use mistral if I have several models downloaded?

@PromptEngineer48 3 ай бұрын

that is hardcoded

@hpsfresh 3 ай бұрын

@@PromptEngineer48 actually not. This written in config. Take a look

@MA_808 4 ай бұрын

Is there a way for links, images and even videos to be parsed into PrivateGPT ???

@PromptEngineer48 4 ай бұрын

Not with this setup. But yaa we could make that work

@Elrevisor2k 7 ай бұрын

Where the knowledge base is stored? It keeps track of all pdfs already processed?

@PromptEngineer48 7 ай бұрын

The are two folders created automatically names db and cache.

@cringeberry3027 8 ай бұрын

It would be great if you will create a video where you work on entire project, consisting dozens of files, related to coding adding, editing some feature. Thinking about that, DeepSeek Coder would be a best choice I suppose.

@PromptEngineer48 8 ай бұрын

Noted! Will try to implement. Deepseeker Coder is very good actually. Beat so many benchmarks. 😄

@cringeberry3027 8 ай бұрын

@@PromptEngineer48 Also, will it be necessary to use MEMGPT to store different contexts in separate chat windows? I'm bit frustrated trying to figure out bundle of all required to perform what I have mentioned in initial comment =)

@PromptEngineer48 8 ай бұрын

MemGPT working solution.. not yet developed. For a big project like youra need some exploration. I think I have to go for contextual sparsity. ..

@cringeberry3027 8 ай бұрын

@@PromptEngineer48 Bing came back with following bundle: Yes, you can use **AutoGen**, **MemGPT**, and **DeepSeek Coder** together to chat with an entire project like a Ruby on Rails (RoR) project consisting of dozens of files. But it got concerns regarding token limit: "It might be challenging due to the token limit. Each file would need to be processed separately, and the total number of tokens across all files would need to be within the token limit. If the total exceeds the limit, you might need to truncate or otherwise reduce the size of your input1. Remember, a token in this context is not necessarily a word. It could be as short as one character or as long as one word. For example, “ChatGPT is great!” is encoded into six tokens: [“Chat”, “G”, “PT”, " is", " great", “!”]

@Alexiy25raffasan 3 ай бұрын

It would be great to feed local AI with project code or framework, and be able to ask it questions about the code.

@PromptEngineer48 3 ай бұрын

nice idea. i will try to implement the same.

@veniagl3984 3 ай бұрын

can I use this to extract pdf information from 100 pdfs? I need the same information extracted from each pdf and store it in rows, so I need a table of 100x (items to extract), i.e. extract Total Assets from a Balance sheet (will be my 1st column) and I need to do this task for 100 companies. can i use this code to do that? I feel that its more like a many to one thing this one, rather than many to many. Thanks so much for your content!

@PromptEngineer48 3 ай бұрын

If I understand that correctly, that could be hard coded. I think we don't need an LLM here

@kashifrit 3 ай бұрын

Can private GPT be run for a web type interface similar to your previous video ?

@PromptEngineer48 3 ай бұрын

Is there anything which is not possible. 😅😀

@harishhari605 3 ай бұрын

can you make a video for the front end as well?

@PromptEngineer48 3 ай бұрын

Front end. okay. which interface would you like.. like gradio, streamlit etc. any specific requirements. my default would be gradio. would that work?

@harishhari605 3 ай бұрын

Yes, Gradio will be suitable! Thank you for your quick response. I appreciate your efforts. I look forward to watching your video

@PromptEngineer48 3 ай бұрын

Okay. On it

@harishhari605 3 ай бұрын

Thank you very much t @PromptEngineer48.Looking forward to it.

@techietoons 5 күн бұрын

Will it recalculate embeddings everytime I add more pdf documents?

@PromptEngineer48 3 күн бұрын

yes

@techietoons 3 күн бұрын

@@PromptEngineer48 I mean it should only compute embeddings for the new documents only, not for entire set.

@_Rithika-xh7hn Ай бұрын

How to create an API for this?

@PromptEngineer48 Ай бұрын

Yes. I can try

@_Rithika-xh7hn Ай бұрын

@@PromptEngineer48 even after ingesting our source documents,i am getting the answer that is not present in source documents.How can I restrict it to only pdf specific?

@jorgitozor 2 ай бұрын

Nice video, very informative! What do you use to generate subtitles? thanks

@PromptEngineer48 2 ай бұрын

Thanks.. Capcut

@DiminencoIulian 3 ай бұрын

is there the posibility to take responses only from your documents?

@PromptEngineer48 3 ай бұрын

yes. a good prompt. in the begining of the prompt mention that you are a chatbot and answer based on these documents only.... some modifications. but this works for openai api. i have tested in my current project that I am doing on some RAG applications.

@rajayogan8884 6 ай бұрын

Does anyone get this warning - UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown (and subsequent crash)

@PromptEngineer48 6 ай бұрын

I didnt get such

@davidaliaga4708 Ай бұрын

Fantastic! Unfortunately it doesnt work :( When doing python ingest,py we get Your system has an unsupported version of sqlite3. Chroma requirwes sglite3 >= 3.35.0

@PromptEngineer48 Ай бұрын

My bad.i have committed a sin by not freezing the library versions

@davidaliaga4708 Ай бұрын

@@PromptEngineer48 is there a way to correct it. I would really like to try your version. (I think I made it worse because after that I sudo apt get sqlite!)

@PromptEngineer48 Ай бұрын

I will have to search my old files.

@gabriel-gr 5 ай бұрын

This has been very instructive, thanks! Is there an LLM that's better than Mistral at working with very technical documents, i.e. lengthy API implementation documents? I set up my environment exactly as instructed, got my docs indexes and could get some answers on them. But things get murky when I go very specific, with incorrect or incomplete answers.

@PromptEngineer48 5 ай бұрын

Try with mixtral 7x8b

@Paulo-ut1li 7 ай бұрын

Thanks, that’s a great video! I’m testing privategpt for some time and I would love to know if you’re experiencing hallucinations from the chat? And yes mistral seems to be a good model but Zephyr and Dolphin seems to give better answers with a little less performance, depending on the context. Therefore, I couldn’t Get rid of some hallucinations, I would say the reliability of information is 45-65%

@PromptEngineer48 7 ай бұрын

Yes. Hallucinations are there. Need a good model in future

@Glade_gotGlaced 2 ай бұрын

CAN SOMEBODY PLEASE TELL ME HOW TO DOWNLOAD GIT ??????

@PromptEngineer48 2 ай бұрын

please install via this command "pip install python-git"

@nd21209 3 ай бұрын

Thanks for a great video! While ingesting a Word document with docx extension, I get an error No module named 'docx'. Is there something I am missing?

@PromptEngineer48 3 ай бұрын

Try to install the module. Paste the error in chatgpt.

@frankbradford2869 4 ай бұрын

Hi, I did what you said with some hesitation but it worked as you said . This is one good program to use to have a good look at a documents content and meaning. Thanks. BTW is there way to let the program give a full response with out telling it to continue with its explanation?

@PromptEngineer48 4 ай бұрын

I think there should be a verbose flag, which you can set to False.

@fabriziocasula 8 ай бұрын

thank you, sorry but i don't see the old chat interface :-) i have 2 questions how can i remove a ingested document that i don't need? ist it possible to chat with the docker interface ? or is it only for terminal??

@PromptEngineer48 8 ай бұрын

1. U can remove the file. Then delete the db folder and __pycache__ folder.. then run python ingest.py again and python privateGPT.py again. 2. Web UI interface is not integrated here right now. But that is some ok my pipe line as well. I am working on that.

@betagroobox 5 ай бұрын

Wonderful, thank you! My dream would be to feed my local model with all my books in epub or pdf format just once and the model will keep a memory of those. Then from there I have so many ideas but not sure if feasible, maybe someone can help? 1) for each book create a mind map of concepts 2) a diagram of how each book is related to the others (citations, same authors, same topic, related concepts) 3) given a question or a topic the system can point me to which book is better to read. Probably impossible at the moment, isn't it?

@PromptEngineer48 5 ай бұрын

Wonderful idea. I will dedicate time for a POC

@betagroobox 5 ай бұрын

@@PromptEngineer48 Awesome! Another one could be to automatically find the book category. Since the system has already ingested all the books it already knows the discussed topics in each book and from there it can assign each to an ontology of category and subcategories. Like non-fiction/self-help, fiction/novel, non-fiction/self-help/personal growth, and so on. I have other 100 of these ideas, ping me if you need more :D

@SujithAbraham 4 ай бұрын

I would also be interested in something like this if possible as it would be amazing to do this in a repository of books that you know. If you could do this with a non-trivial number of books, say, 100-150 (from Project Gutenberg), it would be a great application of local LLMs.

@AlperYilmaz1 7 ай бұрын

man, I'm having separate conversations with different models using the UI, that's unbelievable..

@PromptEngineer48 7 ай бұрын

Nice.

@nufh 8 ай бұрын

About the context window. I have noticed that it cannot exceed over 2k tokens even though Mistral can support up to 8k. From what I have tested so for, it is like the bot identify itself as a GPT-3, is it because of the openai library?

@PromptEngineer48 8 ай бұрын

yes. That is because we start everything compatible with OpenAI API. then we shift to opensource APIs. We could instead work for OpenSource APIs for the start. 😁

@nufh 8 ай бұрын

@@PromptEngineer48 So for this Ollama, the context windows will not be limited with 2k only right? It will be scale based on the model capability.

@PromptEngineer48 8 ай бұрын

Yes.

@nufh 8 ай бұрын

@@PromptEngineer48 I wished that I could test it right now. Windows user need to wait.

@PromptEngineer48 8 ай бұрын

We can try LocalAI. I will come up with a video

@arvindelayappan3266 6 ай бұрын

what is the system configuration that you are using and what is the response time for the query

@PromptEngineer48 5 ай бұрын

Just a basic 8GB M1

@salahdinwaji7498 2 ай бұрын

Thank you for the amazing video! a quick question, are these local llms safe to use with private data? I want to use it for work but idk if the info will be shared with meta.

@PromptEngineer48 2 ай бұрын

U can switch off internet. Safe 🔐 or not, we cannot guarantee, it may so happen that once u connect to internet the data may be transferred.

@salahdinwaji7498 2 ай бұрын

@@PromptEngineer48 okay, so the advantage of running an llm locally is just to save some $$ from API calls?

@samshosho 5 ай бұрын

Thanks for the great effort. I just have a question. When a file is ingested, then i want to ingest different file, should i delete the db folder first? so not to mix older ingested files with the current one i want to ingest. Also, after ingesting one of my csv files, i asked few questions. Then the answers i was getting were far off and was actually from another source, which i didn't provide, it was from a pdf book about getting rich or something. When i only ingested a csv file with numbers!

@PromptEngineer48 5 ай бұрын

Yes u need to delete the db file first

@williamwong8424 8 ай бұрын

great video. now can u do it in streamlit so there's user interface to chat and how can we host it online? like render?

@PromptEngineer48 8 ай бұрын

Okay. Streamlit and render integration got it. Will do that.

@user-xq9no3so4f 7 ай бұрын

Great video!! However if i follow the instructions my results are different. I created the source_documents folder an put in another pdf-file. When i then execute the "python3 ingest.py" the ingestion seems to work fine. But when i afterwards exceute the privateGPT.py and start to interact with the llm it still responses to "Think and Grow Rich"-Book.

@PromptEngineer48 7 ай бұрын

Delete the db and cache folder

@arvindelayappan3266 6 ай бұрын

@@PromptEngineer48 can we not append the pdf files, do we have to keep removing them. when a new file is added and ingested, it should add the document into its cache and should be able to response from both the document isnt it

@kamcarlson1413 5 ай бұрын

@@arvindelayappan3266 did you ever figure this out?

@AdiMinecraft Ай бұрын

i got errors when run ingest.py Traceback (most recent call last): File "C:\xx\Phyton\ingest.py", line 26, in from constants import CHROMA_SETTINGS File "C:\xx\Phyton\constants.py", line 2, in from chromadb.config import Settings ModuleNotFoundError: No module named 'chromadb'

@PromptEngineer48 Ай бұрын

please try to install the chromadb and then continue

@AdiMinecraft Ай бұрын

@@PromptEngineer48 thanks alot

@user-xl5vc3mu9q 6 ай бұрын

Hello, i keep getting error ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.14.1 (from chromadb) (from versions: none) ERROR: No matching distribution found for onnxruntime>=1.14.1

@JDSchuitemaker 5 ай бұрын

I had the error for ChromaDB too. If you Google for them you will probably find an answer. For ChromaDB this solved it for me: - sudo apt install python3-dev - sudo apt-get install build-essential -y

@waynesbigw2305 4 ай бұрын

I guess your ideas only work so far on a linux system. I'm using an Arch based linux distro. I have no idea what "conda" is. No such thing on my system. After that your instructions got muddier and muddier until I couldn't do anything more.

@PromptEngineer48 4 ай бұрын

We will find other solutions for Arch based. No worries. Stay subscribed.

@waynesbigw2305 4 ай бұрын

@@PromptEngineer48 I'm pretty frustrated. Almost all the videos concentrate on macbooks or windows. But some of us use linux only. I've only used linux for the last 25 years.

@PromptEngineer48 4 ай бұрын

I understand.