Installing Private GPT to interact with your own documents!!

No video

Installing Private GPT to interact with your own documents!!

Рет қаралды 47,118

Novaspirit Tech

Күн бұрын

Пікірлер: 117

@user-ei8dc8ug8m 2 ай бұрын

I got the issue fixed and want to share it here - i type poetry install --extras "ui embeddings-huggingface llms-llama-cpp vector-stores-qdrant" instead and it works.

@emammadov 7 ай бұрын

Legend! Thanks a lot! I spent a ton of time trying to find a set of instructions that actually work but to no avail. Until I stumbled across your video. Thanks again!

@dgm1949 5 ай бұрын

Thank you for your videos. As a subscriber to your channel, I look forward to each new one. I would like to make a comment, and perhaps a suggestion for a future video. As a Finance Manager for the last 40+ years, I have come to find these three areas important as it relates to the retrieval of information. What every office need is a way to search: 1) URL's , EXCEL, and PDF on the internet. 2) EXCEL and CSV files locally. (and Securely) 3) PDF and DOC documents locally. (and Securely) And nowadays, you could never get a NON GUI program adopted by the office staff. And of course, all three of the search types would be incorporated into the same GUI. Python based program would be the preferred language. Ollama based. No wrapper programs like streamlit, etc. Thanks for your time, and keep up the good work.

@user-ei8dc8ug8m 2 ай бұрын

I followed the how you instructions until on the part where or at 7:40 min on the video , that to type in ---> poetry install --with ui,local and man i got this error ---> Group(s) not found: local (via --with), ui (via --with). Need help on this part

@explosiveenterprises1479 5 ай бұрын

7:39 Timestamp: poetry install --with ui, local: Group(s) not found: local (via --with), ui (via --with) Seems that these commands no longer work

@fierylight2009 5 ай бұрын

try poetry install -E ui

@lampmanjosh 5 ай бұрын

That worked for me. Thank you!@@fierylight2009

@vipinpg 5 ай бұрын

Framework updated. Check the official website.

@lokomokoyo 4 ай бұрын

Any luck finding direction with this?

@WINHK 4 ай бұрын

@@fierylight2009 thank you so much!!!!!!!😁

@kearneyIT 8 ай бұрын

this is gold, thank you very much and keep up the brilliant work man

@SchoolforHackers 8 ай бұрын

Don: thanks for this! Personally I’m deeply into this topic, so I’ll devour everything you produce on this topic. Obviously, don’t please me personally, but I think you’re at the beginning of a big incoming wave. Hack til it hurts, yes!

@shephusted2714 8 ай бұрын

this is huge and will continue to improve and spread...particularly for commercial apps like lawyer firms and medical practices - expect the LLM ecosystem o expand a ton and things like cxl to kick in and turbocharge the growth in a couple of years - things like models being uncensored, real time and p2p model sharing and datasets will happen and also turbocharge growth - we really aren't there yet - it is going to take 4-5 years to really roll out to entire sme/smb mkt but it should be an interesting ride to say the least. why the limits on tokens? i would like to see things api independent especially with local builds - also it limits the amount of info you can pile into it which is a major limitation

@_BangDroid_ 8 ай бұрын

The implications are revolutionary, and I don't think people especially government realize this. Government will clamp down on AI as soon as they realize the economic effects, I can already smell the stench of lobbyists. Corporations want all the data, especially personal data. I can't see them giving up easy. It's hard enough to lobby in favor of right to repair, fixing phone screens is easy for anyone to understand, but AI is like magic and just so easy to use fear to manipulate policymakers.

@dougle03 8 ай бұрын

Is this available in a container?

@ersineser7610 7 ай бұрын

Thank you for sharing.

@alxt111 8 ай бұрын

Why do you change the python version system wide? Can't you just install 3.11 and create the virtual environment like this "python3.11 -m venv venv"?

@JimWolfe-qe3lu 2 ай бұрын

That's a good question. When I did "sudo update-alternatives --config python3" and set the system version to 3.11, Terminal stopped working. When I set it back to 3.12 (the apparent default for 24.04), Terminal worked again.

@lokomokoyo 4 ай бұрын

Thanks for the video! 5 minutes in and already super helpful. sub'd

@ramsakal1 Ай бұрын

Thanks a lot When we are asking to summarise it summarising only few pages where as i have uplodede pdf of 20 pages.

@Namikaze420 5 ай бұрын

Thank you so much man,,,, this is incredibly useful ... fire content.. a definite subs and follower

@ewookiis 8 ай бұрын

I am a bit wondering why this did not go into a container instead of a venv, since there's dependency twists before this?

@zparihar 8 ай бұрын

Hey bud, I appreciate you showing us however, could you explain more about what private GPT is? And what are the benefits? Is it proxy to chatgpt? What are the hardware requirements? NVIDIA?

@iblackfeathers 8 ай бұрын

what hardware are you using for this demo? i see an rpi keyboard but those could be used as an external keyboard. the documentation mentions using llama.cpp for mac os or nvidia gpu support for windows & linux. this looks promising.

@ewookiis 8 ай бұрын

Actually looks to me like a 400, guessing he's simply using that to access the web gui :).

@nneeerrrd 8 ай бұрын

Care to watch the video? Answer is there

@iomarian13 8 ай бұрын

why didn't you use a venv for installing python 3.11 ?

@danda3688 5 ай бұрын

I don´t know what is wrong but I cannot install the segment with: python3 -m pip install pip, it just doesn´t work, even after changing the: export PATH ="~/.local/bin:$PATH" and i have python3.11.8

@Cookiemantv 2 ай бұрын

i think its updated...some of tutorial in this i have an error and i fixed it one by one... for instance the installation of poetry ui is changed to --extras

@pranavr1099 8 ай бұрын

How well does it work with datasheets? I'm thinking about uploading datasheets of some electronic component and get help with writing drivers

@icarus901 8 ай бұрын

Could you share the hardware and OS specs used in this demo? Or link to a vid where you built it?

@NovaspiritTech 8 ай бұрын

i mention it in the video, nvidia 1070, amd 1700x, and 22.04 ubuntu

@KevinPL10 7 ай бұрын

i am relatively new to this topic and have a question. how can i run this on a server, so that it runs independent from a specific PC? I want to employ it companywide

@DavidHerron 8 ай бұрын

Very interesting. I don't have available any kind of tower PC with graphics card though. I have a Dell Lattitude laptop running Ubuntu. Given the complexity of the setup I'd rather do stuff in a Docker container, or LXD container. Any thoughts on that? I see on Docker Hub several privategpt containers but they haven't been updated anytime recently or else do not have documentation.

@renobodyrenobody 7 ай бұрын

That's exactly what I was thinking. Some containers and installation process is done.

@Reality_Check_1984 6 ай бұрын

@Novaspirit Tech I am having issues with the poetry install --with ui command. I keep getting an error stating that Poetry could not find a pyproject.toml file. Any ideas on how to resolve that?

@dukeseb 7 ай бұрын

It took a bit, I had some hurdles that you didnt have in the video with permissions an installs that didnt exist, but I got through and it was running. then i rebooted and its not running anymore. not sure how to start it back up again also how to make it start on boot. can you help?

@juanamon761 6 ай бұрын

Excelente explicación. Gracias a tu aporte lo tengo funcionando en WLS Ubuntu-22.04 Si alguien desea revisar archivos de tipo docx Tan solo debe ejecutar el siguiente comando, dentro del (venv) poetry add docx2txt

@xsy9093 7 ай бұрын

Nice video! I got it installed, but is there a way to improve the performance of the response? It's often very slow, even if I run it on an RTX 4090 and an i9 13k.

@aketo8082 6 ай бұрын

Thank you. Any special when Python is already installed, for other issues?

@Hex-Mas 8 ай бұрын

The 1080 still delivers in 2023

@user-mfsc-2024 8 ай бұрын

no amd gpu supported

@KarthikMumadi 7 ай бұрын

how to eject ingested files

@MrPeterJMilner 8 ай бұрын

Evening Don, This is interesting and I may well follow your lead on this. Also, just so you are aware, I followed your previous PiMox install video to the letter but could not make it work. It came up with two package missing errors and failed to run, even though it wasnt immediately apparent there were problems with the install process. Cheers, Peter.

@NovaspiritTech 8 ай бұрын

not sure if the pimox works with current kernel for the pi and might have to ignore the install of the kernel in the install script, i was just looking into that to get prox 8 on pi

@MrPeterJMilner 8 ай бұрын

@@NovaspiritTech Well done Don, I have a network of currently 5 or so Pi 4B 8 Gb in Pi NAS Server cases and one NUC 12 Pro i7. The NUC is running Proxmox 8 and the Pi's the traditional Docker and Portainer via DietPi. I would really like all servers to be running Proxmox and be controllable from one console. Ta, Peter.

@ChargedPulsar 4 ай бұрын

Did this work for anyone who tried? I'm having installation problems with exact steps on a clean install.

@GracieDo-ni8in 3 ай бұрын

Thank you for the tutorial. Could you give a hint, though, how should fix the ERROR "Current Python version (3.12.3) is not allowed by the project (>=3.11,

@Sebastian-oz1lj 3 ай бұрын

you need python 11

@RocktCityTim 8 ай бұрын

Thanks!🎉

@moviesync3131 6 күн бұрын

Not offline?

@JDSchuitemaker 8 ай бұрын

Realizing upfront this would not work in my environment (Linux VM within ProxMox on Intel N100) I followed your steps. Low and behold it worked up to uploading a document. When I submit a prompt I see that the program in the terminal crashes immediately. Probably because of no NVIDIA CUDA available in my config. If you know of an alternative that doesn't need a GPU, I would love to try.

@NovaspiritTech 8 ай бұрын

you will need to use a llm that is for your CPU and compile the llama.cpp for cpu instead of cuda

@PaulMillhouse7 7 ай бұрын

Please elaborate on that. How do we compile llama.cpp for CPU instead?

@vipinpg 5 ай бұрын

CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" pip install llama-cpp-python

@JDSchuitemaker 5 ай бұрын

@@vipinpg Thank you! I am going to try that!

@vipinpg 5 ай бұрын

@@JDSchuitemaker try new version of privategpt. It uses ollama insted of llamacpp it is easy. I alreaddy developed few apps in production.

@davidaliaga4708 3 ай бұрын

why you dont just use pyenv? it is much simpler

@bradleywoolf3351 7 ай бұрын

thank you

@iclonethefirst 7 ай бұрын

I followed your steps on Linux Mint 21.3, but now that I'm running it the Browser, I get this Error when trying to use prompts: "NoneType' object has no attribute 'split". What could I have done wrong?

@maxacting 3 ай бұрын

Hey, huge thanks for all the work. I just have question at 7:30. My result is Group(s) not found: local (via --with), ui (via --with). Can somebody help please?

@LatvianGangsta 3 ай бұрын

yes they changed that a bit with is now extras. Look here in the comments or search for the error on google.

@maxacting 3 ай бұрын

@@LatvianGangsta Thanks mate

@maciofacio364 7 ай бұрын

it's working, thanks for that. But... how to do this to work after reboot? do i have to start those few commands after every reboot? Maybe someone have this sorted out?

@_Sepherial 7 ай бұрын

Do you have this installed on a Windows 11 VirtualBox that hasn't been licensed yet. If you can pre-install it and make it available for download. I can get my own Windows license.

@Sebastian-oz1lj 3 ай бұрын

10:15 No arguments expected for "install" command, got "ui"

@PaulMillhouse7 7 ай бұрын

Do you have to have a GPU to run the model?

@PaulMillhouse7 7 ай бұрын

I actually got it working on an old laptop without a GPU. It was slow but it worked. I just did all the other steps except the GPU steps and it worked.

@Rewe4life 2 ай бұрын

with poetry run scripts/setup I am getting authentication errors. It says that I am not allowed to access the mistral7b. even with "poetry run huggingface-cli login --token mytoken" I am unable to finish that step. is there a way around it?

@Rewe4life 2 ай бұрын

figured it out myself: 1. create account at huggingface 2. under settings create a read access token 3. accept the message at mistral7b 4. edit setup file: hf_hub_download( repo_id=settings().llamacpp.llm_hf_repo_id, filename=settings().llamacpp.llm_hf_model_file, cache_dir=models_cache_path, local_dir=models_path, resume_download=resume_download, token="mytoken", # add this ) AutoTokenizer.from_pretrained( pretrained_model_name_or_path=settings().llm.tokenizer, cache_dir=models_cache_path, token="mytoken", # add this ) 5. run scipts/setup again

@chrmrvn5764 2 ай бұрын

poetry install -E ui

@dmbrv 8 ай бұрын

great video

@amethyst_academy 7 ай бұрын

I ran into this error. Cuda out of memory. how to solve this. Please help

@vipinpg 5 ай бұрын

What is your card?

@scotttoward 7 ай бұрын

What an absolute nightmare to install this is impossible unless you are a dev I followed this to the letter over 3 different guides and got the furthest with this one but what a nightmare

@DonStrenz 3 ай бұрын

3 minutes in we find out this is for a MAC. 😞

@Canna_Science_and_Technology 6 ай бұрын

What embedding model is it using?

@vipinpg 5 ай бұрын

By default version use BAAI/bge-large-en New one useses nomic-embed-text with ollama

@sebastianknapek1467 8 ай бұрын

i followed your video. worked well, however after restart i cannot run it anymore... when i execute PGPT_PROFILES=local make run i got following error... poetry run python -m private_gpt make: poetry: No such file or directory make: *** [Makefile:36: run] Error 127 working on OpenSuse. any idea??

@AndreiPetrescu 7 ай бұрын

Having the same issue... did you manage to figure it out?

@sebastianknapek1467 6 ай бұрын

yes i did. i have installed once again poetry . but i am running OpenSuse Thumbleweed and its a rolling distro... @@AndreiPetrescu

@vipinpg 5 ай бұрын

@@AndreiPetrescu add poetry to path variable.

@naveedshariff1723 4 ай бұрын

bro free ah idhu ?

@g.s.3389 8 ай бұрын

does it work without nvidia card?

@BCBUD 8 ай бұрын

Yes

@hsnorh8180 8 ай бұрын

does it support other languages besides english?

@NovaspiritTech 8 ай бұрын

yes depends on your model

@hsnorh8180 8 ай бұрын

@@NovaspiritTech thanks...👍

@HackerHoodie 5 ай бұрын

Send me the coffee

@ScottPlude 8 ай бұрын

this just broke my terminal after installing python 3.11

@nils-erikolsson3539 8 ай бұрын

What does this do except make you own computer use more electricity? I mean compared to the free ChatGPT that run on thousands of GPUs on some supercomputer gpu cluster. Why use local resources for something that ChatGPT does and is free (3.5)?

@NovaspiritTech 8 ай бұрын

privacy

@SchoolforHackers 8 ай бұрын

So I can draw from my own documents to train output. Like: Write a new novel in the style of my old novels, about the era of …

@ewookiis 8 ай бұрын

The same reason you had your own locker at school and not a shared one.

@gabscar1 7 ай бұрын

Other than privacy: fun, learning, tinkering, etc.

@user-ei8dc8ug8m 2 ай бұрын

if you can come back on me on this or anyone for me to fix this i highly appreciate it

@Hex-Mas 8 ай бұрын

What i do for alt python is just name it python310 or python311 and i even still even use python2 lol

@thinkdomotic4915 8 ай бұрын

👍❤️

@Atrivion 6 ай бұрын

Watching this all confused, I really wish PrivateGPT could just make a 1 click installer that does everything. I am so lost lol

@ChargedPulsar 4 ай бұрын

Do you want free cookies delivered to your address as well?

@Atrivion 4 ай бұрын

@@ChargedPulsar Preferably yes. Predipped in milk too. Do you enjoy being an ass online? Is real life hard for you so that you feel the need to act like an ass online to feel big and strong again?

@gmog7857 7 ай бұрын

You are confusing everybody, you could have just downloaded the latest version instead of all these confusing mombo jombo

@michaelkrailo5725 4 ай бұрын

The newest version of what? This is a project on github and not much has changed that I can tell from the documentation. What exactly are you talking about? After reading the documentation, it's not confusing at all.

@Sebastian-oz1lj 3 ай бұрын

Sorry but this guide is garbage i followed it step by step and had multiple issules. aftr 3 days i give up