Your NVIDIA GPU can power a free ChatGPT (Chat with RTX)

Рет қаралды 11,120

Күн бұрын

Chat with RTX is a ChatGPT-style application that uses Retrieval Augmented Generation (RAG) to allow you to chat with your own data on your own local NVIDIA GPU/PC.
Links mentioned:
Download Chat with RTX - www.nvidia.com/en-au/ai-on-rt...
NVIDIA RAG blog post - developer.nvidia.com/blog/rag...
My RAG tutorial (code from scratch) - • Local Retrieval Augmen...
Sign up to NVIDIA GTC 2024 - nvda.ws/3GUZygQ
PDF links:
Attention is all you need paper - arxiv.org/abs/1706.03762
Embeddings paper - vickiboykis.com/what_are_embe...
Math for ML book - mml-book.github.io/book/mml-b...
Charlie Walks (my novel) - www.charliewalks.com
Nutrition Textbook - pressbooks.oer.hawaii.edu/hum...
Other links:
Download Nutrify (my startup) - nutrify.app
Learn AI/ML (beginner-friendly course) - dbourke.link/ZTMMLcourse
Learn TensorFlow - dbourke.link/ZTMTFcourse
Learn PyTorch - dbourke.link/ZTMPyTorch
AI/ML courses/books I recommend - www.mrdbourke.com/ml-resources/
Read my novel Charlie Walks - www.charliewalks.com
Connect elsewhere:
Web - dbourke.link/web
X/Twitter - / mrdbourke
Twitch - / mrdbourke
ArXiv channel (past streams) - dbourke.link/archive-channel
Get email updates on my work - dbourke.link/newsletter
Timestamps:
0:00 - Intro
0:17 - Chat with RTX requirements
054 - What is Chat with RTX?
1:51 - Downloading Chat with RTX
2:06 - Installing Chat with RTX
2:09 - Make sure to upgrade your NVIDIA Graphics Driver
2:41 - Chat with RTX setup steps
3:04 - What are INT4 models?
3:25 - Opening Chat with RTX
3:48 - Chat with RTX running in the browser
4:33 - Allowing Chat with RTX to work with your own custom data
5:11 - Turning your own custom data into embeddings
6:43 - Question 1: What are embeddings?
7:18 - Question 2: What is the attention mechanism?
7:51 - Question 3: What is matrix multiplication?
8:31 - Question 4: What is Charlie's secret project? (asking a novel)
9:06 - Question 5: What are the macronutrients? And their functions?
10:07 - Outro/NVIDIA GTC
#AI #NVIDIA #GTC24

Пікірлер: 32

@rahmatullahbarkat1002 3 ай бұрын

very detailed use case. thankyou

@gorkemocmenn 2 ай бұрын

dude.great job

@astrocoastalprocessor 4 ай бұрын

🎉 Thank you! This is just the kind of video I've been hoping to find

@mrdbourke 4 ай бұрын

Good timing!

@astrocoastalprocessor 3 ай бұрын

@@mrdbourke It worked flawlessly

@amitkumarchejara6042 4 ай бұрын

This is awesome, but I don't own a GPU yet, still learning!!

@MK-ce7im 3 ай бұрын

best youtuber out there

3 ай бұрын

Do you know whether is it possible to have something similar to DALL-E with this tool?

@ibriarmiroi 2 ай бұрын

Thank you very much as always. Though out of the context of tis video, i was wondering what would be the replacement for Tensorflow examination Certificate since it would be discontinues on 31st of may this year ? and also , would be possible to make video about the available Certificates online ? since your expertise opinion on each Certificate would be Gold ! thank you

@MK-ce7im 3 ай бұрын

this used to work fine but after changing my folder path for the docs, I can no longer open it. I wish I can change it somehow without opening it

@kumarvuppala5286 3 ай бұрын

what is best hardware required ?

@fabiosoave6622 2 ай бұрын

Thank you for the very comprehensive tutorial! I'm interested in creating an interactive chat BOT to respond to Facebook messages related to medical topics. What I'm asking is, given the issues some have reported regarding the management of very large datasets of scientific articles, how can we obtain a system that is both comprehensive and effective? How can we organize the material questions correctly to make the most of the program's capabilities? Thank you.

@MED_Laaguidi 5 күн бұрын

can i use it to translate a pdf from english to other languages by keeping the same style and designs of the pdf?

@doctorfunhome7842 2 ай бұрын

Quadro T600 can run it ?

@erentas7391 4 ай бұрын

you are back at last.

@mrdbourke 4 ай бұрын

yo yo yo!

@justusbatekerd 4 ай бұрын

Did you have any errors when you installed the software? I have had numerous errors with the sentence-transformers not being installed. Can anyone help with this?

@brendendas 4 ай бұрын

Did you find a fix yet? I had the same problem which was addressed on a forum and the solution solved the issue for me.

@sidrajhans Ай бұрын

This beautiful RTX 4090 is love. I wish I had one. If you ever feel like donating, sign me up.

@animetransition1030 3 ай бұрын

Yeah make an app like this

@FernandoLXIX 4 ай бұрын

Don't load a huge dataset. I uploaded every college digital file I had and the thing went crazy. After a few hours of it trying to "digest" the data I killed the program and had to reinstall because there doesn't seem to be a way of telling it to stop reading your data. I'll try again with a way samaller dataset as you did in the video.

@gordonthomson7533 4 ай бұрын

What GPU are you running? I’m awaiting 4080 super and hoping to squeeze a load of value via this…

@FernandoLXIX 4 ай бұрын

Don't. After some testing I can tell you it's quite useless. The answers it provides are very bad and short. It doesn't care that much about the data you provide it with. Bare in mind it's a "demo". Maybe in a few months or years it becomes decent. For now, just get chatGPT 4 instead@@gordonthomson7533

@dexterroy 4 ай бұрын

My GPU is an older RTX 2070. After downloading this nvidia package, I realised it's not going to run on my GPU. I created a similar chat bot that can do Q&A using a pdf file of your choice. I'm using llama-2-7B there. I had to use Google Colab to get 16GB GPU. But man, it's so slowww.. @Daniel Can you please do a video on pdf chat using langchain and one of those free LLM models?

@bigdhav 4 ай бұрын

I have a 4070, but i cant get it to work due to some strange dependency.

@mrdbourke 4 ай бұрын

Which one is that? Have you installed NVIDIA drivers?

@ybaran 4 ай бұрын

😮

@AnshumanKumar007 4 ай бұрын

Will this on my RTX 3060 ? I have a laptop. Wish I bought the RTX 3090 back then, I could have afforded it. Still can but now I am hearing of layoffs in my org, so I am gonna have to save up in case I get axed. EDIT: It doesn't seem it can. I wish my 3060 had like 16 gbs of VRAM :(((((

@gordonthomson7533 4 ай бұрын

That’s odd, minimum is 8gb