Let's use Ollama's Embeddings to Build an App

No video

Let's use Ollama's Embeddings to Build an App

Рет қаралды 18,198

5 ай бұрын

In this video, we are going to build an app that searches KZfaq transcripts for semantic matches, making it possible to find the right videos for questions folks have.
Find the repo at github.com/technovangelist/videoprojects
Be sure to sign up to my monthly newsletter at technovangelis...
And if interested in supporting me, sign up for my patreon at / technovangelist

Пікірлер: 51

@southVpaw 5 ай бұрын

Your content is excellent, thoughtful, and easy to follow. I appreciate you being here. Please keep making embedding apps. I'm finding so much functionality with an LLM + embedding model beyond just straightforward RAG.

@hand-eye4517 5 ай бұрын

you are really nice to listen to thanks sir, and the subject matter and walkthrogh is everything , much appreciated... and you actually provide the repo.. you are a winner

@vikrantkhedkar6451 4 ай бұрын

The way tou created your own store and used a matching algorithm directly is really helpful to understand what vector databases work under the hood rather than jusr using them directly. Great video will ne using this concepts in a project will update here if anyone is interested. Great video 😀

@voncolborn9437 5 ай бұрын

Interesting project. I've been mulling something like this for some time. I'm an ML newbe so I really had no idea where to start. I do now. I will be following along, listening intently. Thanks Matt.

@piero957 5 ай бұрын

Thank you Matt, learning a lot from every video!

@sinayagubi8805 5 ай бұрын

Can you put this series im a playlist? I am really interested in using embeddings to enable a language model to act as a support agent for a company knowing all the information about the company and being able to pull up links.

@mbottambotta 5 ай бұрын

love it, thank you Matt! how did you end up choosing `nomic-embed-text`? is that model appropriate for all LLMs? or is embedding orthogonal to the LLM you end up using and can you use `nomic-embed-text` whether you work with llama-2 or mistral?

@AutisticThinker 5 ай бұрын

I literally had this same idea a couple weeks ago, but as an enhancement to KZfaq so people can find the best content.

@AINMEisONE 4 ай бұрын

You really are a good teacher! Thanks! So to be clear, you will make an app, that will do the following: 1. take a group of transcripts that deal with same genre, 2. create a index, or database, that can be searched, for a specific question, 3. ollama will be trained for example you talked about making a certain funtion that performs something in 1 video, and in the other you extended this knowledge to include that snippet mentioned in video 1, and. combined it with video 3 to formulate a task, that resolves the question asked?

@solutionhacker 5 ай бұрын

Wonderful video and explanation, as always!

@petrapetra7551 5 ай бұрын

Hi, Matt. Thanks for your Video. I would like to suggest if in your Video Series, you could evaluate the LanceDb database is a DB Vector. Developed in Rust von packages for Python and Javascript. Thank you and I continue to enjoy your Videos.

@DonutMongo 5 ай бұрын

Thanks! Great timing, was just thinking about that how to get control over my news consumption with local llms :-)

@Meyu-Sys 5 ай бұрын

Matt I was using an AI model and saved a session I accidentally deleted that session. I checked the ollama directory and the manifest file for the session as well as the blobs were still there. Is there any way to recover the session?

@technovangelist 5 ай бұрын

Ollama doesn't really have a concept of session that you can recover. If you are using a GUI frontend for Ollama, some of them may have a way of saving sessions.

@CraigRussill-Roy 5 ай бұрын

Supabase for the win !!! I use it for vectors and the api is so easy

@TalkingWithBots 5 ай бұрын

Man it's brilliant, I need to show it to my friend ❤‍🔥

@squiddymute 4 ай бұрын

can you create embeddings from images using something like llava ?

@thaiscvo 5 ай бұрын

please make a step by step video using nomic embed in ollama and a open source llm to build a chat with large docs app

@chrisBruner 5 ай бұрын

Interesting use case.

@Rikearon 5 ай бұрын

LangChain would help do some heavy lifting regarding text manipulation and splitting

@technovangelist 5 ай бұрын

That’s a helluva lot of extra cruft for a tiny bit of convenience

@tanvayvaidya4285 5 ай бұрын

Hi how we can send multiple request ollama. Does this support Multithreading

@Edward_ZS 5 ай бұрын

I was in one of your videos. There was a github that allows to translate the model files, so I know what each one is. But I can't find it.

@technovangelist 5 ай бұрын

This one? Sync Ollama Models with Other Tools kzfaq.info/get/bejne/i8yYi8WoppvYmIU.html

@fearnworks 5 ай бұрын

Great content!

@SlykeThePhoxenix 5 ай бұрын

Are you going to use Docker Compose so it's easy to setup?

@technovangelist 5 ай бұрын

This video is just about embedding

@technovangelist 5 ай бұрын

Docker on Mac doesn’t use the gpu so I’d doesn’t make sense for me to include that. And nowhere near as easy as a straight native install

@RedWhiteBlue209 4 ай бұрын

This is a great video with excellent content. Using TypeScript is a bad choice. Python is a lot more widely used, and you want more people to understand the video fully when they first view it and spread the news. Doing it again with Python?

@technovangelist 4 ай бұрын

Typescript/Javascript tends to get more views. It’s used by far more folks than python as shown by every survey for years. I find it easier to work with and understand. And faster.

@RedWhiteBlue209 4 ай бұрын

@@technovangelist Isn't right that Python is the mostly popular programming language for AI, esp. in learning and research?

@technovangelist 4 ай бұрын

Maybe but for no good reason. Python has no benefit here over any other language.

@technovangelist 4 ай бұрын

But thankyou for strengthening the argument for doing more JavaScript/typescript content.

@RedWhiteBlue209 4 ай бұрын

I don't know. I watched many AI-related youtube videos. They include school classes and videos from businesses and individuals. Most of the time, they offer implementations in Python or even code working in Colab. If you do a search, you can find out that Python is widely taught in colleges, but not TypeScript/Javascript. Maybe I am wrong. Your video has excellent content, and I feel that I didn't get the most out of it.

@HistoryIsAbsurd 5 ай бұрын

Its like you read my mind lol Was just thinking "How can I do this"

@technovangelist 5 ай бұрын

Oh no, it was your mind I was taking over.....

@HistoryIsAbsurd 5 ай бұрын

This...this make so much sense...this explains everything!@@technovangelist

@iham1313 5 ай бұрын

Why and how is „a list of videos“ not an answer? which would be exactly what a RAG is. but as you point out: yours is different. i just don‘t see how. no offense; just trying to understand.

@technovangelist 5 ай бұрын

It is. That’s the answer I provide. Normally rag would give the results of a vector db search to the model to generate an answer

@truehighs7845 5 ай бұрын

And why not just embedding the text transcript?

@technovangelist 5 ай бұрын

As one thing? Then stuff wouldn’t come up in a match

@truehighs7845 5 ай бұрын

@@technovangelist What you did is very advanced, and probably more effective, but sometimes I wonder if we are not reinventing the wheel, models are able of quite good needle in a haystack with plain text, as long as you remain in a reasonable length of text per promtp. What I did notice is when creating the jasonl it can be inconsistent, but then it takes some trial and error the get the prompt right, or an extra call to repair it, which with Openai becomes quite expensive, so yeah, I am cheating... :) But today I am mourning: bitsandbytes update installed a new triton 2.2.0, that broke flash attention, xformers and, because why not, torch, so I am looking at error logs today, thinking wtf am I doing under the rain wasting my time while I could jump in the car and drive to the south of Spain as they have 35 degrees. But I might isolate the beginning of the flow, for just for dataset creation, it's already a decent tool. It was working quite well before I try to lump half of the finetuning library of Github in my venv. Only I, like you, last time I programmed something, was an online casino in Basic with the 'rnd' function, on a 286. So even the most basic development takes me on a cathartic journey contemplating the abysmal ignorance I bring to the table in terms of development, so I always try to kiss. Peace! 😄 *Edit actually the casino was the opposite of "online" ...

@technovangelist 5 ай бұрын

Actually in a lot of cases they are very bad at plucking out the right concepts from a long text. You don't have to spend much time with chatgpt or any of the local models to experience that. But this isn't doing that anyway. This one is all about embeddings which is different from asking a model a question

@truehighs7845 5 ай бұрын

@@technovangelist Yes I agree, but both solutions, IMO give back 'lumps' of the original text and rag from what I have tested, embedding with various dbs and models is always inferior to finetuning that returns a more organic answer, yet less precise. I guess it's going to depend on the application, probably a mix of both is going to be the best, even if I am only half impressed with OpeanAI's Agents that keep forgetting they have a context. Let us know when you have something we can test, whatever works my friend!

@JeffMcJunkin 5 ай бұрын

@@technovangelist I've wondered about taking an LLM-generated summary of the raw transcript and using that summary as the embedding, though. That way you get closer-to-deterministic length so it fits nicely in one embedding. With the length of your videos, I'm betting 32K context length would be enough for the transcript + instructions prompt, plus the output (probably even with chain of summaries).

@florentflote 5 ай бұрын

@allurbase 5 ай бұрын

Hey Matt, which model can I use for embedding text locally? If you have a code sample lying somewhere it would be awesome. THNX

@technovangelist 5 ай бұрын

for code samples, look at the ones associated with the video...my github is technovangelist and the repo is videoprojects. the model to use is the one in the video...nomic text embed

@allurbase 5 ай бұрын

@@technovangelist found it 🎊🎊