Text Embeddings, Classification, and Semantic Search (w/ Python Code)

Рет қаралды 46,325

Күн бұрын

👉 CXOs, VPs, & Directors, I'm offering AI services here: shawhintalebi.com/
In this video, I introduce text embeddings and describe how we can use them for 2 simple yet high-value use cases: text classification and semantic search.
More Resources:
🎥 RAG: • How to Improve LLMs wi...
📰 Read more: medium.com/towards-data-scien...
💻 GitHub: github.com/ShawhinT/KZfaq-B...
▶️ Series Playlist: • Large Language Models ...
[1] • What Are Word and Sent...
[2] R. Patil, S. Boit, V. Gudivada and J. Nandigam, “A Survey of Text Representation and Embedding Techniques in NLP,” in IEEE Access, vol. 11, pp. 36120-36146, 2023, doi: 10.1109/ACCESS.2023.3266377.
[3] owasp.org/www-project-top-10-...
--
Book a call: calendly.com/shawhintalebi
Socials
/ shawhin
/ shawhintalebi
/ shawhint
/ shawhintalebi
The Data Entrepreneurs
🎥 KZfaq: / @thedataentrepreneurs
👉 Discord: / discord
📰 Medium: / the-data
📅 Events: lu.ma/tde
🗞️ Newsletter: the-data-entrepreneurs.ck.pag...
Support ❤️
www.buymeacoffee.com/shawhint
Intro - 0:00
Problem: Text isn't computable - 0:42
Text Embeddings - 1:42
Why should I care? - 3:15
Use Case 1: Text Classification - 5:49
Use Case 2: Semantic Search - 12:40
Free gift for watching: 23:50

Пікірлер: 58

@ShawhinTalebi 4 ай бұрын

👉More on LLMs: kzfaq.info/sun/PLz-ep5RbHosU2hnz5ejezwaYpdMutMVB0 -- References [1] kzfaq.info/get/bejne/d554eLNzybG6fXU.htmlsi=PA4kCnfgd3nx24LR [2] R. Patil, S. Boit, V. Gudivada and J. Nandigam, “A Survey of Text Representation and Embedding Techniques in NLP,” in IEEE Access, vol. 11, pp. 36120-36146, 2023, doi: 10.1109/ACCESS.2023.3266377. [3] owasp.org/www-project-top-10-for-large-language-model-applications/

@ccapp3389 4 ай бұрын

Love that you’re bringing real knowledge, insights and code here! So many AI KZfaqrs are just clickbaiting their way through the hype cycle by reading the same SHOCKING news as everyone else.

@tylerpoore97 4 ай бұрын

I mean, the guy clickbaited the thumbnail. Also, this is insanely old news at this point(if considered news at all). Video content was on point, but we shouldn't be promoting clickbait methods.

@ccapp3389 4 ай бұрын

I clicked this video for technical explanations and code, not news. There are plenty of dudes reading off the same SHOCKING news across AI KZfaq. I got exactly what I wanted from this video and feel like the title was clear.

@krishnavamsiyerrapatruni5385 2 ай бұрын

I have learnt so much by watching the entire series. Thank you so much Shaw! I think this is one of the best playlists out there for anyone looking to get into the field of LLMs and GenAI.

@ShawhinTalebi 2 ай бұрын

Great to hear! Feel free to share any suggestions for future content :)

@obaydmir8353 3 ай бұрын

Clear and understandable explanation of these concepts. Thanks and really enjoyed!

@PRColacino 3 ай бұрын

Congrats man! Keep going with more real examples with code sharing

@aldotanca9430 3 ай бұрын

Exceptionally clear as always!

@ShawhinTalebi 3 ай бұрын

Thanks Aldo :)

@BrandonFoltz 3 ай бұрын

Great video. The practical use cases for embeddings themselves are undervalued IMHO and this video is fantastic for showing ways to use embeddings. Even if you use OpenAI embeddings, they are dirt cheap, and can provide fantastic vectors for further analysis, manipulation, and comparison.

@ShawhinTalebi 3 ай бұрын

Thanks Brandon! I completely agree. Agents are great, but they seem to overshadow all the relatively simple text embedding-based applications.

@ethanlazuk 3 ай бұрын

SEO here, enjoyed your examples of semantic search and explanation of hybrid search. Great vid and easy to follow. Will explore your channel. Cheers!

@pramodkumarsola 4 ай бұрын

You are the real guy to subscribe and learn

@blackswann9555 3 ай бұрын

Excellent work sir! ❤

@ifycadeau 4 ай бұрын

Wow! Thank you for breaking this down, been trying to figure it out!

@ShawhinTalebi 4 ай бұрын

Glad to help!

@dr.aravindacvnmamit3770 4 ай бұрын

Excellent!

@avi7278 4 ай бұрын

Great format subd

@superresistant0 Ай бұрын

I discovered this concept and it is so useful

@greatwall2003 Ай бұрын

Thanks, useful material 👍

@mr.daniish 3 ай бұрын

Love you shaw!

@ShawhinTalebi 3 ай бұрын

❤️

@cinematicsounds 4 ай бұрын

Thank you very good information, will try to make a database for audio sound effects using vector databases text to audio

@eliskucevic340 3 ай бұрын

Iv been using embeddings for awhile but i find that agents can call specialized tools that can be very useful depending on the applications.

@ShawhinTalebi 3 ай бұрын

Thanks for sharing your insight! Indeed agents and embeddings solve different problems. However, some agent use cases could be reconfigured to be solved with text embeddings + human in the loop.

@uzairmalik7084 12 күн бұрын

Hey Shaw thanks for this wonderful series. I have completed it and learned so many new things but one thing I felt is that the code is very high level and it feels like to me that I have to remember most of the things during coding while practicing with those hugging face models. Do you have any suggestions for that?

@ShawhinTalebi 12 күн бұрын

I think the best way to solidify your understanding is to apply it to real-world use cases.

@pepeballesteros9488 3 ай бұрын

Many thanks for the video Shaw, great content! One simple question: when using OpenAI's embedding model, each resume is represented by an embedding vector. Is this embedding computed as the average of all word vectors?

@ShawhinTalebi 3 ай бұрын

Great question! Embedding models do not operate on specific words, but rather on the text as a whole. This is valuable because the meaning of specific words is driven by the context it appears in.

@sherpya 4 ай бұрын

it's possible to extract software names from the query with a text classifier and apply only e. g. apache airflow to kw search? also what db do you suggest? is postgres with vector db good?

@ShawhinTalebi 4 ай бұрын

Good question. While I haven't seen a text classifier used for KW search, that could be a clever way to implement it. There are several DBs to choose from these days. I'd say go with what makes sense with the existing data infrastructure. If starting from scratch, Elastic search or Pinecone might be good jumping off points.

@aldotanca9430 3 ай бұрын

lanceDB is also quite good.

@KrisTC 4 ай бұрын

I have watched most of the videos in this series and found them really helpful. Something I am looking for that I haven't seen you cover yet. Is some more guidance on preparing data for either RAG or fine tuning. I am sure you have practical tips you can give. I have a large old codebase, we have loads of documentation and tutorials etc, but it is a lot of someone to pickup. This new world of GPTs seams perfect for building an assistant. I will be able to work through it ok, but I suspect there will be a load of learnt best practices or pitfalls to avoid that are a bit more subtle. For example I am looking through our support emails / tickets, lots of them all start with please send logs :) and after a load of back and forth we have info. This is much like a conversation with ChatGPT. For fine tuning is it best to fine tune on a whole thread? Or each chunk of the conversation?

@ShawhinTalebi 3 ай бұрын

Great suggestion! I plan to do a series on data engineering and this would be a great thing to incorporate into it. For you use case, the best choice would depend on what you want the assistant to do. For instance, if you want the assistant to mimic the support rep, then you'd likely want to use each message in the thread with its appropriate context (i.e. preceding messages).

@KrisTC 3 ай бұрын

@@ShawhinTalebi thanks for the tip. That’s what I ended up doing it. Not yet tried actually fine tuning yet. Just finished my data prep. Looking forward to you next series 😊

@Whysicist 4 ай бұрын

LDA - Latent Dirichlet Allocation is kinda trivial these days… Matlab text analytics toolbox works great on pdf’s with bi-grams… a la bag-of-N-Grams. Cool… thanks…

@tamilinfomite 3 ай бұрын

Hi Shawhin, Thanks. I ran into a problem. I tried to use Sentence_transformers model by installing it. It always givens an error no file found config_sentence_transformers.json' in the .cache/huggingface/... folder. Your help is appreciated

@ShawhinTalebi 3 ай бұрын

Not sure what the issue could be. Did you install all the requirements on the GitHub? github.com/ShawhinT/KZfaq-Blog/tree/main/LLMs/text-embeddings

@AlexandreMarr-uq8pw 2 ай бұрын

Can only two kinds of classification be made? If I have lots of types, for example, product classification, can it be applied?

@ShawhinTalebi 2 ай бұрын

You can have several target classes. Here's a nice write-up about doing that with sklearn: scikit-learn.org/stable/modules/multiclass.html

@alroygama6166 2 ай бұрын

Can i use these embeddings with bert based models instead?

@ShawhinTalebi 2 ай бұрын

Yes! In fact, sentence transformers has a few bert-based embedding models: sbert.net/docs/pretrained_models.html

@avi7278 4 ай бұрын

finally someone who speaks with their hands more than I do, lol...

@ShawhinTalebi 3 ай бұрын

😂😂.. 👋 👍

@toddai2721 2 ай бұрын

I call him the hand whisperer.... but really loud.

@chamaljayasinghe4210 2 ай бұрын

✌✌🧑‍💻🧑‍💻

@skarloti 3 ай бұрын

This is not always a good solution if we have multilingual text. I see that LLM context 1M token/character They offer other solutions with functions and external API calls.

@ShawhinTalebi 3 ай бұрын

I'm curious about this. I've seen embedding models that can handle multiple languages, so I'd expect them to work pretty well. Can you shed any more light on this?

@tylerpoore97 4 ай бұрын

Soo, unlike your thumbnail, this has nothing to do with agents... Why mention them?

@ShawhinTalebi 3 ай бұрын

Thumbnail is "Forget AI agents... use this instead". I explain this a bit @3:15.

@onjajaboy 3 ай бұрын

are you persian

@ShawhinTalebi 3 ай бұрын

Yes :)

@cirtey29 3 ай бұрын

By end of next year all the drawbacks of LLMs will be erased.

@ShawhinTalebi 2 ай бұрын

I hope so!

@user-yu2wr5qf7g 2 ай бұрын

cool...

@bentobin9606 3 ай бұрын

is text embedding same as text tokenization done in training ?

@ShawhinTalebi 3 ай бұрын

Good question! These are different things. Tokenization is the process of taking a some text and deriving a vocabulary from which the original text can be generated, where each element in the vocabulary is assigned a unique integer value. Text embeddings on the other hand, take tokens and translate them into meaningful (numerical) representations. I talk a little more about tokenization here: kzfaq.info/get/bejne/meCmicyTp5jQgnU.htmlsi=FwqmkB9Ltyq45n0w&t=348