Thanks!

that's awesome to hear, thanks :)

Happy to hear it!

Totally

Glad it helps - may want to try retrieval + reranker system for improved name recall 😅

You are literally educating corporate people. Waiting for next session, Thanks for the efforts

​@@jamesbriggshah really Sorry James 😂 I have just reranked the names in my RAG

How would an LLM know how to optimize for semantic search? I would expect an expansion on the ideas would be better. I need to finish my LLM Ops to get the answer,

​@@edwardmitchell6842 The typical process we currently observe for Retrieval-Augmented Generation (RAG) is as follows: 1. After segmenting the document, embedding is directly performed, and the results are stored in the vector database. 2. For user inputs or queries, embedding is directly carried out, and a similarity search is conducted within the vector database. It is a peculiar solution to rely solely on mathematical similarity matching for a single question and multiple factual paragraphs. As humans, when addressing a customer's request or solving a problem, we are unlikely to simply copy and paste the original question into Google's search box and hit enter. Instead, we tend to abstract and summarize the question and requirements before conducting a Google search. To achieve more precise content retrieval, my personal approach involves: 1. After segmenting the document, each paragraph undergoes summarization using a Language Model (LLM). This summary is then embedded and treated as a comprehensive index for the original document. Together with the original document paragraphs, this summary index is stored in the vector database. During subsequent retrieval and matching processes, the focus is solely on this summarization index, streamlining the matching operation. 2. For user input/questions, a summary is generated using the LLM, embedded, and then queried against the vector database to match the summary index of each document paragraph. After retrieving multiple documents, the results are then evaluated and selected by the LLM, ultimately producing the answer. The current Rag demos and new services, such as rerank provided by community, are all efforts focused on QUERYING. Perhaps we should explore more in terms of how to store, organize, and index documents. An additional hidden benefit of this approach is that, assuming our documents are in English, if a user inputs a Chinese question, direct embedding would inevitably fail to retrieve any content. Through summarization and subsequent embedding, we can translate the original input into English before processing.😄

thanks Shaheer :)

Good point. Do you know about any possibly useful metric/benchmark?

no sponsor from Cohere, I'm sharing what I do in production to make search better

there are many benchmarks comparing bi-encoders (embedding models) to crossencoders (rerankers), but I'm not aware of any for Cohere's model compared directly to ada-002. Nonetheless you can read here txt.cohere.com/rerank/ (it only shows comparison with elastic)

@@jamesbriggs thanks a lot for the reply, so in production you push things after an empirical evaluation? Would it be possible to have a link to some benchmarks? Thanks a lot again

@@frazuppi4897 yeah it's a lot of fast moving projects for me at the moment, so we do a lot of empirical assessments, for reranking you can see benchmarks for some of the best performing embedding + reranker models here huggingface.co/BAAI/bge-reranker-large#baai-embedding They unfortunately don't compare reranker to encoder directly beyond a few statements on rerankers being more accurate - they do explain in better language than I the reason for this though

yes they're fine-tuned specifically for calculating similarity scores - you take a pretrained transformer model, add 1-2 linear layers on to the top of the output logits of the model, and fine-tune on a dataset that would contain records like [sentence A, sentence B, similarity score]

llama-index have a lot of great retrieval tooling - I haven't been able to dive too deeply into it yet but from what I've seen they (1) do support this type of retrieval (ie with reranking), and (2) can likely improve accuracy, but I don't think you can get much faster than what we do here

Why is the embedding taking so long?@@jamesbriggs

I believe Cohere ReRank doesn't use a separate LLM model.. it relies on its algorithm/model

I know it's on the roadmap, but it's not in there yet

you can read about bi-encoders (embedding models) and crossencoders (reranker), there is information compression with the bi-encoder approach as you are encoding generic embeddings, with the crossencoder you are feeding the query and original text, the transformer must then decide, on that specific query, how relevant the document is

@@jamesbriggs copy that - thanks a lot!

we set the `top_n` parameter to `3`, logically the reranking scores every document, then we take the top 3 scoring docs

I did not watch the video but I am interested in knowing what leaderhoard you are referring to

on huggingface /spaces/mteb/leaderboard@@heywrandom8924

@@heywrandom8924 probably this one huggingface.co/spaces/mteb/leaderboard - I'll be talking about other embedding models in upcoming video, but yes it's true, ada-002 is far from best performing

​@@jamesbriggsthank you (:. I didn't watch that specific video as I am not sure what the keywords in the title mean and I am not sure it's relevant to me. I just checked it out a bit and the video looks cool (:.

I probably could have phrased better, there are two parts: 1. Embedding models encode the full sequence into a single vector, transformers work with vectors but they contain a single vector for each token - but these will be compressed through a single layer before producing the similarity score, so there is still compression into a single vector happening, but... 2. Reranker models have the full context, ie they see both the query and the document that they must compute similarity for. An embedding model must produce a single vector embedding for every possible query

makes sense. thanks for explaining!@@jamesbriggs

My first impression was that the need to rerank means that the rank was to optimal to begin with. Your explanation above helped me better understand this. Ultimately this capability should be integrated and not require a different tool.

Sentence transformer has cross encoder models in hugging face you can try them but there quite old

@@haristan1960 Yes true .. I found many cross encoders on SBERT and hugging face . Thanks

check out bpe-reranker huggingface.co/BAAI/bge-reranker-large/tree/main

it's pretty wild

Yes some more detail into working of reranker would be useful.

see here kzfaq.info/get/bejne/jblhqLl-rM7Qh4U.html

Hey Jordan! I haven't tested the open-source cross-encoders/rerankers for a long time - so I'm not sure - they generally get less attention than the encoder models but I'm sure there must be some good rerankers out there

I’m looking into using bge reranker large, however haven’t gotten it to work yet.