MISTRAL 7B explained - Preview of LLama3 LLM

Рет қаралды 8,403

10 ай бұрын

Simple inference code for your local PC with less than 8GB GPU: MISTRAL 7B Instruct. From Mistral AI. Mistral 7B beats LLama 2 7B and LLama 2 13B.
The next evolution of LLama models by Meta?
Grouped-query attention explained. CTransformer and GGUF and GPTQ code implementation. Mistral 7B on free COLAB NB. Live demo.
#ai
#aieducation
#coding

Пікірлер: 29

@parhatbazakov1091 10 ай бұрын

My favourite channel! Man, you are a hero! Impossible not to love!

@sebastiansosnowski3859 10 ай бұрын

this model is small enough that you probably could run it on a google pixel 7 pro. I really love the developments in making models that are small enough for on-device inference

@theunknown2090 10 ай бұрын

How? It needs 16gb in fp16

@sebastiansosnowski3859 10 ай бұрын

@@theunknown2090 quantized models of course

@nathanbanks2354 10 ай бұрын

Interesting overview of the company. Thanks for going through it. Overall I'm super impressed--Mistral does better than CodeLlama-13B despite even though it's fine-tuned for coding. I skipped your instructions; I just ran it locally using text-generation-webui like all my other models. Mistral has better formatting with this interface than CodeLlama--I had trouble with the indentation using default settings with CodeLlama--but the quality of the programs is actually better too. TheBloke also already has Mistral in all sorts of formats on Hugging face. However I'm still using GPT-4 when I actually want to write code. I only have 16GB of video RAM, so I don't usually run anything higher than 13B even with quantization. I haven't figured out how to make LoRA's for any model yet, mostly because the quantization complicates things. Doing everything through the the web interface doesn't help. It's nice to have a model that's small enough I could fine-tune it using full FP-16 weights. Thanks for the overview!

@1Esteband 10 ай бұрын

I've been playing with Synthia-7B-v1.3 trained on Mistral 7b and I am very impressed

@code4AI 10 ай бұрын

Context: SynthIA (Synthetic Intelligent Agent) is a LLama-2-7B model, trained on Orca style datasets. Llama legal restrictions apply.

@VaclavKosar 10 ай бұрын

Hello! Great video! Do you share the inference Colab somewhere?

@TheManuforest 9 ай бұрын

Great Video ! ... I Wonder if this model can be applied with a RetrivalQA function when asking questions based on my own docs . Thanks !

@WIM42GNU 10 ай бұрын

Great video, one more question that needs to be answered: How to fine tune it?

@ricardocosta9336 10 ай бұрын

Same as the other ones no? dataset, LoRA and the gpu go brrrrrrrrr

@yannickpezeu3419 10 ай бұрын

Thanks !

@code4AI 10 ай бұрын

Welcome!

@ajaychinni3148 9 ай бұрын

Compared to llama chat , how well is Mistral able to do chating compared to llama chat which has done RLHF to make chat great

@1Esteband 10 ай бұрын

Would you be so kind to share the Colab Notebook?

@naevan1 10 ай бұрын

Hi ! Since you're my go-to tutorial content creator on LLM's I wanted to ask your opinion on something. I'm thinking of doing a thesis on detecting depression from tweets using classical methods like sentiment analysis from HF , but , more importantly, Im thinking of fine tuning an LLM like MISTRAL or LLaMMA to detect specific phrases which would indicate depressive thoughts, absolute wording( always,never, etc.) or First person pronoun usage( I am a fool, I am lazy, I must work hard) things like that. Would you think that would be possible, that I can fine tune it to detect and extract those ? 😅 Think it would be a nice thing to explore..

@chanderbalaji3539 10 ай бұрын

Yes I think you can do that, please refer to Alpaca dataset creation process. Should cost more than sentiment analysis though if you don’t already own your gpu.

@naevan1 10 ай бұрын

@@chanderbalaji3539 thanks a lot. Makes sense ,with specific instruction and output maybe I can steer the LLM that way .

@ryanlowe0 9 ай бұрын

Anyone else notice it said released in Europa? Who knew they were training LLMs on the moons of Jupiter now?

@jeffwads 10 ай бұрын

Sadly it isn’t that good at reasoning.

@ricardocosta9336 10 ай бұрын

:( really? Lets fine tune it then :p

@nathanbanks2354 10 ай бұрын

I definitely noticed that. GPT-4 will sometimes give a Shakespearean Sonnet when I ask for a Petrarchan sonnet, but at least it understands what it did wrong, and sometimes fixes the rhyming scheme when I mentioned the problem. Mistral gave a Shakespearean Sonnet and didn't even know it made a mistake. The Iambic Pentameter for both GPT-4 and Mistral was pretty good. But Mistral would try to fix it and give me 14-lines that don't match any rhyming scheme. Llama-13B didn't produce a sonnet at all. Llama-70B is too slow on my machine--no idea what it does. The fact I'm comparing a 7B parameter model to GPT-4 at all is phenomenal.

@code4AI 10 ай бұрын

New video will be released in 2 hours: Reasoning on MISTRAL 7B, with ICL and multi-agent conversations. Also compare to LLama2 70B chat.

@TheAzraf123 10 ай бұрын

You're a G

@ricardocosta9336 10 ай бұрын

@@code4AI R U a robot? Dude you need to sleep too.

@None_ya_B 9 ай бұрын

Anyone know why the models tend to be woke? They will somtimes rant about social justice. I am guessing it's because they are made by tech companies and they are known for being pretty one sided in their belief and hiring processes but don't know for sure.

@onlineguy4442 9 ай бұрын

No thats not entirely the reason. It's possible a company could have done that during human feedback given to the model. However, most companies rely on the internet for datasets and this is where most company warn you about misinformation by the AI and also biasism.

@onlineguy4442 9 ай бұрын

Biasism can be avoided in many ways such as having a human evaluating the model for unwanted behaviors after training. You can also try to avoid bias from the model by collecting only high quality data. However, it is still possible for the model to be bias in the end because all these extra steps are just trying to reduce the chances of bias.