Fine-tuning a CRAZY Local Mistral 7B Model - Step by Step

Fine-tuning a CRAZY Local Mistral 7B Model - Step by Step - together.ai

Рет қаралды 28,477

2 ай бұрын

Fine-tuning a CRAZY Local Mistral 7B Model - Step by Step - together.ai
👊 Become a member and get access to GitHub:
kzfaq.infojoin
📧 Join the newsletter:
www.allabtai.com/newsletter/
🌐 My website:
www.allabtai.com
In this video i fine tune a Mistral 7B model with a self generated dataset. We will go step-by-step over on how to create the dataset, convert to JSONL, upload it, start the fine-tuning, download the model, convert the model to .gguf and test it.
GH llama.cpp:
github.com/ggerganov/llama.cpp
Reddit:
www.reddit.com/r/LocalLLaMA/comments/19bk5w3/i_only_said_hello_finetune_going_off_the_rails/?context=3
00:00 Local Fine Tune Intro
00:18 Flowchart
01:14 Create Mistral 7B Dataset
05:59 Check Dataset
06:56 Upload Dataset
07:26 Start fine-tune job
10:58 Convert model to gguf
12:39 Testing our fine tuned model
16:09 Conclusion

Пікірлер: 88

@luisguimaraes7348 2 ай бұрын

You explained in one video what several courses struggle to explain in dozens... Take a lot!

@GeorgeVuckovich3 2 ай бұрын

I’ve learned more from your channel than any other on YT. Thanks, Kris!

@OutstandingLife 2 ай бұрын

2:32 I remember when 25 examples was more than enough to get started with fine tuning. And now less than a year later, 300+ examples is the minimum. It’s amazing how much I’ve learned from this community and your videos. Thank you 🙏

@dmy_tro 2 ай бұрын

25 examples qualifies and few shot prompting, not finetuning.

@seankernitsman6055 2 ай бұрын

Thanks, Kris, for teaching and keeping us up-to-date in this crazy AI wave! Practical code tutorials are always helpful and appreciated.

@darrylrogue7729 2 ай бұрын

Absolutely loving the content on this channel! Your insights on using AI for fun projects are both informative and inspiring. Each video feels like a new adventure into the potential of AI, and your clear, engaging explanations make even complex topics easy to understand. Can’t wait to see what you’ll explore next!

@saahilsachdev 2 ай бұрын

It's amazing at the way you explain the concepts in a super simple manner. Kudos to you. Keep the banging content coming our way :)

@silentphil77 2 ай бұрын

Love your work man, learnt so much being a member!

@STRIKE01 2 ай бұрын

Thank you! I'm definitely going to try this. It's the first time I see the process somewhat clear!

@JasonBevel 2 ай бұрын

Love being a member, Kris! Always overdelivering value. This video was a great overview of fine tuning your own model. Simple enough for me to understand. 😂

@AzureRaven6969 2 ай бұрын

Doing amazing work, Kris!

@watcher1326 2 ай бұрын

Talk about an AI with attitude. Thanks for the walk through. I haven't used fine tuning and could use the additional information.

@harristengku7153 2 ай бұрын

You know from the value that you produce in your channel. I might just purchase whatever paid content you have. Truly amazing, think about starting a Skool community man. You're great.

@Co0k1em0n5t3r 2 ай бұрын

I really new to this but maaaaan I enjoy the hell out of these videos. I look forward to making my own as soon as I can

@ghosting-cm7zf 2 ай бұрын

your videos never disappoint! Keep it up with AMAZING content.

@openminddream 2 ай бұрын

Always informative and enthusiastic!

@nexuslux 2 ай бұрын

Thanks for the detailed video.. fine tuning seems worthwhile!

@ilhamNurFachri Ай бұрын

This is the most straight forward fine tuning tutorial on youtube ever!

@djjeffgold 2 ай бұрын

Thanks for the great content and repos. Super helpful content.

@pushpaaji 2 ай бұрын

Amazing. Will try it out today for finetuning

@vaibhavmishra1100 2 ай бұрын

You are doing really great and we are learning a lot by you

@PhunkNPharaoh 2 ай бұрын

This is awesome! Great job!!

@carstenli 2 ай бұрын

Another great video, thanks Kris!

@blacktyg3r 2 ай бұрын

Interesting, but the time it took to train on that small dataset is.. a lot. As always, thanks for presenting new stuff :).

@AG-yy3fq 2 ай бұрын

Thanks for sharing Kris :)

@GeronimoTutusaus 2 ай бұрын

Thanks for everything ❤.

@ejh237 2 ай бұрын

Woot Woot!! Thanks for the fantastic Video! Oh, and for the pick 5 😉

@avgplayer 2 ай бұрын

Value for the money. Thanks Kris :)

@Kasyx 2 ай бұрын

Thanks! It was very insightful.

@johnsummers101 21 күн бұрын

Leaving a comment for the youtube algorithm. Thanks for showing an A to Z with this!

@weekendmakeit7760 2 ай бұрын

Great Info!

@saeedasgaritehrani 2 ай бұрын

OMG, I was just surprised multidimensionally XD I mean everything is great man. content is beyond rich and email part and answer was the most hellarios moment of my past 2 weeks.just tnx XDDD

@josefsteiner8616 2 ай бұрын

Awesome Video, thank you for you content.

@ShikariNagasa 2 ай бұрын

This looks awesome actually

@digitalasylum369 2 ай бұрын

This is so cool! I'm gonna make one of these for each of my AI Vtubers. And together is such a great platform, especially with the free $25 in credits!

@ygoralves4664 2 ай бұрын

Amazing video, Kris!!

@ForTheEraOfLove 2 ай бұрын

Hey there! I am thinking of signing up and was wondering, how's the membership going for you? Have you had a chance to implement his code? If so, is it working as expected for you?

@Ryan.Youtube 2 ай бұрын

Looks good!

@DavidChristal 2 ай бұрын

Thanks!

@joepearson7928 2 ай бұрын

I would love to see you explore front end options or adapting modalities with local models.

@Daweiro 2 ай бұрын

Thanks great job :)

@eatingalienbrains 2 ай бұрын

Things are starting to get more and more local baby! 😁

@FallenChapter 2 ай бұрын

locals forever! :D

@abs9994 2 ай бұрын

Great info

@luisguimaraes7348 2 ай бұрын

super cool

@trashchenkov 2 ай бұрын

Nice technical video! Can you dive into the topic of LLM evaluation? For example we fine-tuned model using approach like you showed. How to check the quality of results?

@Canna_Science_and_Technology 2 ай бұрын

Nice, I’m going to do this tomorrow…

@MugenMangekiou 2 ай бұрын

That's so hilarious, please share this model with us, I would like to make some tests

@carthagely122 2 ай бұрын

Most intersting Channel and my english is Bad but hé make things look very soft thanks alot

@elidelia2653 2 ай бұрын

very interesting work! thanks for the lessons. I wonder how much of this programming, if not all of it, can be done autonomously. Probubly an expensive agent task, but curious to push the envolope of AI's capabilities. Fully autonomous assistant populations who live in the internet as we live in societies. Fascinating and inevitable.

@nenesh66 2 ай бұрын

Nice video :)

@positivevibe142 2 ай бұрын

Hmmm, so you call a model "local" while you already shared your info in that .jasonl with the AI Platform using API Key!?!? Am I missing something here!

@positivevibe142 2 ай бұрын

@@VioFax Right. It's just kind of negates the the notion of 'locality' and the reason why people/users go with offline/local venues.

@p1nkfreud 2 ай бұрын

@@positivevibe142yeah I want local so that when the grid goes down and takes the internet with it, I can still learn to build and repair things and grow seeds and hunt food. With the knowledge from local LLMs. Not sure what you guys are doing that you have to hide it so much? Your AI gf get a little frisky? 😂

@positivevibe142 2 ай бұрын

@@p1nkfreud Funny ... lol ... but really this is a very crucial thing! Like for a small business, scientific researcher, medical facility, financial agency, legal firm, laboratory facility, ....etc ..... believe or not the list longer than you can imagine ..... Secret Boyfriends lab research 😁

@Joooooooooooosh 2 ай бұрын

@@p1nkfreudset aside about $10K if you want to train locally

@len5499 Ай бұрын

How else would you fine-tune the model? Some of us don’t have massive compute power that requires a substation to power it. 😅

@jonatan01i 2 ай бұрын

Hello Sir! You say at 12:40 that there is a video showcasing how to set things up. I failed to find it, can you provide a link please?

@nickcochrane6568 20 күн бұрын

Anybody else getting 401 errors when uploading the jsonl files? Tried it in powershell, CLI, python, know my API key is set correctly, inference models work perfect.

@hqcart1 2 ай бұрын

can you make a video about sentiment? I have a dataset and i want a model to give a score between 0 to 100 (bad to good) about each statement.. I have no idea where to start.. thank you.

@vivekkarumudi 2 ай бұрын

I mean the speech was actually pretty cool I wonder how you were able to do it

@DemiGoodUA 2 ай бұрын

How can I fine tune model on own codebase? Is it possible to make dataset from a code?

@junmagic8847 2 ай бұрын

what are the specs of your pc?

@CypherDND 2 ай бұрын

I can't wait until everything is completely local. They can't legislate what is on your local hard drive! Hope I'm one of the lucky winners! :D

@cognivorous1681 2 ай бұрын

The future is local!

@greatsarmad5147 2 ай бұрын

I subscribed to the channel but i don't get why it's not letting me in in the github pls someone explain) thx I also tried signing up on the website but i go the 40 promps and tha't it

@DannyGerst 2 ай бұрын

hilarious😂

@David_Mederake 2 ай бұрын

The best Channel for AI for me. On Point, not too much and every shinything and i learn better english... Thnaks for all @AllAboutAI 🙏

@andreasrosenfelder9094 2 ай бұрын

Sometimes talk to yourself - grandmother wisdom

@jojojkk1335 2 ай бұрын

Which u luck for what u doing 👍👋

@spencerfunk6697 Ай бұрын

Cannot find code for the data set maker. I wish u would have gone over how to build that part a bit more cause I can’t find anything about using lm studio to make datasets

@khoahocgia7671 2 ай бұрын

You can build a website for people to hire for data training, where you take a portion of the money. You assist them in training and return trained files in quantized format. We have a high demand for personal assistant training, especially for private document sets for individual purposes. Therefore, I think you should consider this. We're not skilled in coding, don't want to complicate each step, but we have large PDF, DOCX documents that need training, and we need the output to be quantized to the lowest "Q2" level to run on popular devices like iPhone or Android, not large-scale quantum versions. We focus on the popularity of handheld devices, as not everyone owns a server or wants to pay a large amount for data running through APIs. I can offer prices from $20 to $50 for a 100MB document set. I hope you'll consider offering a rental training model soon.😊

@duckyblender 2 ай бұрын

This is a great idea - creating this soon. I'll keep you updated :)

@simonstrandgaard5503 2 ай бұрын

Similar to your text datasets. I had luck training using axolotl, and it took around 1 hour using two A40 instance with around 40gb vram. On vast. For around 1 usd per hour. I have not yet tried Together, but Together seem simpler to configure than axolotl.