What is the going rate for an AI scientist that can actually cobble together a functioning solution with significant return on investment. It has to be at least 500k. I’m not talking about those subpar Chemists/Engineers PhD “Data Scientist”, but an actual AI scientist that can build from concept to production. Curious

The explanation was excellent. Thanks a lot!

Hello Julien, great video! Still have a question if there is no table with clear content description, such as CV, can Amazon textract extract pdf file information and arrange into certain boxes that would be possible to fill information into different tool?

will you do a tutorial on Distilkit? Please ? And the evolution merging ? I think Pruning is a great way to extract a strong model from a larger model : But how ! what are the do's and Donts? and really how to evaluate the information inside the tensors to select the correct layers to extract ? is there a way to preview how it would result before applying a prune ? IE how do you choose to select a subset of layers to run instead of a whole stack of layers ?.. In some pre-training people have expanded layer by layer ... some have begun with a full layer stack : if they expanded layer by layer Pruning is a real viable option for that model : but if they rained the layer stack as a whole then pruning is not prudent as the layers are inversely connected over the ranges: but the expanded models are heavily connected : hence identifying the data held in specific layers , as well as being able to run a subset of layers ... i would expect it to be like extracting a peft from the selected subset of layers and running that peft or merging that peft to a model ? im not great at these tasks or fully versed , ... but if your with the merging technology that means they have a greater understanding of what is happening in a pre-trained model and its layers : in theory it should not be hard to extract a specific subset of layers to make a new one ! it would only be the alignment after that is required as they still contain thier probability its just the layers are not highly connected: ie alignment should be done with a dataset that the model was already previously trained on ! hence aligning existing knowledge with the layers and aligning the tensors together to return this hidden data ! so before distilling i would suggest fine-tuning the model on a specific dataset for 1000 steps and over-train the model ! on these examples : then when the model is distilled retrain on the same dataset to align ! so you have your baseline to match ! here it would suggest that the previous data that was trained did indeed align ( and infer that all previous data also aligned to the new layer stack ?) again papers are always nice , but experiments and actual doing is always better ! , it nice to see the final result of the distillation ... what we gleen from this is that data does not need to travel a long distance through many layers to prove a result or prediction : in fact i personally believe that the longer the context you wish to use the more layers you need to hold the sequence ! so with smaller models that have great pre-training they will be great for pre-trained tasks with conversational sized contexts but smaller than the larger layer stack : but i would say they perform the same on the same task set : given a small context size ! and the loose when using larger context : hence for simple tasks they are perfect ! hence a Slim model can be considered an ideal agent to become a dedicated Tool ! ie: entity detector , documenter ( but not large summary writer ( smaller essays ) ...etc ...

Every few weeks I have to read some threads in r/machinelearning and search out content like this to flush the buzzwords and singularity fanfiction that is so prevalent these days out of my brain. Thanks for posting! I'm looking forward to listening to the full discussion.

before i watch this ( i will coment after as well i like surprises) - > How do you train a model for funciton calling ? i see that you had a dataset , it has the fake calls inside : and i have use these datasets before and prompted them as they prompted the models : hence yes my model can call functions ! But : in training should we actually have the model coorectly set up to do the actual function call ? or have a set of results to fake the actual function call , so the model during training can access theses calls ! << as it would sem that funciton calling can only really be trained on a set of converstaional history containing the back and forth <<< and the funciton calling datasets do not always prvide this , often they provide some of the calls ? maybe not truly exaxctly as they were commited as , we do see the verbose outputs that these models produce and i would expect that we should be training the models on the full verbose data , which is not the case with these datasets ? So my questio of actually function calling or returning the actual expected values for th emodle to format the response correctly and the the series of calls and responses ? as in geeral the datasets also onoly leave the output seperate ( when the output is the actual process and not jut the answer ? the problem i highlght is the fact that the model did not Pause between function calls or attempt the internal conversation so it might not even reflexct a true possible answer from the modlel , such as synthestic data is : So the model is never really trained on function calling . I also found that if you want to do function calling with the model , it should be correctly alligned , ie most function calling is done on a hosted model , so that means that model needs to be trained as a messaging model (chatML) and if you take it to gguf ( it also needs to stay like this ( if it will be on ollama or lm studio ( hosted gguf ) ... ) but for general usage in a gpt4all unhosted it needs to be in the chatQA format or it also will not work : And if it is in tensors then you can change it on the fly to suit your needs ! SO when creating a model for a specifi purpose ( ie ReACT ) these are also the considerations ( ie the whole react process , not just the ins and outs for the data ) ... as well as make sure to train as chatml ! im not sure your dataset was a CHatML styled dataset , so i had to wangle it the Q/A prompt method ! ... As Always i expect a great video !

Amazing performance! So use this model for RAG or also fine-tune? Look forward to your video on distillation. Thanks for the awesome content.

Sadly anything under 8-12B is garbage for anything other than transcribing or other lightweight menial task

19:55 I think one or all of the creators (Pala Tej Deep, Rishabh Bhardwaj, Soujanya Poria)...are also fans of J _Dilla_ ...legendary music "sampler".

So basically the LLM is just redirecting based on predefined rules to corresponding python functions and print results? Is this how agents generally work?

Finance has always adopted technological breakthroughs first. I work at a tech company, and fortunately or unfortunately, finance will not adopt AI because as the person said, too much risk.

Very important ❤

hi i have a dataset with json files of 5gb how can i use that data? on openai inside throws very little data

no audio

great , I agree sometimes you have to spend a lot of time on IAM than the original problem what a mess

Thank you very much Julien for this high-quality excerpt! Could you please attach the slides in the description, as well as under the other videos?

Julien, this was a great video

Super cool! Just why in speculative decoding the validation part made by the bigger model is faster? I don"t understand how validation works

Awesome video Julien. When is part 3 coming?

Merci Julien!

Is there any chance to get the slides? Its very well organized and presented. Thank you so much for your work✨🔥🔥

Will it work on stabilityai/sd-turbo? I am unable to make it work 😢

Julien, this was a great video and walking down memory lane. Thank you for taking the time to do this. One question: If the pytorch folks natively supported cpu/gpu/tpu via torch.compile then is the integration with Open XLA to support future AI accelerators only?

i think the quantization step is very important , as it reduces the size of the model : it would be good to be able to push it to huggingface then it can be quantized by gguf my repo then the model will be 4.5 gig !! as it is a very large download ... ( esepcially for us slow consumers ( i used to be i a fast internet country but now im in the slow ones ))>> hence cloud dev is the only way to have the same funcitons , without requiring up-to-date technology .. hence i think the pay as you go service is very important !! << Dont forget the little guys ! ( how can i teach my local learners who do ot even have laptops ? computing ... you know it ca even be doen from a good phoe or tablet .. so i feel the market could be ( ema ) countrys ... as well as india a big consumwer of clouds services ( also financially depreived ( in large quantitys ) ) << as well as the STUDENT ! <<

ok i think the code platform to operate on your model is very good as this will have to be the way to get the custom models and datsets up ...Good stuff

I really hope we can get the huggingface datsets to use with this :: As i have already placed my great data sets there ! ( so convienient )

that merge sir!!... its a good merge i also done it :: :Many ways ...All experiments : to leanr about how the merges worked ( before your great video on merging techniques ) I would have put the mistral 2 as the base model : as you could justify that you are upgradiing to biomistal to the instruct 2 version of the mistral: so by leveraging the base model ( mistral 1 , and placing the new desired base model ie mistral 2 ... the merger will use the new tokenizer from mistral 2 and the same config, and ... merge the rest of the model list into that model : I can see that you used the mistral 1 as the base model because it was the original base model for the bio mistral , but i would not consider the BASE RAW model to be the base model but as a QVector .. to align the bio mistral to the new chosen base model : i could have chosen the base model for the bio mistral ? but this would have l=kept the bio mistral the same as it would have only grabbed the deltas from the two mistral models ? hence its not much improvment : but it would preserve the Tokenizer from the bio mistral which may have added conceptula meaning and relationn to the unique domain bio mistral data : If i had mixed a bio mistral and a code mistral ( dolphinCoder ) then i would have chosen a neutral base model ( ie the most common base among the collection of models being merged On to the Base model ) <<<< The base model is the model which is the priority base :? Maybe ? Nice video as usual

yes but the teirs are very expensive ? - they need a py as you go option ; so we can probably load the reuired credits needed to do a single job ... its very restricted ? i did not even see where to pay ? easily ? How to get my current mistral onto the site for trainning ? as i base trained it with my personal data and changed my settings : SO i cannot train on the base and output the peft for attachment ? Will this conect directly to hugging face models and datasets? ...alowing the import and export of the final models? FOr me it would be good to be able to have amy model (stored/ Not hosted (while training project in being done ( ie a month or two : ) testinng it with the deployment (very good ) then uploading to huggig face when complete to deleate and reimport the next model ... ( i primary use 7b model but i would like to try to mix modalitys ie add a encoder from a audio/image input model ? ... so adding an optional steps of encoder decoder model .. or just the decoder modality .. hence the test deployment is vital for this as this is where huggingface fails ( we need to be able to get free endpoints for upto 7/8b model ( perhaps only for short visits only but at least to test ! ) .... the site looks like it will be easy to use without having to sit in a jupiter notebook to perform the daily tasks : SO a pay as you go feature is vital to rapid expansion ? ( monthly really at that price is more than open source user can afford ) ( hence colab 10-20 euro ) lasts quite along time !! and can do a serious train if need be :( monitored is thier only downfall )( availablity issues too for the best serving!! i hope you overcome this usablity issues as i think its key to living up to the phrase we are wishing to help the open source devleoper ... i think the test rig is the service and not the hosting ( as hugginf face spaces !! <<< really is very excellent for this and over time i expect will raise performance as well as allowing for a single gpu to manage (7-8 b models) ...( for training and devlopment ( hf is not good ) so each for its purpose : I also like the fact that the user does not need to understand the requierements of the machine configuration also !! !<< as this has also blocked me from speding on cloud services .. ( even hface ) ( the colab pricing teirs is actually quite Fair , their Google Cloud ( if you can get it to connect up looks actually well crafted despite being censored and restricted in various countrys due to thier politcal affiinitys ( hence your data is at risk with google ) hence independant such as open osource providers are safe as you choose what to share ! << and anything can be deleted and uploaded at will 1<< Freedoms in the hand of the user.. SO there is a lot to consider with my post sorrry ... But i wish you guys luck with this and im exicted to watch the whole series before deciding ( but 300... phew No Way(i Wish ) ... My current goal with my models is ( recall ) and Personality ...( as well as im-plementing the methodologies such as chain of thoughts and visual pacial reasoning etc, this is what i train my mistral for ... ) i have noticed you do not need a lot of data to train for such funcitonality ... in fact 1000-2000 is enogh to teach a model a new trick ....Ie over fit (loss 0.2/0.1 /0.0064 ) the task on 100 samples first before training it for the task, 1000-2000 samples ( to be fit (0.5 loss) ( training the lm head ( with most layers activated ie Push 345,000,000 parameters ) .... then train normally (8-16) lora ..... on a large dataset of 10,000.... fit any where under loss of 1 ... ranging from 1.2-1.5 to 0.5 loss ( this allows the model to become a predictor ) .... now the task is trained .. the prompt can be removed and the model realigned to alpaca ....( allowiing for the new methodology to be absorbed in the model : , rememebring the prompt , you can activate the task using the same prompt !! <<) .... so lots of small training sessions..

Great series!

Hi Julien, I followed exact steps for Meta's newer model Llama 3.1 8B Instruct and get this error on Sagemaker, "The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. The tokenizer class you load from this checkpoint is 'PreTrainedTokenizerFast'. The class this function is called from is 'LlamaTokenizer'." Any thoughts? Please please help.

Thanks for sharing, amazing content

That was great and it helped me so much! Is there this possibility to have the presentation slides?

Need to know how to communicate with chat models that are running using python code. I’m struggling to find this information.

Naw Julien you left HF??

Hello, thank you so much for doing this video, the only question I have is that some of the models do not have a deploy option in the model card (for example gliner models). Is there a way to use these? Many thanks!

Is there a course to understand this here?

Congrats on joining Arceee! Great demo! Keep inspiring and empowering us! <3

Thank you Julien ! Do this work with Vision Language Models as well ?

Very helpfull thank you so much. One question. When it comes to pricing. Does Vertex AI bill you based on the usage (how many seconds the model is running) or for as long as the model is running 24/7?

is sagemaker endpoint a good option for llama3 production use?

Finally, I found an awesome video that shares the story of Hugging Face and presents their products. Thank you for creating this!

i cud c that this video got uploaded 3 years back, is that still valid , I mean the features navigatons

Is there a way to connect this to a custom domain? I want to create a simple web app of chat interface

Thank you!

Thanks so much for the crystal clear explanations! You understand them so well and it's even more amazing how you show them in bullet points and graphs to make your audience understand as well!

Hi Julien, thank you for the valuable contribution. your vids are impressive, love it. May i have one question, that do deploying Llama3 on sagemaker have met HIPAA compliance, Because the data is highly confidential. Hope you will help with an answer. Thank you!

Can i use these transformers models for persian language?

very informative as always !

Hi I have question , I have deployed finetuned llama3 on AWS , but it generate repeated answers if I adjust payload it cut off the end of response , please provide me solution on this I have multiple times changed max lengths, max tokens size also still facing that issue

Hey Julien, Where can we find the training model video for food dataset? Also, I am trying to use a model and deploy it on Hugging Face Inference, but it errors out saying I need a config.json file. I'm not sure how to create it. Any leads would be really helpful. Thanks!