This VLM can be your MultiModal AI with less than 6GB Memory!!!

Рет қаралды 5,102

Күн бұрын

🔗 Links 🔗
Moondream Project page - moondream.ai
Moondream v2 Demo - huggingface.co/spaces/vikhyat...
❤️ If you want to support the channel ❤️
Support here:
Patreon - / 1littlecoder
Ko-Fi - ko-fi.com/1littlecoder
🧭 Follow me on 🧭
Twitter - / 1littlecoder
Linkedin - / amrrs

Пікірлер: 39

@1littlecoder 2 ай бұрын

Hands-on Tutorial kzfaq.info/get/bejne/r82inc-C1tvPqIE.html

@vikhyatk 3 ай бұрын

Thanks for making this video! The new version performs comparably on benchmarks to moondream1, the big difference is that because it doesn’t use restrictive datasets during training it is Apache 2.0 and allows commercial use (unlike moondream1, which was non commercial only).

@marcfruchtman9473 3 ай бұрын

Thank you for making this video. So, I find it interesting that it was trained on Mixtral. It looks really powerful from the examples. (The images in the camera had the text inverted, so that might be one reason why it wasn't performing quite right).

@hemachandhers 3 ай бұрын

bro kindly do a fine tunning of moondream2

@SonGoku-pc7jl 3 ай бұрын

thanks!!! interesting in colab and cases of use are you propose al minut two ;)

@1littlecoder 3 ай бұрын

Thank you!

@satyamtiwary6220 3 ай бұрын

i like moondream a lot, wish it was open so we could train one as well and try to make it better at ocr, document reading etc

@amortalbeing 3 ай бұрын

its opensource! it says this in their website! A tiny open-source computer-vision model that runs everywhere and kicks ass.

@satyamtiwary6220 3 ай бұрын

@@amortalbeing I saw the repo, i didn’t find a training script though

@kamleshpaul414 3 ай бұрын

how can we finetune this model ?

@DikDik-hm8dl 3 ай бұрын

do you have a discord? if yes let me know.

@DexterityX01 3 ай бұрын

What do you have in mind?

@__________________________6910 3 ай бұрын

I'm also big fan of light weight model.

@1littlecoder 3 ай бұрын

Nice :)

@starbuck1002 3 ай бұрын

I want to train a model that turns a bad image description into a well formatted, stable diffusion prompt. Do you have any tips for getting started?

@AbudyAwad 3 ай бұрын

What i would do in your situation is first generate a bunch of images which i know the prompt to, and then get descriptions for those images from a vlm, and then id make the dataset such that it is formatted like this: Make a prompt for this description: . Prompt for description: . And you make a script to automate this, so you eventually get a bunch om prompt-description pairs. And then you fine tune a small llm with that dataset. viola, description to prompt ai model

@rahuljaguar5638 3 ай бұрын

@1littlecoder Can you please make a video on PromptSource ?

@Radik-lf6hq 3 ай бұрын

❤

@user-bd8jb7ln5g 3 ай бұрын

Anybody know if this can run on local LLM or LMM applications - without manual setup?

@elitegamer3693 3 ай бұрын

These models can have massive impact on agriculture and security sectors.

@1littlecoder 3 ай бұрын

Absolutely

@zacboyles1396 3 ай бұрын

Hopefully we can avoid the security psychos looking to mandate religious face garments, like we’ve experienced recently, in what they themselves described as intentionally increasing societal collective stress to induce obedience. This having an (unintended?) consequence of overwhelming nocebo side effects on terrified people. Imagine these models at every corner, alerting whatever system that you are out of compliance with doing/saying/wearing _____. Or is this simply our future…

@1littlecoder 3 ай бұрын

@@zacboyles1396 That's exactly the problem with models that don't care of the human bias. Unfortunately not a lot of people give any damn about this. Few years ago I gave a talk on this topic kzfaq.info/get/bejne/e6l2nq191bfVc4U.html but I don't think not a lot of people in tech agree

@ArunKumar-mp1di 3 ай бұрын

Is it 6gb GPU ram? Llava 1.6 works on my laptop without any GPU using ollama

@Jvo_Rien 3 ай бұрын

Hi, just checked and it seem that it requires (at least on collab) 6GB RAM + 6 GB GPU RAM + 32GB disk space

@mrrohitjadhav470 3 ай бұрын

@@Jvo_Rien will it run on 4gb gpu + 8gb cpu?

@Jvo_Rien 3 ай бұрын

@@mrrohitjadhav470 If you are low on GPU RAM, you face a tradeoff: either you do not perform a full GPU offload (resulting in a slower model) or you use a quantized version (leading to a decrease in model quality).

@maulikmadhavi 19 күн бұрын

moondream2 page says it has limitations of ocr and counting. I am curious how do you know the model is trained with mixtral model. Is there any research paper or blog?

@Vigilence 3 ай бұрын

It’s not bad at all. Not as good as cog agent vqa, Llava 1.6, or qwen-Val-max, which is king, but it’s moving up in quality.

@maulikmadhavi 19 күн бұрын

llava is not open for commercial purposes.

@Shykidss 3 ай бұрын

Can I run it on my rpi 5 4gb ram

@AbudyAwad 3 ай бұрын

Very slowly, and i wouldn't recommend it, because then itll have to use swap memory because it need 6gbs while you only have 4. And using swap memory a lot will wear out the ssd.

@Shykidss 3 ай бұрын

@@AbudyAwad Okey then Ill buy 8gb version