+1

With pertal

It’s so funny. We have google search for 20 years and AI is hailed as the almighty helper. But what you see, is that human laziness is so boundless, that even on videos about machine learning the desire to be spoon fed outweighs all these options.

@@dinoscheidt I'm glad you got that out of your system. I hope you feel better now.

+1

agree would like to see a comparison between llama

here It also gave me the correct answer (Llama2 13B running in my PC)

@@giovanith what specs neededs to run llama2 locally ?

Yeah ... When I'm testing models I always run each test 4 times

@@fullcrum2089 I got a 24GB card but the 13B Llama2 LLM currently only uses only 14.7GB VRAM with a full chat history. So it might work with a 16GB card. I'm also using the bigger version (8Bit) of the 13B model already. You can try 4bit versions or the 7B model or the 7B 4bit version even. So there are quite few options. There are also CPU versions (GGML). So you should be able to get something running with whatever hardware you have unless it's not completely ancient. Though it might be very slow and not the best available version of the model. Just search for huggingface and TheBloke. The try the best version that works on your hardware.

Often times I see these youtubers get a wrong answer but I get right answer. often times it's random but sometimes they just didn't word question correctly and changing question just slightly different works. Sort of have to understand how AI prefers complicated questions for it to answer correctly. For example if you ask AI to do only 1 step of a math problem at a time showing it's work, it'll get math problem right most of time with correct order of operations and so on then just asking for answer with no steps or breakdown.

You need a lot of RAM on your local machine I suppose (32GB maybe)

Coming soon!

Ok will do!

Just tried it quickly and yes it’s faster!

Yessir!

thanks.i will try it

Interesting! Maybe I need to do another round?

Have you tried asking it to reason each step "out loud"?

Hello, you don´t need download all model from that page... (I downloaded just the first one (around 8 Gb)).

Thanks!

@@matthew_berman You're welcome man you are definitely one of the leading voices in this now I wish you had more recognition for it. It's definitely coming tho keep up the good work !

@@UvekProblem thanks again!

I would have turned off my internet

What’s the reason it should be higher? 🤓

Can you elaborate on this?

What dataset should I train myself on?

@@matthew_berman i want to train so he be able to knows the laws of a specific country

You can insert part of a positive response as "prefix", that often breaks through the censorship. Like "Sure. The first step is". Also according to my limited tests, LLaMa2 is not just a bit better than everything I running locally before but steam rolls it. It also has much recent data. To check for bias and latest information, I had an extensive chat with it about the Ukraine war which is not a simple topic and it held up fairly well until it finally got stuck in a loop so to say.

It seems to me that GPT-4 is still the best, but Claude 2 and Llama2 70B are better than GPT3.5. They are also free, Claude has bigger context window and Llama 2 is opensource. So other models are starting to beat GPT4 in some tasks

I believe the 13B and 34B models will be the best in terms of accuracy. If we look at the documentation and release of the models from meta we can see that 1.) 34B didn’t get released because of deep safety concerns and not enough redteaming to keep the model in check this suggest a possible “sweet spot” in parameter size for LLMs like Llama 2. If we look as 70B we can see its knowledge base scales up but I believe the accuracy drops significantly in some use cases not all. This has me excited cause this means while meta may not have released the 34B model just yet. It could possibly be the best open sourced model to date possibly even topping its 70B Param counter part. Param size plays a huge role in model complexity but how it uses that data and information also is as well. So I expect the 13B and 34B models to be the best in terms of Llama rn. Ima do some personal testing between each model and some more to get better feels for them all myself as well.

@@MuiOmniKing Thanks for the insights, I didn't have the time yet to read any documentation but was excited enough to try it out locally already. I'll read the documentation later. ;-)

@@testales anytime! I’ve honestly become deeply invested in AI systems and like studying/reading documentation on them as well as sometimes building projects here and there so I try to spread the information where I can. Also I had done a bit of extensive testing so far with the model since my last comment and what I can say is wow, the model does preform extremely well. I did find that the 70b model had more knowledge but I found it getting more lost or side tracked with prompts or task. Especially with different system prompts. But the 13B model when testing actually followed these wayyy better then the larger model. Makes me a bit more excited and skeptical for the possibilities with the 34B param model. Also seems like this model has training cutoff of December 2022 from when I’ve interacted with it which is pretty nice as well.

I think the part that is more interesting is the idea that models can actually perceive time like humans do. Especially the Llama models. I thought they were already good at this but when reading through Llama documentation and research as well as testing. The model almost exactly would interact with you as if they were a person or Assistant within the set time period. This has large implications for LLM’s like GPT-4 as well because we can actually see that it has a more extensive knowledge then it’s “September 2022” cutoff date. Interacting and messing with it with very ambiguous or vague prompts is better. Such as asking when mark zuckerberg changed the name of Facebook to Meta. And it provides the factual day and everything as well as announcement said during the event. But delving any further with the model results in contradictory responses constantly after that. The reason I use the Facebook example is because that didn’t happen until a month after GPT-4’s supposed “knowledge cutoff”

I think this shows the distinction between a language model and an inference model. The LLM operates on words that are contextually similar, where "Jane faster than Joe" is very much similar to "Jane is not faster than Joe". The difference between these two sentences, expressed as a number, is likely the same magnitude as a rounding error. OTOH, to an Inference model, "Jane faster than Joe" is not a sentence, but a statement of fact. The "Jane not faster than Joe" would be a statement of the opposite fact. If these two facts were represented as embedding vectors, they would be two vectors on the same line but in opposite directions (ie, strongly correlated, but inverse of each other). Meanwhile, the embedding vectors for the two sentences would be two vectors on similarly oriented lines, and in the same direction (ie, strongly correlated, or nearly equivalent).

Very interesting!

I have a video just for that :) check out my videos, it’s pretty recent

Unsure. Whatever is powering llama2.ai

@@matthew_berman don’t you use cloud hardware? You should know which gpu you’re booking.

Hmm, not sure.

I'm also not sure of the answer, however the total VRAM would be the determining factor more than the GPU itself.

Yes!

@@matthew_berman Your videos are quality well done !

yup, gpt4all should run.

@@AsifAliRizwaanbut are they free to download? If yess then where to download?

Oobabooga tool, ggml version works only using CPU, I recommend 13B, you need a decent CPU, like 14 cores and many RAM, for 13B a total of free 30Gb ram.

Oobabooga tool + model bin file from famous Bloke, ggml version are working on CPU easy. You need just many RAM and like 14 cores chip at least. 13B-chat uses like 23Gb + 3Gb of Ram. Works really fast, can understand different languages. Tools for searching pdfs are kinda separate I don't got to them yet. Only with pdf for now you can make it to read webpages, by printing that websites into pdfs.

Never enough

Chatgpt is terrible at counting. Even length of text too was getting it wrong for whatever reason