First local LLM to Beat GPT-4 on Coding

First local LLM to Beat GPT-4 on Coding | Codellama-70B

Рет қаралды 21,333

Күн бұрын

The best coding LLM is here and you can run it locally. Codellama-70B is able to beat GPT-40 on HumanEval.
🦾 Discord: / discord
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Patreon: / promptengineering
💼Consulting: calendly.com/engineerprompt/c...
📧 Business Contact: engineerprompt@gmail.com
Become Member: tinyurl.com/y5h28s6h
💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).
LINKS:
Announcement: ai.meta.com/blog/code-llama-l...
ollama.ai/library/codellama
labs.perplexity.ai/
TIMESTAMPS:
[00:00] New Code llama-70B
[01:03] Comparison to GPT-4
[02:10] How was it trained?
[03:21] Running it with Ollama
[04:35] Test it on Perplexity
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...

Пікірлер: 41

@engineerprompt Күн бұрын

If you are interested in learning more about how to build robust RAG applications, check out this course: prompt-s-site.thinkific.com/courses/rag

@marcinkrupinski 4 ай бұрын

Thanks for this video!

@vivekpadman5248 4 ай бұрын

Love these short video

@renierdelacruz4652 4 ай бұрын

Great Video, thanks so much.

@raghuvallikkat3384 Ай бұрын

CAN YOU PLEASE DO A VIDEO ON FINETUNING CODELLAMA WITH AWS SAGE MAKER , DETAILING THE THE DATA SET CREATION PROCESS ALSO

@test12382 4 ай бұрын

Good sir 🎉 but I am confused. When I use codellama on browser via hugging face it fails to understand my question and has hard time staying on topic even when I use chatgpt to make the prompts. Is codellama only good at coding things, not debugging and brain storming together? I find chatgpt very helpful with that

@engineerprompt 4 ай бұрын

Benchmarks is not always the great representation of what is actually possible. Might want to run this locally and change the prompt template if possible. I have seen some conversations that the performance is impacted by the prompt template.

@yunusycle97 4 ай бұрын

How much RAM was this model using for inference on your local machine?

@sherpya 4 ай бұрын

150GB 😂 or you can run quantized for 1/2 or 1/4

@alx8439 4 ай бұрын

As per video, he download 4bit quantized gguf version which weighted 38 gigs. It's around the same amount of VRAM you need to run it (or RAM if you're among an unlucky gpu-poor ones)

@divvylorelearning 4 ай бұрын

Please do a video in setup localgpt for creating docker image and deployment

@engineerprompt 4 ай бұрын

great idea, will look into it.

@user-mz2ei2nx2p 4 ай бұрын

what is the code production quality between a Q4 and a Q8 version? Q8 has less errors in code?

@engineerprompt 4 ай бұрын

Q8 will give you better responses.

@uiobhatt9409 4 ай бұрын

🎉🎉🎉

@Canna_Science_and_Technology 4 ай бұрын

I was playing around with this model on a local build. Very buggy and it kept repeating code.

@engineerprompt 4 ай бұрын

I think the major issue is their new weird prompt template. You need to make sure it's correct

@wayne8797 3 ай бұрын

Is there actually any open source models that is actually good and not overhyped for views?

@MrBlazzerBoy 2 ай бұрын

Well, did you find one?

@RichardGetzPhotography 4 ай бұрын

c'mon, that was funny.

@engineerprompt 4 ай бұрын

@user-pf9jv1fl2n 4 ай бұрын

That wasn't C++

@SirHackaL0t. 4 ай бұрын

Which version will run on a Raspberry Pi 5 8Gb?

@f4ith7882 4 ай бұрын

none of them, probably. It's a 70 billion parameter model.

@jaysonp9426 4 ай бұрын

😂😂😂

@sherpya 4 ай бұрын

no much local 70B 😅

@temp911Luke 4 ай бұрын

GPT4 is around 88.0, not 67.0 : )

@Ricolaaaaaaaaaaaaaaaaa 4 ай бұрын

Where did you get this number from?

@InnocentiusLacrimosa 4 ай бұрын

@@Ricolaaaaaaaaaaaaaaaaaprobably from some dark and smelly tunnel 😉

@emmanuelgoldstein3682 4 ай бұрын

HumanEval leaderboards. @@Ricolaaaaaaaaaaaaaaaaa

@BlayneOliver 4 ай бұрын

I tried 70B and it was an embarrassment. Hating when KZfaqrs click bait by saying these models are anywhere near GPT

@JJBoi8708 4 ай бұрын

Were you using a quantized version?

@jaysonp9426 4 ай бұрын

You mean GPT-4. Can you share more?

@srijonp4 4 ай бұрын

Gpt 4 is updating everyday , codellama is better than a previous version of gpt 4 which was released earlier in 2023

@test12382 4 ай бұрын

Can you tell us more? I used it via hugging face and it was having hard time understanding my prompts even when made with chatgpts help. It was too much effort to communicate

@jaysonp9426 4 ай бұрын

@@test12382 were you using the instruct model?