Llama code 70b video coming soon!

Need to sign in to use the plugin. No thanks. That is not completely local.

I typically don't rely too heavily on AI when coding. I use TabbyML, which has a limited model, but it works for me. It's completely open-source and includes a VSCode extension too. It's free and doesn't require login. I use the DeepSeekCoder 6.7B model locally.

since you have to sign in, does it sends any data upstream when you use local models?

The problem with Cody is that it just autocomplete with local models, a thing you can do with many VsCode Extensions like LLaMA Coder, an many more. All the nice features use the online version, which is extremely limited in numbers of requests if you go for the free plan (a bit of expansion on the monthly numbers of these would make things better to test or to grow a serious interest later leading to a better plan). Also there is a, not indifferent, number of extensions that do those nice features (chat, document, smells, refactoring, explain and tests) the same all in one extension and for free using local models (ollama or openai compatible endpoints). Cody does these features a little better and has a better interaction with the codebase, probably due to the bigger context window (at least from my tests) and a nicer implementaion/integration in VScode, but unless you pay you're not gonna really benefit from them cause of the low free number of requests you can afford, which aren't really enough to seriously dive in.

Matt Williams, a member of the ollama team, shows how to make this work 100% free and open source in his video "writing better code with ollama"

Does this method also allow for completion for large code bases like you went over in a previous tutorial using universal-ctags? Or do you have to still download and use universal-ctags? I think it was your aider-chat tutorial. I do not work with pyhton so using this vscode extension and cody is much better for me (front end developer using HTML, CSS and JS).

Is it cody that understands? I think it is the LM that does. Also, why $9 if I am running everything locally?

How is it local if i have to authorize with 3rd party 😮

Thank you very much for your replay. What type of HW is required to run this model locally

can you select the OpenAI one and run it through LM Studio locally too?

I know it is a sponsored video, but is there any open source alternative to Cody extension? We need a completely local solution, because Cody may use telemetry and gathering some information behind the scenes

The only time I'm coding - is while being on flight. I'm so glad I can use LLM from now on!

Wait, you have GitHub Copilot enabled there too, which shows up in your editor. Are you sure that the completion itself is not provided by the GitHub Copilot extension and not Cody with the local model?

can it only work with ollama? or what if i have a server from llama.cpp running on desired/same port as that of ollama, will it not work? what url (complete, including port) does ollama output, so that I can make my server running on same url. of course it will be localhost like localhost:8080 (original as where llama.cpp server runs) localhost:8081/v1/chat/completion (if api_like_OAI is used). so what does ollama output?

can you use this for reviewing PRs?

What an amazing new development! Thanks for you video. A question: can I use this to complete translate a Python code repository to C++ with the goal to make it run faster? How exactly would we go about doing this?

how does Cody compare to the Cursor extension with GitHub copilot?

can it be run using text generation web UI

I followed your instructions and I failed at 2:38 because I'm using Linux I'm seeing a different output. And thanks for your assistance.

I chose the ollama local model, can cody only use codellama:7b-code? Can I switch to other models that can't be used, or where can I modify them?

thanks for the video, this is absolutely a blessing of an assistant

Thanks so much! We need a video on how to train a local model via LM Studio / VS / Python

Might I ask M2 Max with how much memory and is 14inch? Thinking about get a Max 14" as well. Thanks

Even if its not world changing breakthroughs, the speed at which all this tech is expanding can not be overstated. I remember one of the research labs was talking about how every morning they would wake up and another lab had solved something they had just started/were about to start. This is a crazy time to be alive, stay healthy everyone.

Hi, thanks! I use chatgpt 3.5 for generating python code by just describing what I want. It kind of works... In your opinion is this solution you propose better than gpt 3.5?

I'm pretty capped using 2080s (8gb)/16gb ram--have you tried out HS spaces yet? Would love to figure a way to test out dolphin mixtral etc

That's great! But how is it magically linked to the ollama? How to specify other ollama hosted model (13/34b)?

I am using 2017 MacBook Air. Will using it be instantaneous?

Can it reference variables in other files, same directory, while working in a separate file?

Do I need a GPU of 4 GB vRAM or more for the 7b? Also, Python is the easiest of the programming languages, can I use cody locally for C/C++ or C# and other more robust languages?

Is it possible to use it in a Windows and WSL system? If yes how we should install LLaMA?

I'm looking for a local code assistant. I don't mind supporting the project, with a license for example, but I don't want to log in each use or at all. How often does this phone-home? Will it work if my IDE is offline? Pass.

how do i get the cody settings when useing webstorm? or can i only do this on vscode?

Am I misunderstanding something or are you advertising this as an open source solution while it still dependent on a 3rd party service? What exactly is cody? I would have assumed if it is completely local, it's just a plugin that lets you use local models on your machine. Yet you describe it as having multiple versions with different features in each tier, including a paid tier. How exactly does that qualify as open source?

No matter what i do, i always get this when asking for instructions - "retrieved codebase context before initialization". Is there a discord where we can get support for this?

I seem I seem to have missed something even though I followed steps exactly, I can't tell if I'm using local model or not. But when I unplugged my modem, it didn't respond until I plugged it back in. So I'm doing something wrong. I am running Windows with WSL Linux Subsystem. Typically I can install and run anything Linux/Ubuntu and I do have the ollama server running. 🤷🏻‍♂

is their an extension for visual studio rather than VS code?

How can you are not enjoying creating unit test

So the local one is the 7b version, not the 70b? Or is it a typo in the release?

Seems to conflict with omni pascal extension/code completion, not sure if both can be used ? Any ideas ?

Despite the extension being available in the marketplace of VSCodium, after registration, it attempts to open regular Visual Studio Code (VSC) and doesn't function properly. It's unfortunate to encounter developers creating coding helpers that turn out to be broken tools.

any plan to roll this into a visual studio extension for c++ c# coding.

Will the Chat function be available with Ollama soon?

I have a similar setup, but I'm encountering difficulty getting Cody to function offline. Despite specifying the local model (codellama) and disabling telemetry, the logs indicate that it's still attempting to connect to sourcegraph for each operation.

Can anyone tell me if thre is a difference in code production between Q4 and Q8? i mean Q8 will produce less errors? is it more ''complete''? thnx!

Also it's already downloaded, what is the pull for ?

How do you deploy this on an a completely airgapped network? No network connections during install.

Is it better than chatgpt4?

Tried on windows and couldn't get it to connect to my ollama. The dropdown was set to "experimental-ollama" and "codellama" but when I asked in the chat "what can you do" it would reply with "i'm claude from anthropic" so not sure what is up with that

Is this just for Mac users?

Must the pull be placed in some special folder ? This is not explained, I doubt this will work, the way I did it, don't want models on SSD C drive but HD G drive to experiment with it and safe space on SSDs which really need it like windows updates, got twice 4 TB on SSD but still...

Tried it with C cause python apperently not installed in vs code by default, didn't work for C code, but I see cody is working somewhat, a yellow light bulb appears. I came here for code translation, though code generation is interesting too and similiar, but can cody translate code too ? from go to delphi/pascal ? Is what I am interested in...

what about window？~anyway to run llama ?

ollama : The term 'ollama' is not recognized as the name of a cmdlet, function, script file, or operable program is there a working tutorial for windows 10?

Apparently they only support the given models. And the llama one actually only uses coda-llama13b. Basically it can't run something like Mistral or other llama models. Am I right?

Hi @matthew_berman, I just tried following your instructions and it seems that the Cody provider dropdown never lists any of the models I pulled. All i see is "experimental-codellama:7b", even though I actually downloaded the 13b parameter model. Is there another option? Also with the free account, if you're running a local model, are we still held to the 20-chat/500-autocomplete limits?

No local models when using JetBrains plugin?

Can you write in the Title for what OS this Tutorial is?

It’s not actually fully offline. It still uses their services for embedding and caching even when using local models.

Thanks for this awesome tutorial, how to do that for Windows os??

I get some strange window says: edit instruction code, I guess I have to tell it with to do... generate fibonnaci sequence code perhaps ?

This is so cool, but doesn't the Cody login kind of invalidate the local benefits? A 3rd party still gets access to your code.

Does anyone knows if the 500 autocompletions per month on the Free tier, also applies if we run codellama locally?

Can you please also include steps for those who are using Windows

Does anybody know if this can code Swift UI ?

can i use that without internet ?

what's the point of a local model if you have to sign into some random service to use it???

Great but what are the the requirements for Microsoft servers and clients?

how do I do it if Ollama is on my LAN?

Curious how they compare.

I just tried running this while being offline and it doesnt work how is this local?

Now the only thing I need to figure out is how to add a command to the cody pop up menu or something to add: "translate from go language to pascal language" so I don't have to re-type this constantly... testing big translation now...

Seems like ollama is compatiable for both linux and max. How to use it for windows user?

Damn... That is super dope.

Thanks for this awesome tutorial

However I did not yet install go extension, maybe if go extension is installed, maybe cody can then do code translation from go language ? hmmm not sure yet.... probably not... but very maybe..

*sigh*Any idea when Ollama is coming to windows??

Do you know any open source projects like this ?

Love your channel Matthew! For me however, 100% Local is not having to have an account with an external vendor to run your coding assistant completely locally. I'm looking for just that.

How about using the same idea but with gpt pilot

cody settings: provider: Now it says experimental ollama, curious how to connect it to the pull/download... watching video/continueing

Can it do c#?

So, Pro is free for 2 more days? 😁

You lost me at the terminal step, how you get into ollama, is that it's folder ?

🎯 Key Takeaways for quick navigation: 00:00 💻 *Introduction to Local Coding Assistants* - Introduction to the concept of a local coding assistant and its advantages, - Mention of the coding assistant "Codi" setup with "Olama" for local development. 01:07 🔧 *Setting Up the Coding Environment* - Guide on installing Visual Studio Code and the Codi extension, - Instructions on signing in and authorizing the Codi extension for use. 02:00 🚀 *Enabling Local Autocomplete with Olama* - Steps to switch from GPT-4 to local model support using Olama, - Downloading and setting up the Olama model for local inference. 03:39 🛠️ *Demonstrating Local Autocomplete in Action* - A practical demonstration of the local autocomplete feature, - Examples include writing a Fibonacci method and generating code snippets. 05:27 🌟 *Exploring Additional Features of Codi* - Description of other useful features in Codi not powered by local models, - Examples include chatting with the assistant, adding documentation, and generating unit tests. 07:04 📣 *Conclusion and Sponsor Acknowledgment* - Final thoughts on the capabilities of Codi and its comparison to GitHub Copilot, - Appreciation for Codi's sponsorship of the video. Made with HARPA AI

This is awesome

Nice! Seems to work pretty well on my linux laptop. Would be great if I could save my 10 euros a month for copilot.

I'm saving this

Does this need a dedicated GPU?

Please make a tutorial on how to use AlphaGeometry

Ok it worked, kinda funny: I wrote first two lines and last line and the rest cody did after telling it to "generate fibonnaci sequence code"... thanks might be usefull some day, bit flimsy, but interesting, next I try if it can translate code too function Fibannoci : integer; begin var a, b, c: integer; a := 0; b := 1; while b < 100 do begin writeln(b); c := a + b; a := b; b := c; end; end; end;

Would it be possible to use MPT-7B instead?????????????????????????? Anybody has an idea?

Can you do it on a real PC?

Enterprise AI is the best alternative for OpenAI, always helpful with coding questions

It also automatically opened a command prompt... can proceed from there... plus there is an item in the start menu... probably linked to this messy installation.

1:17 - Um, why Visual Studio Code when there's VSCodium available?

It's not working... I get an error where it says it can't find the model :(

It now looks like once code is selected from pull down list: CodyCompletionProvider:initialized: experimental-ollama/codellama:7b-code

Nova editor needs support for this.