Ollama

Ollama is an application to download, manage and run open-source LLM Models

Full docs can be found on Github. Quick version is:

  • ollama pull llama2
  • ollama run llama2
  • ollama list
  • ollama run llama2 “Summarize this file: $(cat README.md)”

It also runs an API, typically on port 11434, to serve models.

curl http://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt":"Why is the sky blue?"
}'

VS Code Integration

I am using continue.dev VS Code extension, which can be installed using command ext install Continue.continue (Ctrl-P to access command prompt). Click on the Logo Logo opens the CodeLlama extension sidebar.

First Use

At the bottom of the Continue sidebar, click on the ➕, select Ollama provider, then select the codellama model as described https://ollama.ai/library/codellama. NB There is a python specific model to try!