Ollama is an installable runtime for running LLMs locally.

Basic commands:

• ollama pull xxx
• ollama run xxx
• ollama serve
• ollama ls

An alternative to Ollama is LM Studio.

Stop ollama serving: sudo service ollama stop

Disable it: sudo systemctl ollama stop + sudo systemctl ollama disable
This can be necessary for running Ollama separately in a Docker container, e. g. in Self-hosted AI starter kit.

Set a higher context window for serving: OLLAMA_CONTEXT_LENGTH=32000 ollama serve

Models are usually stored in /usr/share/ollama/.ollama/models

Update a model in Ollama to set a higher context length and then save it:

Commands

/set nothink - disables thinking when running a model.

Coding CLIs interopt

ollama launch opencode or claude - but this requires a big context window.

Ollama Python interface

You can run Ollama and connect a Python script to it, creating a programmatic interface:

OCR Models with Ollama