41 What you’ll need

Software requirements:

A desktop or laptop computer — more on hardware in the next module.
The Ollama software.
A large language model. Ollama makes it simple to download one.
Ideally, another piece of software that talks to Ollama — either ellmer, the R package, which can make calls to Ollama, or an editor extension like Continue, which gives you agentic coding and autocomplete.

41.1 Ollama

The main software you need to get is Ollama. Go to ollama.com and download and install Ollama (you may need to seek IT approval if you are doing this on a work computer). Then you have several choices for models to download directly from Ollama. These are open source models — read the page for each one to work out which suits your application. There’s more on choosing a model in a later module.

Ollama works by setting up a localhost server. This is like a web server that you would access over the internet, except it runs locally on your own computer. It just sits there waiting until you make a request of the LLM. Then what it’s going to do is load a large language model into memory and pass that request to a locally run large language model.

41.2 The commands you need

Ollama comes with a clickable interface, but I find it more convenient to use the terminal. Here are some of the key commands.

ollama serve                  # start the server (the desktop app does this for you)
ollama pull qwen2.5-coder     # download a model
ollama run qwen2.5-coder      # chat with a model (downloads it first if you don't have it)
ollama list                   # see which models you've already downloaded
ollama ps                     # see which models are currently loaded in memory
ollama stop qwen2.5-coder     # unload a model from memory now
ollama rm qwen2.5-coder       # delete a downloaded model from disk

Full list: CLI reference.

Challenge

Install Ollama, then run ollama pull qwen2.5-coder:1.5b followed by ollama run qwen2.5-coder:1.5b and ask it an R question you know the answer to (note running this code will download a 1GB file). Then run ollama ps in a second terminal to see the model sitting in memory. How does the answer compare to what you’d get from a cloud model?