Section 8: Running local models

This section walks through the setup for local models: installing Ollama, understanding the hardware you need, choosing a model, and then three applications — autocomplete, agentic coding, and processing text and literature in R.

Running a model locally is Option 1 from Keeping data secure from the cloud: because nothing leaves your computer, it sidesteps the data privacy questions that come with sending your data to a provider.

Connolly, R. M., D. V. Fairclough, E. L. Jinks, E. M. Ditria, G. Jackson, S. Lopez-Marcano, A. D. Olds, and K. I. Jinks. 2021. “Improved Accuracy for Automated Counting of a Fish in Baited Underwater Videos for Stock Assessment.” Frontiers in Marine Science 8: 658135. https://doi.org/10.3389/fmars.2021.658135.