+++
date = 2024-01-13
title = "Running Local LLMs on macOS and iOS"
description = ""
draft = false
+++

# Requirements

I\'ve recently started playing with large language models (LLMs), mostly
in the popular chatbot form, as part of my job and have decided to see
if there\'s a consistent and reliable way to interact with these models
on Apple devices without sacrificing privacy or requiring in-depth
technical setup.

My requirements for this test:

-   Open source platform
-   On-device model files
-   Minimal required configuration
-   Preferably pre-built, but a simple build process is acceptable

I tested a handful of apps and have summarized my favorite (so far) for
macOS and iOS below.

> TL;DR - Here are the two that met my requirements and I have found the
> easiest to install and use so far:

-   macOS: [Ollama](https://ollama.ai/)
-   iOS : [LLM Farm](https://llmfarm.site/)

# macOS

[Ollama](https://ollama.ai/) is a simple Go application for macOS and
Linux that can run various LLMs locally.

For macOS, you can download the pplication on the [Ollama download
page](https://ollama.ai/download/mac) and install it by unzipping the
`Ollama.app` file and moving it to the
`Applications` folder.

If you prefer the command line, you can run these commands after the
download finished:

```sh
cd ~/Downloads && \
unzip Ollama-darwin.zip && \
mv ~/Downloads/Ollama.app /Applications/
```

After running the app, the app will ask you to open a terminal and run
the default `llama2` model, which will open an interactive
chat session in the terminal. You can startfully using the application
at this point.

![Ollama](https://img.cleberg.net/blog/20240113-local-llm/ollama.png)

If you don\'t want to use the default `llama2` model, you can
download and run additional models found on the
[Models](https://ollama.ai/library) page.

To see the information for the currently-used model, you can run the
`/show info` command in the chat.

![Model
Info](https://img.cleberg.net/blog/20240113-local-llm/ollama_info.png)

## Community Integrations

I highly recommend browsing the [Community
Integrations](https://github.com/jmorganca/ollama#community-integrations)
section of the project to see how you would prefer to extend Ollama
beyond a simple command-line interface. There are options for APIs,
browser UIs, advanced terminal configurations, and more.

![Ollama
SwiftUI](https://img.cleberg.net/blog/20240113-local-llm/ollama-swiftui.png)

# iOS

While there are a handful of decent macOS options, it was quite
difficult to find an iOS app that offered an open source platform
without an extensive configuration and building process. I found LLM
Farm to be decent enough in quality to sit at the top of my list -
however, it\'s definitely not user friendly enough for me to consider
using it on a daily basis.

[LLM Farm](https://llmfarm.site/) is available on TestFlight, so
there\'s no manual build process required. However, you can view the
[LLMFarm repository](https://github.com/guinmoon/LLMFarm) if you wish.

The caveat is that you will have to manually download the model files
from the links in the
[models.md](https://github.com/guinmoon/LLMFarm/blob/main/models.md)
file to your iPhone to use the app - there\'s currently no option in the
app to reach out and grab the latest version of any supported model.

Once you have a file downloaded, you simply create a new chat and select
the downloaded model file and ensure the inference matches the
requirement in the `models.md` file.

See below for a test of the ORCA Mini v3 model:

  Chat List                                                                 Chat
  ------------------------------------------------------------------------- -------------------------------------------------------------------
  ![](https://img.cleberg.net/blog/20240113-local-llm/llm_farm_chats.png)   ![](https://img.cleberg.net/blog/20240113-local-llm/llm_farm.png)

[Enchanted](https://github.com/AugustDev/enchanted) is also an iOS for
private AI models, but it requires a public-facing Ollama API, which did
not meet my \"on device requirement.\" Nonetheless, it\'s an interesting
looking app and I will likely set it up to test soon.