diff options
Diffstat (limited to 'content/blog/2024-01-13-local-llm.md')
| -rw-r--r-- | content/blog/2024-01-13-local-llm.md | 24 |
1 files changed, 12 insertions, 12 deletions
diff --git a/content/blog/2024-01-13-local-llm.md b/content/blog/2024-01-13-local-llm.md index 5f1ca3b..ede130e 100644 --- a/content/blog/2024-01-13-local-llm.md +++ b/content/blog/2024-01-13-local-llm.md @@ -14,10 +14,10 @@ without sacrificing privacy or requiring in-depth technical setup. My requirements for this test: -- Open source platform -- On-device model files -- Minimal required configuration -- Preferably pre-built, but a simple build process is acceptable +- Open source platform +- On-device model files +- Minimal required configuration +- Preferably pre-built, but a simple build process is acceptable I tested a handful of apps and have summarized my favorite (so far) for macOS and iOS below. @@ -25,8 +25,8 @@ and iOS below. > TL;DR - Here are the two that met my requirements and I have found the easiest > to install and use so far: -- macOS: [Ollama](https://ollama.ai/) -- iOS : [LLM Farm](https://llmfarm.site/) +- macOS: [Ollama](https://ollama.ai/) +- iOS : [LLM Farm](https://llmfarm.site/) # macOS @@ -86,8 +86,8 @@ repository](https://github.com/guinmoon/LLMFarm) if you wish. The caveat is that you will have to manually download the model files from the links in the [models.md](https://github.com/guinmoon/LLMFarm/blob/main/models.md) file to -your iPhone to use the app - there's currently no option in the app to reach -out and grab the latest version of any supported model. +your iPhone to use the app - there's currently no option in the app to reach out +and grab the latest version of any supported model. Once you have a file downloaded, you simply create a new chat and select the downloaded model file and ensure the inference matches the requirement in the @@ -96,10 +96,10 @@ downloaded model file and ensure the inference matches the requirement in the See below for a test of the ORCA Mini v3 model: | Chat List | Chat | -|-------------------------------------------------------------------------|-------------------------------------------------------------------| +| ----------------------------------------------------------------------- | ----------------------------------------------------------------- | |  |  | [Enchanted](https://github.com/AugustDev/enchanted) is also an iOS for private -AI models, but it requires a public-facing Ollama API, which did not meet my -"on device requirement." Nonetheless, it's an interesting looking app and I -will likely set it up to test soon. +AI models, but it requires a public-facing Ollama API, which did not meet my "on +device requirement." Nonetheless, it's an interesting looking app and I will +likely set it up to test soon. |