From dc1261d703ae1e1a14841b030888e3f87ff7c38f Mon Sep 17 00:00:00 2001 From: Christian Cleberg Date: Wed, 1 May 2024 21:45:52 -0500 Subject: prettier formatting and rewrap lines --- content/blog/2024-01-13-local-llm.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) (limited to 'content/blog/2024-01-13-local-llm.md') diff --git a/content/blog/2024-01-13-local-llm.md b/content/blog/2024-01-13-local-llm.md index 5f1ca3b..ede130e 100644 --- a/content/blog/2024-01-13-local-llm.md +++ b/content/blog/2024-01-13-local-llm.md @@ -14,10 +14,10 @@ without sacrificing privacy or requiring in-depth technical setup. My requirements for this test: -- Open source platform -- On-device model files -- Minimal required configuration -- Preferably pre-built, but a simple build process is acceptable +- Open source platform +- On-device model files +- Minimal required configuration +- Preferably pre-built, but a simple build process is acceptable I tested a handful of apps and have summarized my favorite (so far) for macOS and iOS below. @@ -25,8 +25,8 @@ and iOS below. > TL;DR - Here are the two that met my requirements and I have found the easiest > to install and use so far: -- macOS: [Ollama](https://ollama.ai/) -- iOS : [LLM Farm](https://llmfarm.site/) +- macOS: [Ollama](https://ollama.ai/) +- iOS : [LLM Farm](https://llmfarm.site/) # macOS @@ -86,8 +86,8 @@ repository](https://github.com/guinmoon/LLMFarm) if you wish. The caveat is that you will have to manually download the model files from the links in the [models.md](https://github.com/guinmoon/LLMFarm/blob/main/models.md) file to -your iPhone to use the app - there's currently no option in the app to reach -out and grab the latest version of any supported model. +your iPhone to use the app - there's currently no option in the app to reach out +and grab the latest version of any supported model. Once you have a file downloaded, you simply create a new chat and select the downloaded model file and ensure the inference matches the requirement in the @@ -96,10 +96,10 @@ downloaded model file and ensure the inference matches the requirement in the See below for a test of the ORCA Mini v3 model: | Chat List | Chat | -|-------------------------------------------------------------------------|-------------------------------------------------------------------| +| ----------------------------------------------------------------------- | ----------------------------------------------------------------- | | ![](https://img.cleberg.net/blog/20240113-local-llm/llm_farm_chats.png) | ![](https://img.cleberg.net/blog/20240113-local-llm/llm_farm.png) | [Enchanted](https://github.com/AugustDev/enchanted) is also an iOS for private -AI models, but it requires a public-facing Ollama API, which did not meet my -"on device requirement." Nonetheless, it's an interesting looking app and I -will likely set it up to test soon. +AI models, but it requires a public-facing Ollama API, which did not meet my "on +device requirement." Nonetheless, it's an interesting looking app and I will +likely set it up to test soon. -- cgit v1.2.3-70-g09d2