For those interested in using an AI language model (often referred to as an LLM or "Large Language Model") like ChatGPT as a digital assistant, there may be concerns regarding data privacy and how information is handled. The features of Microsoft’s CoPilot might be appealing, but potential privacy issues could be a deterrent. Additionally, some users may wish to avoid subscription fees or being locked into third-party ecosystems, opting instead for open-source software. For these reasons, running an AI model locally could be the ideal solution.

Initial exploration into running LLMs locally on Linux might lead to the use of command-line tools like Ollama. While Ollama is fully functional and offers a powerful terminal experience, it may not provide the most user-friendly interface for beginners.

Another option is LM Studio, a cross-platform solution that operates locally on Windows, macOS, and Linux. LM Studio is powerful and flexible, featuring a traditional graphical user interface (GUI). However, this flexibility can sometimes be overwhelming or lead to potential issues. Additionally, it is worth noting that only the command-line tool of LM Studio is open source, while the rest of the software remains proprietary. Although there is nothing inherently wrong with closed-source software, open-source projects have the advantage of being able to continue development even if the original developer ceases work.

This journey may eventually lead to Alpaca, which serves as a graphical frontend for Ollama. The key takeaway is that Alpaca is designed to be easy to use.

Alpaca is streamlined and user-friendly. It can be installed on any Linux distribution via Flathub, and it includes the Ollama backend. There is no complex configuration required; simply choose an AI model to download and start interacting.

Users might begin with Meta’s newly released Llama 3.1, an open-source model available in 8 billion, 70 billion, and 405 billion parameter sizes. Parameters measure a language model’s complexity, with higher numbers indicating greater capability. The 405 billion parameter model, for example, requires a substantial 231GB download. For reference, ChatGPT 3’s parameter size is 175 billion. Running these extensive models could challenge even the most robust consumer PCs, but smaller parameter models are generally sufficient for serving as effective digital assistants and chatbots.

A useful feature of Alpaca is its native integration into the system’s notifications. This allows the application to be minimized while it processes responses, with notifications alerting the user when an answer is ready.

If a particular model does not meet expectations, removing it and freeing up disk space is straightforward. The “Manage Models” menu allows users to search for, download, and delete a wide range of models.

The Alpaca project was initiated in June 2024, with the first stable release launching just a few weeks ago. While improvements are being implemented consistently, one issue may affect certain AMD GPU users: Alpaca may utilize CPU resources instead of GPU, leading to slower processing. This issue appears to be related to ROCm, but the developer is aware of the problem and is working on a solution.