My quest to turn a basic Network-Attached Storage (NAS) server into a functional AI workstation all started with one simple thought—could a lightweight model handle routine tasks effectively, even with tight memory limits? Over time, I'd played around with different AI inference engines on various devices, but this idea really got me thinking when I started looking into single-board computer setups. The goal was to see whether a small model could work well for everyday productivity tasks, without needing high-end hardware.
For this experiment, I used a TerraMaster F4-424 Max running TrueNAS. The system was equipped with 32GB of RAM overall, but because of how memory-hungry ZFS and other apps are, I could only spare 8GB for the AI part of the project. I tried several inference engines like llama.cpp and vLLM, but neither worked out. The llama.cpp container failed to detect my processor when I tried deploying it with YAML code. The vLLM setup would just freeze during deployment. After a bit of trial and error, I settled on LocalAI. Once I set it up with 8GB of RAM and 4 CPU cores, the container started right away and everything worked smoothly.
Selecting the Right AI Model
I chose to run the Gemma-4-E2B model, which boasts an impressive 5.1B parameters overall but only uses 2.5B during inference. This efficient use of resources lets the model function well on my NAS, which is powered by an i5-1235U CPU. In active use, the model consumes around 6.2GB of memory, which is perfectly suitable for my setup. When tested, it generated tokens at a rate of 10.5 t/s for text and basic document processing, which is nearly double what my Raspberry Pi can achieve.
One key feature of Gemma-4-E2B is its ability to detect and process images. After enabling the MMPROJ file, the performance did drop a little, but it still worked well for simple tasks like scanning images or performing OCR. When paired with LocalAI, the setup was more than capable of these tasks, eliminating the need to run a dedicated GPU workstation just for basic AI-related productivity work.
Enhancing with Embedding Models
To add even more AI power to my NAS, I included the nomic-embed-text-v1.5 model. Unlike traditional large language models, this one is an embedding model, meaning its main function is to convert text and images into vectors. These vectors get stored in a vector database, placing similar data close together. This process helps apps that use retrieval-augmented generation find the right text chunks quickly, which in turn improves accuracy.
Integrating AI Tools
I integrated both Gemma-4-E2B and nomic-embed-text-v1.5 with several tools like Paperless AI, Open Notebook, and Blinko. All of these tools benefit from the vector database and model support. This setup removed the need to run a separate, high-powered workstation just for AI productivity tasks, which cut down both energy use and physical space requirements.
In the end, this experiment proved that even with limited hardware, complex AI tasks can be handled effectively by using the right models and tools. The combination of Gemma-4-E2B and nomic-embed-text-v1.5, supported by the LocalAI framework, created a powerful yet efficient system ideal for everyday productivity needs. It was a success in both function and resource management.
The system has continued to perform well, with the NAS running both models 24/7 without major issues. The memory usage remains stable, and the power consumption is significantly lower compared to a traditional GPU-based setup. I've found that the performance is sufficient for most of my tasks, from coding assistance to document summarization and even some basic image analysis. The ability to use the web UI in LocalAI to switch between models has made managing the system much easier, and I no longer need to rely on terminal commands for most operations.