Connecting a local large language model (LLM) to Home Assistant might seem like a smart way to keep your smart home private and responsive. The idea is to avoid relying on external cloud services while still getting a personalized AI voice that can control devices. In practice, however, most NAS devices aren't built for this kind of work. They run all day, which is great for storage, but their processors and memory are not up to the task of running complex AI models. After testing seven different local LLMs on a typical NAS setup, I found that even small models struggle beyond basic commands like turning a light on or off. The challenge lies in the hardware limitations of these storage-focused devices.
NAS hardware limits LLM performance
The test used a Ugreen DH4300 Plus, a popular NAS with an 8-core ARM Rockchip RK3588C and 8GB of memory. At first glance, these specs might look decent, especially with Ugreen's claim of 6 TOPS of AI performance. However, tools like Ollama—commonly used to run LLMs in Docker containers—don't support the Rockchip NPU through regular Docker setups. This means that any local LLM must instead rely on the CPU for processing. CPUs are not designed for intensive AI work, and this quickly becomes a bottleneck. On the Ugreen, this limits models to around 2 billion parameters for regular use. While quantized versions of 3B or 4B parameter models can technically run, they take longer and struggle under the same system that also powers Home Assistant and other services.
This problem isn't unique to the Ugreen. Take the Synology DS224+, for example, which has a four-core Intel Celeron J4125 and just 2GB of memory by default. While you can upgrade to 6GB, that's still not enough to comfortably manage an AI model when other services like DSM and Home Assistant are also running. Even a 3B parameter model becomes hard to justify for frequent smart home use. These NAS systems simply weren't built for the computational load of AI inference, no matter how much memory or storage they offer.
Only one model performed reliably
Out of the seven models tested, only Qwen3 1.7B stood out as a consistently fast and accurate option. The rest either failed to handle complex requests or took so long to respond that they became impractical for everyday smart home use. Qwen3 managed to deliver precise and timely results across the full range of tasks I threw at it, from simple device control to ambiguous or multi-device commands.
The smallest model, FunctionGemma 270M, was the fastest in terms of response time. However, it often had trouble distinguishing between devices with similar names or in situations where multiple actions might seem possible. A home-optimized variant of FunctionGemma showed marked improvement by being trained specifically on 240,000 Home Assistant user requests. Still, it struggled when requests went beyond its narrow training scope. Qwen3 0.6B also performed quickly but faced similar challenges with ambiguous or multi-step commands.
Home Assistant itself adds another layer of complexity. When an LLM processes a request, it needs to understand a large amount of context. This includes the names, current states, and available tools for all the exposed smart home entities. As the number of devices increases, the size of the input prompt grows, placing more strain on the model. For smaller models like those with less than 2B parameters, it's best to keep the number of entities under 15 to stay within reasonable performance limits. Larger models, while more powerful, require even more system resources to function efficiently.
Better hardware can help, but only so much
The TerraMaster F4-424 offers a more powerful option with its Intel N95 processor and 8GB of DDR5 memory. This setup is clearly superior to ARM-based NAS systems when it comes to running containers and AI models. However, even with these improvements, most of the processing still happens on the CPU unless specific hardware acceleration is available. While it's technically possible to run larger models like 3B or 4B parameter versions on the F4-424, the results often aren't worth it unless you have the hardware to support GPU or NPU acceleration.
Home Assistant also plays a critical role in how well an LLM performs. Unlike a standard chat application, where the prompt is usually straightforward, Home Assistant provides a much richer context. It includes details like device names, states, and functions, which are all fed into the AI model before any request is processed. This increases the workload significantly, and without enough processing power and memory, it can lead to slower performance or incorrect results. In this environment, it's usually better to use a smaller but highly trained model rather than a larger one that lacks the hardware to keep up.
The takeaway is clear: while the idea of running a local LLM with Home Assistant is appealing, the reality is that most NAS devices aren't built for this kind of task. Even if a model loads successfully, it might not be fast or accurate enough for practical use. The Ugreen DH4300 Plus, Synology DS224+, and TerraMaster F4-424 all have their limitations, and the performance of any LLM will depend heavily on the system's available memory, processor capabilities, and whether hardware acceleration is accessible. For now, the most reliable and efficient choice is Qwen3 1.7B, which proved to be the only model that consistently delivered both speed and accuracy in real-world smart home scenarios.