Home Assistant Voice and local LLMs

I received my Home Assistant Voice (HAV from here on) this week and after a bit of playing around with the configuration I now have a pretty decent, fully local voice assistant. (At least after we got over the „doesn’t understand how to turn on the lights in the living room“ stage.) This is not a review yet, just a quick documentation how to get local models to run and do their job.

The onboarding documentation is pretty good, but hosting the full stack locally involved a bit of searching for the right project to get everything wired together. Once you follow the documentation and get your little plastic box connected you want to install two plugins, one for Whisper (speech to text) and one for Piper (text to speech). Both via the Wyoming Protocol.

The biggest issue during the setup was the WiFi range of the box being subpar and not making it down a floor to the access point providing the network for 2.4GHz appliances.

To host whisper and piper I am currently using two Docker images on my service host.

Whisper

services:
  whisper:
    container_name: whisper
    restart: always
    command: --model medium-int8 --language en
    volumes:
      - /mnt/fast/whipser/data:/data
    image: rhasspy/wyoming-whisper:latest
    ports:
      - 10300:10300

and Piper

services:
  whisper:
    container_name: piper
    restart: always
    command: --voice en_US-lessac-medium
    volumes:
      - /mnt/fast/piper/data:/data
    image: rhasspy/wyoming-piper:latest
    ports:
      - 10200:10200

The image repositories also give you an idea about available models.

Once you have these two containers running you can configure them, publish them and expose them to your network. I needed to restart Home Assistant once to make the HAV be able to connect to the voice assistant. Before the restart it got stuck and threw an error.

All in all I am pretty impressed by the performance and by the quality. It is as good as Siri for turning on and off lights and setting timers (after I ironed out the first few misunderstandings). I have not gotten Ollama to work, it simply ignores the prompt — so my HAV is currently not talking like Dominic Toretto when turning on a light… yet.