Working with a team of 10+ other to build a fully local voice assistant on a Raspberry Pi 5 to act as the brain for a 0.75 m tracked robot. Wake-word detection, speech-to-text, language understanding, and speech synthesis all run on-device with no cloud dependency and no network requirement, which keeps latency predictable and the robot functional anywhere.

As the lead, I help other subteams so I have a general understanding of all systems. Additonal teams include: Controls, Mechanical, Composites, Electrical, Social/Business.

The pipeline chains openWakeWord for always-on listening, whisper.cpp for transcription, a quantized 3B-parameter model under llama.cpp for intent parsing, and Piper for voice output, landing at roughly two to three seconds from spoken request to robot motion. Rather than letting the model drive hardware directly, a state machine sits between the two: the LLM's job is only to map loose natural language onto a fixed command vocabulary covering drive modes, head and arm actions, and a hard safety stop. That boundary means an unexpected model output becomes a rejected command instead of unplanned motion.

The arm is a servo-driven four-degree-of-freedom design covering shoulder, elbow, wrist, and gripper, with joint torque sized against worst-case fully extended load and driven through a PCA9685 controller on a supply rail isolated from the Pi to keep servo current draw from browning out the compute. Mounts and enclosure parts were modelled and 3D printed to integrate the microphone array, speaker, and power hardware for bench operation.

Status: Neck dynamics and arm kinematics working independently on the bench. Current work is integration onto the tracked base, then Voice pipeline.