Cross-platform deployment and characterization of an on-device generative conversational agent

Recent advances in Language Models (LMs) enable conversational systems that interpret natural-language requests, select application actions, and generate user-facing responses. However, existing studies typically either examine complete assistants with limited deployment evidence or benchmark individual components in isolation. This work advances the state of the art through a unified cross-platform characterization of a local speech-to-text-LM-text-to-speech pipeline combining Moonshine, Piper, and four quantized LMs ranging from 1.2B to 8B parameters. The pipeline is deployed on an Nvidia Jetson AGX Orin, a Raspberry Pi 5, and an STM32MP257F-EV1, and evaluated in thermostat and washing-machine use cases. Action-selection capability is assessed on 200 labelled explicit and implicit requests per use case, while deployment is characterized through metrics such as memory footprint, speech Real-Time Factor, decoding throughput, power consumption, etc. Granite 3.3 (8B) achieves the highest Macro F1-scores, whereas Qwen3.5 (4B) provides the best capability–resource trade-off. Speech processing is faster than real time on the Jetson and Raspberry Pi, while the STM32MP2 supports smaller LMs in memory but is unsuitable for responsive CPU-only conversation. LM decoding accounts for over 90% of turn duration, and memory-Roofline analysis identifies model footprint, runtime support, response length, and streaming execution as the main optimization levers.