Speaking to the Edge: Conversational Control of Smart Home IoT with LLM AI and Deterministic Gatekeeping

Smart homes inject additional control layers into domestic objects, often using sensor actuator-microcontroller devices which, when networked, have come to be known as the Internet of Things (IoT). Voice control has increasingly been applied as a user interface here, with systems like Amazon Echo streaming spoken command audio out to cloud-based processing for analysis. Control statements are returned to effect home device state changes, with a spoken output generated and delivered over a smart speaker or similar.

Previous versions of this technology have suffered the brittleness problem, where "expert" or "knowledge-based" systems perform acceptably in controlled conditions but fail rapidly given command diversity. The advent of Large Language Models (LLMs) and transformer architectures in speech and language processing has seen flexible, powerful reasoning capabilities become widely available, and brought about the possibility of robust conversational control of smart homes (and other environments), constituting a new wave of Artificial Intelligence (AI) technology.

The current state of the art, then, is conversational control via cloud-based services that use the new AI. For smart home control, there are two significant problems with this: the negative security and privacy implications of cloud-based data processing from the home; the unreliability of remote services given older domestic systems that set a high standard in this respect. This motivates the question: what happens if the cloud is replaced with locally-computed conversational control?

This thesis addresses that question via six contributions: (1) a taxonomic classification of smart home technologies; (2) a dataset of smart home commands, capturing diverse interaction scenarios; (3) a comparative ASR evaluation under smart home conditions; (4) an analysis of local LLM reasoning for smart home control; (5) a deterministic safety mechanism, validating LLM-generated control outputs; (6) an integrated and evaluated locally-operated smart home assistant, demonstrating its feasibility as a flexible, privacy preserving alternative to cloud-based assistants.