Hey Boo: A Voice-First AI Operating System for Home and Life Automation

1. Abstract

Hey Boo is BOOJEE's voice-first AI assistant — the primary user interface to the Boojee ecosystem and to a person's entire digital life. The guiding design philosophy is simple: say Hey Boo, and it does the rest. Rather than a narrow-purpose smart-speaker skill or a walled-garden cloud assistant, Hey Boo is architected as an intent-routing operating layer that dispatches commands to the appropriate specialist: a Home Assistant hub for smart-home control, a Boojee service for business tasks, or a specialist AI agent for complex multi-step work.

This white paper documents Hey Boo's architecture, design principles, current capabilities, honest phased build path, and comparison with major competing voice assistants. It is written to be genuinely useful — not to market vaporware. Claims marked LIVE represent deployed, working functionality. Claims marked ROADMAP represent planned features that do not yet exist.

2. The Problem: Fragmented, Cloud-Locked Assistants

2.1 The Fragmentation Tax

The average person in 2026 manages five or more digital environments that do not speak to each other: a smart-home platform (Google Home, Amazon Alexa, Apple HomeKit), a business productivity suite, a personal calendar, a set of service applications (cleaning, food delivery, transportation), and a growing collection of AI tools. Each requires a separate app, a separate login, a separate mental model. The cognitive overhead of this fragmentation is enormous.

Voice assistants have been positioned as the solution to this fragmentation problem, but they have instead become another siloed product category. Google Assistant is being deprecated in favor of Gemini — a general-purpose chatbot that is not optimized for home control. Amazon's Alexa is deeply tied to Amazon's commercial ecosystem and processes all voice commands in Amazon's cloud. Apple's Siri is locked to Apple hardware and Apple services.

2.2 The Cloud Dependency Problem

Every dominant voice assistant streams audio from your home to corporate cloud infrastructure for processing. This creates three categories of risk that are rarely discussed in marketing materials:

• Privacy risk: Voice recordings of conversations in your home — including ambient sound before and after a wake word — are transmitted to third-party servers. Multiple voice assistants have used these recordings for human review and AI training without explicit user consent.
• Reliability risk: If the cloud service experiences an outage, your smart home stops responding. This is not a hypothetical — Google Home, Amazon Alexa, and Apple HomeKit have all experienced notable outages that rendered users' home devices unresponsive.
• Deprecation risk: Google has already deprecated Google Assistant for Gemini. This is the first of what will likely be many instances of a major cloud assistant making a product decision that breaks users' workflows. When you are dependent on a cloud assistant, you are dependent on that company's strategy.

2.3 The Extensibility Wall

Existing voice assistants have skills or actions ecosystems that allow third-party extensions, but these are constrained by the platform provider's business interests, approval processes, and capability limits. Building a genuinely custom action — one that connects your voice commands to your own infrastructure, your own business logic, your own data — is effectively not possible on any of the major commercial platforms.

Hey Boo is designed to solve all three problems simultaneously: local-first smart-home control (no cloud dependency for device commands), an extensible adapter/plugin registry (any capability can be added as an adapter), and a business integration layer that connects your voice to the full Boojee ecosystem.

3. The Vision: Voice as the OS for a Human Life

3.1 Boojee as an Operating System

The Boojee ecosystem is built on a single organizing principle: an operating system for a human life. Just as a computer operating system abstracts away the complexity of hardware and provides a unified interface for applications, Boojee is designed to abstract away the complexity of running a modern life — property, business, health, finances, logistics, relationships — and provide a unified interface to all of it.

Hey Boo is the primary user interface to this OS. Not an app. Not a dashboard. A voice. The idea is simple but radical in its implications: you should never need to think about which app to open. You should just say what you need and the system should figure out how to fulfill it.

3.2 Humans Decide, AI Executes

The Boojee design philosophy is explicit about the relationship between human judgment and AI execution. Humans are responsible for decisions: what to build, what to buy, what to prioritize, what to allow. AI is responsible for execution: how to carry out the decision, which specialist to delegate to, how to confirm completion, how to handle errors.

This is different from the "AI decides for you" paradigm that is emerging in some AI products. Hey Boo does not make spending decisions, book appointments without confirmation, or take irreversible actions without explicit authorization. Approval gates are a core design principle — not an optional feature.

3.3 The Tip of the Spear

In BOOJEE's product hierarchy, Hey Boo occupies a unique position: it is the tip of the spear. Every other Boojee product — AI Front Desk, Boojee Clean, Boojee Booking, Boojee Check-In, Boojee Menus, Boojee Marketplace — is accessible through Hey Boo. The user does not need to know these products exist by name. They say what they need and Hey Boo routes the command to the right service.

This architecture has a compounding property: every new Boojee service that is added makes Hey Boo more capable without requiring the user to learn anything new. The voice interface stays constant; the capability set expands underneath it.

4. Architecture: The Hey Boo Pipeline

4.1 The Voice Pipeline

Hey Boo's voice pipeline consists of four stages: voice capture, intent understanding, action dispatch, and voice confirmation. Each stage is designed to be modular and replaceable — the STT (speech-to-text) layer can be swapped without changing the intent understanding layer, and so forth.

4.2 Voice Capture (STT)

In the current implementation, voice capture uses the Web Speech API available in modern browsers (Chrome, Edge, Safari). This allows Hey Boo to run in any browser without requiring a native app installation. The user taps a microphone button (or uses keyboard shortcut), speaks their command, and the Web Speech API transcribes it to text.

The web-based approach has a practical limitation: it requires a button tap to initiate listening — it cannot do always-on wake-word detection in a browser tab. The roadmap includes a Progressive Web App (PWA) with a persistent background service worker for always-on detection, as well as hardware speaker support. These are clearly labeled as roadmap items; the current implementation does not claim hands-free wake-word support.

4.3 Intent Understanding

The intent understanding layer has two components that work in parallel: a local JavaScript classifier (the HeybooIntents registry) and a Lambda-backed cloud brain.

The local classifier is a deterministic intent-matching module (assets/heyboo-intents.js) that routes known command patterns without a network round-trip. This ensures fast response for common commands even if the cloud brain is unavailable. It handles smart-home scenes, Boojee service routing, capability queries, and multi-intent plans.

The Lambda brain (AWS Bedrock Nova) handles ambiguous commands, multi-turn conversation, and intents that require understanding context from the session. It is called first (4-second timeout) with the local classifier as a fallback. If the Lambda brain returns a recognized intent, that takes precedence over the local classifier's result.

4.4 Action Dispatch

Action dispatch is the core of Hey Boo's value proposition. Once an intent is understood, the system dispatches to the appropriate handler:

• Smart-home scenes: Compose and send Home Assistant API calls to the user's hub. In the current deployment (no hub provisioned), these are shown as dry-run previews — the exact calls that would fire if a hub were present. This is an honest representation: the routing is real, the execution requires hardware.
• Boojee service routing: Navigate the user to the correct Boojee service page with relevant context parameters. The 29+ service registry is maintained in the HeybooIntents module and routes are live.
• Builder routing: Navigate to the AI Web Builder with any extracted context (business type, description) pre-populated.
• Unknown capture: Commands that don't match any registered intent are captured via the Boojee intake webhook with full context (command text, timestamp, session ID, page URL). Nothing is silently dropped.

4.5 Voice Confirmation (TTS)

Voice confirmation uses AWS Polly neural TTS (in cloud deployments) or the browser's Web Speech Synthesis API (in the current web implementation). The response is a short, natural-language confirmation of the action taken: "Routing you to Boojee Clean," "Here are the Home Assistant calls for your arrival scene," "I've captured that — your request is on its way to the team."

The voice is configured with a slightly slower rate (0.94) and slightly elevated pitch (1.04) to convey warmth and professionalism without sounding robotic. AWS Polly Neural voices are specifically chosen because they are substantially more natural than standard TTS voices.

5. Smart Home Integration via Home Assistant

5.1 Why Home Assistant

Home Assistant (HA) is the world's most widely deployed open-source home automation platform. It supports over 3,000 integrations covering virtually every smart-home device category: lights (Philips Hue, LIFX, Kasa), locks (August, Schlage, Yale), thermostats (Nest, Ecobee, Honeywell), appliances, security cameras, door sensors, presence detection, and much more.

Critically, Home Assistant runs locally on hardware in the user's home — a Raspberry Pi, a dedicated HA Green hub, or any Linux server. This means that once Hey Boo is paired with a Home Assistant instance, smart-home commands can be processed entirely on-premises with no cloud dependency.

The alternative — using commercial smart-home clouds (Google Home, Amazon Smart Home, Apple HomeKit) — introduces the cloud dependency and privacy concerns described in Section 2. Hey Boo's architectural choice to build on top of Home Assistant is a deliberate decision to prioritize local control and privacy over integration convenience.

5.2 The HA Integration Model

Hey Boo integrates with Home Assistant through the HA REST API. The integration requires two configuration values:

• HA_URL: The base URL of the user's Home Assistant instance (e.g., http://homeassistant.local:8123 for local network access, or an externally accessible URL for remote control).
• HA_TOKEN: A long-lived access token generated in the HA user settings. This token is stored in the Hey Boo Lambda function's environment variables — it is never exposed to the browser or included in API responses.

When a smart-home intent is recognized, Hey Boo composes the appropriate HA API calls. A "turn off all lights" command might result in a single call to the light domain's turn_off service with entity_id: all. An "arrive home" scene might call three or four services in sequence: unlock the front door, set the thermostat to the occupied temperature, turn on the entry lights, disarm the security system.

5.3 Supported Scene Categories

Hey Boo's current scene registry includes the following categories, each composed of multiple HA service calls:

• Arrival scenes: "Arrive home," "I'm home" — unlock, lights on, thermostat adjust, security disarm
• Departure scenes: "Leave home," "I'm leaving" — lock, lights off, thermostat eco, security arm
• Bedtime scenes: "Goodnight," "bedtime," "lights out" — all lights off, door lock check, thermostat night mode
• Morning scenes: "Good morning," "wake up mode" — gradual lights, thermostat day mode, coffee maker start
• Work scenes: "Work mode," "focus mode" — office lights to work level, do not disturb
• Grounds/outdoor: "Start the mower," "water the garden," "outdoor lights on"
• Security: "Arm the system," "check the doors," "lock everything"

5.4 Extensibility

The scene registry is not hardcoded — it is a JSON configuration that can be extended with custom scenes for any HA domain. Users can add custom scenes for their specific devices without modifying the core Hey Boo system. The adapter/plugin registry pattern (described in the Vision section) means that new HA integrations can be added as adapters without touching the intent routing layer.

Full Home Assistant integration documentation →

6. The Brain: Intent Routing and Memory

6.1 The Intent Classification Hierarchy

Intent classification in Hey Boo works through a three-layer hierarchy:

1. Lambda brain (cloud, primary): AWS Bedrock Nova processes the command with full session context and returns a structured intent result. This layer handles ambiguous commands, multi-turn context, and novel phrasings that don't match deterministic patterns.
2. Local classifier (client-side, fallback): A deterministic JavaScript classifier that matches against registered command patterns. Handles the majority of common commands without a network round-trip. Fast, reliable, offline-capable.
3. Unknown capture (last resort): Any command that neither layer can classify is captured via webhook. This ensures zero-dropout — every user command results in either an action or an explicit capture.

6.2 Memory Architecture

Hey Boo's memory system is layered, with each layer having different scope and persistence:

• Working memory (current turn): The command being processed and its resolved intent. Exists only for the duration of a single command.
• Session memory (30 minutes): The full conversation history within a browser tab. Stored in the Lambda brain with a 30-minute expiry. This enables multi-turn conversations: "Book a cleaning" → "For next Friday" → "Book it" works because Hey Boo remembers what "it" refers to.
• Long-term memory (30 days): User preferences, frequently used services, recent project context. Stored in DynamoDB with a 30-day TTL. Scoped per user — no cross-user data.
• Semantic graph (roadmap): A graph of relationships between entities in the user's life — properties, vendors, services, projects, contacts. This is the long-term vision that enables truly intelligent orchestration: Hey Boo understands not just your commands but the relationships between the things you manage.

6.3 The Bedrock Nova Choice

The current intent routing brain uses AWS Bedrock Nova. This was chosen for several pragmatic reasons: it is available without waiting for Anthropic model access approval, it is well-suited to classification tasks, and it runs in the user's own AWS account. The architecture is designed to allow a Claude-class model swap when Bedrock access is provisioned — the interface between the Lambda brain and the rest of the system does not change.

It is important to be honest about what this means: Nova is a capable intent router and classification model, but it is not as capable as the most advanced language models for complex reasoning tasks. The architecture accounts for this by designing Hey Boo as an orchestration layer rather than relying on the AI model to do everything — the model's job is to understand intent and dispatch, not to perform the work itself.

7. Boojee Ecosystem Integration

Hey Boo is the front door to every Boojee company and service. The intent routing system maintains a registry of 29+ Boojee services with their canonical URLs, descriptions, and keyword patterns. When a command matches a service intent, Hey Boo routes the user directly to the appropriate service with relevant context parameters.

7.1 Service Categories

• Home services: Boojee Clean (cleaning dispatch), Boojee Maintenance (contractor marketplace), Boojee Check-In (short-term rental kiosk), Boojee Menus (QR menu platform)
• Business tools: AI Front Desk (SMS/chat receptionist), Boojee Web Builder (AI site generator), Boojee Booking (appointment system)
• Financial: Boojee Tax, Boojee Signals (trading intelligence), Boojee Marketplace
• Personal: Boojee Health, Boojee PM (property management), Estate Log, Expert Line
• Legal & entity: Boojee Entity (LLC formation), Boojee Legal, Boojee Security (smart-home security consulting)

7.2 The Context Parameter Model

Hey Boo doesn't just route to services — it passes context. A command like "Book a cleaning on Friday" routes to Boojee Clean with the date parameter pre-filled. "Build me a site for my bakery" opens the Web Builder with "bakery" pre-loaded as the business type. This context extraction is handled by the local classifier's extractParams function and enriched by the Lambda brain for more complex commands.

8. Capabilities: What Hey Boo Can Do

8.1 Live Capabilities (Current Deployment)

• Smart-home scene routing (dry-run): Recognizes home automation commands and shows the exact HA API calls that would fire. Live execution requires a provisioned hub.
• Multi-turn conversational brain: Nova Lambda with 30-minute session memory. Context carries across turns in a conversation.
• 29+ Boojee service navigation: All services accessible by voice command. Routes with context parameters where applicable.
• Fuzzy intent matching: "boook a clening" correctly routes to Boojee Clean — typos and accent variations are handled.
• Multi-intent plans: "Clean my house and check my leads" produces a two-step sequential plan with individual navigation buttons.
• Unknown command capture: Every unrecognized command is logged via webhook with full context.
• Neural voice output: Polly TTS (cloud) / Web Speech Synthesis (browser) for voice confirmation.
• Command history: Last 8 commands displayed for replay, per session.

8.2 Roadmap Capabilities (Not Yet Live)

• Live smart-home control: Real device commands require HA_URL + HA_TOKEN provisioned in the Lambda environment. The routing layer is complete; the hub connection is pending hardware deployment.
• Always-on wake word: Hardware speaker or always-on PWA with persistent background detection.
• Agent orchestration: Hey Boo as CEO of specialist AI agents (Legal, Finance, Travel, etc.) with the audit contract.
• Proactive briefings: Background monitoring + morning summary ("what happened while I slept").
• Semantic memory graph: Graph of relationships between entities in the user's life.
• Mobile automation: Control of phone apps, notifications, GPS-triggered actions.

Full capabilities documentation →

9. How Hey Boo Compares

The following comparison is honest — we do not claim Hey Boo wins every dimension. We are a newer product with a smaller ecosystem. Where commercial assistants have genuine advantages (breadth of third-party integrations, mature hardware), we say so. Where Hey Boo has genuine advantages (privacy, local control, business integration, extensibility), we say so.

10. Privacy and the Local-First Model

10.1 Design Principles

Hey Boo's privacy model is built on four principles:

1. Local-first for home control: Smart-home commands are processed by the user's local Home Assistant instance. The only data that leaves the local network is the text transcript of the voice command — the actual device control happens entirely on-premises.
2. Your infrastructure, your data: Intent routing uses the user's own AWS Bedrock account. Session and long-term memory are stored in the user's own DynamoDB table. No shared SaaS database holds your data.
3. No training on your data: AWS Bedrock does not use customer data to train or improve its models. This is a documented commitment from AWS. Your commands are not used to train Nova or any other model.
4. Explicit permission gates: Every capability that Hey Boo can access — every device, service, or integration — must be explicitly configured. There are no implicit permissions. High-risk actions (spending, deleting, sending, booking) require approval confirmation.

10.2 What Data Leaves Your Home

To be precise about what data is transmitted when you use Hey Boo:

• The text transcript of your voice command (after local Web Speech API STT processing in the browser)
• A session ID (UUID, generated per browser tab, not linked to your identity)
• A timestamp
• The page URL (heyboo.boojee.estate/...)

Audio is not transmitted — the Web Speech API converts audio to text locally in the browser before sending. No voice recording is sent to the Hey Boo backend.

10.3 Current Limitations

Honesty requires acknowledging current limitations of the privacy model:

• The Web Speech API in most browsers is powered by Google's cloud STT service (in Chrome/Chromium) or Apple's cloud STT (in Safari). This means the audio does leave your device through the browser's built-in STT service — just not through Hey Boo's infrastructure. True local STT requires a different implementation (Whisper or similar running on-device).
• The Lambda brain uses AWS Bedrock Nova, which runs on AWS infrastructure. Commands are processed on AWS servers. AWS's privacy guarantees apply.

The roadmap includes investigating fully local STT (on-device Whisper) for users who require complete audio privacy. This is labeled as a future consideration, not a current capability.

11. Roadmap: Phased Build Path

Hey Boo is built on the principle of honest phasing: each phase ships something real and working before the next phase begins. No phase is announced as complete until it is deployed and verified.

12. Glossary