Smart speakers peaked at setting pasta timers and playing Spotify playlists.

If you ask them to coordinate your household, you hit a brick wall. They don’t know who is picking up our daughter from gymnastics. They don’t know that my wife already stopped by the grocery store. And if you try to build a custom assistant using commercial cloud APIs, you face an uncomfortable tradeoff: handing your family’s daily schedules, private photos, and text messages over to a vendor. And tokens are far from free.

A week ago, I wrote about our hybrid LLM setup where cloud models do my software development while a dedicated local server runs our home. I mentioned that our local models power an evolving family agent.

Meet Sadie.

Sadie the cardinal, our family AI mascot

Sadie is our private family AI assistant. She is named after a cardinal (a familiar neighbor here in Raleigh, NC), and her job is simple: take the friction out of running a busy household without compromising our privacy and being a not-small monthly bill.

She is not a commercial gadget. She is a customized, almost entirely local platform built on open-source foundations.

Here is how she works, what she does, and why keeping her local changes the entire equation.

The Nuance of Personal Software

The prevailing narrative says that building an AI assistant requires billions of dollars of cloud infrastructure. You either rent a generic chatbot from a tech giant, or you settle for a brittle smart speaker that mishears your timer requests.

The reality is more nuanced.

A family does not need a trillion-parameter frontier model to organize its life. A family needs orchestration.

When an assistant has deep context about your home, your calendar, and your routines, a well-tuned 27B or 35B open-weights model running on a quiet box in your office will beat a cloud model every single time. It beats it on latency, it beats it on cost, and it beats it on trust.

Family life is intimate. It involves school records, medical appointments, travel plans, and photos. That information does not belong in a cloud training dataset. Even with Zero Data Retention - why take the risk?

By building Sadie as personal software, we get an assistant with full access to our digital lives, bounded strictly by our home network.

The Engine Room: Dedicated Inference

Sadie does not run her own LLM inference inside her web container. She talks directly to our dedicated home inference server over our home network. They’re both hard-wired boxes.

That server is an Infplane Hilbert box powered by an AMD Ryzen AI MAX+ 395 APU with 128 GB of unified memory. We allocated 96 GiB directly to dedicated VRAM in firmware, backed by Mesa 26.0.3 and the open-source RADV Vulkan driver on Ubuntu 26.04 LTS.

Over llama.cpp and a lightweight router called llama-swap, the box hosts a persistent fleet sized specifically for our three-person family:

  • Chat (Qwen3.6-35B-A3B at Q8_0): Our primary conversational model. It is a sparse Mixture of Experts (MoE) that generates tokens at roughly 54 tokens per second across three concurrent slots with 262k native context.
  • Hard Reasoning & Vision (Qwen3.8-27B at Q4_K_M): The heavy lifter for planning, complex troubleshooting, and document analysis. With Multi-Token Prediction (MTP) speculative decoding, it clocks ~29 tokens per second.
  • Task Utility (Gemma-4-E4B): A lightning-fast 62 t/s worker for classification, summarization, tag generation, and quick title rewriting.
  • On-Demand RAG (Qwen3-VL-Embedding-2B & Qwen3-Reranker-0.6B): Memory and retrieval models loaded dynamically with an idle timeout to keep VRAM available for conversations.
  • Speech-to-Text (Whisper large-v3-turbo): Running on a separate port via whisper.cpp, transcribing family voice inputs at 22x real-time speed.

The inference box is an appliance. It serves OpenAI-compatible REST endpoints, exposes an SSH tunnel through Cloudflare for remote administrative maintenance, and stays quiet.

Sadie lives on a separate host on the network and uses that engine room to do her work.

Forking Open WebUI

Rather than reinventing a chat interface from scratch, we started with Open WebUI (OWUI).

Open WebUI has a clean license structure that permits forks and custom platforms for teams with fewer than 50 users. For a three-person family, it is an ideal starting scaffold: robust authentication, chat histories, role management, and extensible tool integration out of the box.

We took the core of OWUI and branched it. We stripped out the generic multi-tenant plumbing and tailored the frontend and backend services specifically for our household.

First and foremost: security. Sadie is strictly gated to our family’s OAuth accounts. There are no guest logins, no anonymous signups, and no public endpoints. If you aren’t one of the three of us, you can’t reach her.

Second: tool specialization. We wired Sadie directly into our domestic infrastructure.

Here are the core capabilities we have built so far.

# Sadie's integration topology
agent:
  identity: "Sadie"
  mascot: "Female Cardinal"
  auth: "Family OAuth Only"
  inference_host: "http://192.168.1.43:8080/v1"
  stt_host: "http://192.168.1.43:8081/v1"
  services:
    - homeassistant
    - immich_datastore
    - unified_calendar
    - bidirectional_sms_email

1. Tight HomeAssistant & Zone Reminders

Sadie has first-class access to HomeAssistant. She can inspect sensor states, toggle lights, verify if doors are locked, and check environmental monitors.

The killer feature here is location-aware reminders.

HomeAssistant already tracks our family’s mobile devices across geographical zones: home, work, school, grocery stores, and our gym. Sadie connects our reminder lists to those state changes.

If I tell Sadie on Tuesday: “Remind me to pick up air filters when I leave the office on Friday,” she does not just set a dumb alarm. She registers a trigger in HomeAssistant. The moment my phone transitions from the work zone to away on Friday afternoon, Sadie pushes an urgent notification to my phone.

2. A Single, Unified Family Calendar

Family scheduling is a classic coordination failure. Everyone has their own calendar, school events get buried in emails, and people show up at the wrong place.

Sadie manages a single, authoritative family calendar. But she maintains explicit metadata about who is attending each event.

When you ask Sadie: “What does tomorrow look like?”, she does not dump twenty unrelated meetings on you. She tailors the summary:

  • What the household has together (family dinners, school pickup).
  • What you specifically need to do.
  • Where coverage is missing (for example, if both parents have conflicting commitments during soccer practice).

3. Bi-Directional SMS and Email

An assistant that only lives in a browser tab is useless when you are standing in a hardware store parking lot.

Sadie has two-way integration over SMS and email:

  • Inbound: Any of us can shoot Sadie a quick text or email. “Add 2x4s and wood glue to the garage list,” or “What time is the vet appointment on Thursday?” Sadie parses the message, executes the tool or database lookup, and replies immediately via SMS.
  • Outbound: When an automated trigger fires (a zone exit reminder, a high priority school notice, or a laundry alert), Sadie reaches out over our preferred channel.

4. Immich Photo Integration

Photos are among the most sensitive data a family owns. We run Immich as our self-hosted companiton to Google Photos. Each family member has their own Immich account, and our phones automatically back up camera rolls to our home server.

Normally, Immich isolates user libraries for privacy. But Sadie integrates directly at the database and datastore layer.

With read access across the underlying storage, Sadie can search across the entire family’s visual catalog:

  • “Find pictures from our trip to the Blue Ridge Mountains where we were hiking by the waterfall.”
  • “Show me the photos of our dog wearing the funny birthday hat from two years ago.”

Because our inference box has a multimodal model and local vector embeddings, Sadie can parse visual queries and locate the exact shots without sending a single byte of our family photos to an external cloud API.

5. Automated Housekeeping & Inbox Triage

Sadie runs scheduled background workflows to clean up clutter.

She reviews household notification inboxes, discards promotional junk, extracts tracking numbers for incoming packages, and flags genuine deadlines (like school registration forms or utility bills). She logs the key dates onto the calendar and archives the raw emails.

The Pragmatic Boundary: Outsourcing Media

I believe in running local software wherever it counts. But pragmatism means recognizing when local hardware is the wrong tool for the job.

Sadie is almost 100% local, but we deliberately outsource image and video generation to cloud APIs.

Running high-end image diffusion models (like Flux or Stable Diffusion XL) or generative video models locally requires massive VRAM spikes and consumes significant compute. If we kept an image generation model resident on our Hilbert box, it would displace our 35B chat model or evict our reasoning context.

We generate text, process logic, transcribe speech, parse embeddings, and read documents locally. When someone in the family asks Sadie to generate a fun birthday card graphic or an illustration, she makes a quick, one-off call to a hosted image API, returns the asset, and gets back to business.

Don’t let ideological purity ruin a practical system. Outsource the heavy rendering; keep the private data at home.

What’s Next on the Roadmap

We are treating Sadie as our living laboratory for family automation. She is already handling our daily schedules and reminders, but a few major milestones are queued up:

  1. Instacart Integration: We want Sadie to monitor our weekly meal plans, maintain an active pantry inventory, and automatically populate an Instacart cart for review and checkout.
  2. Dedicated Smart Speakers: We already run speech-to-text with Whisper on :8081 and fast text-to-speech pipelines. The next logical step is placing custom, privacy-first microphones and speakers in the kitchen and common areas. We want to replace commercial smart pucks with dedicated hardware wired straight to Sadie.
  3. Pantry Management: We’re not sure what this might look like, but we already have a tool to scan barcodes easily. So it’ll take some sort of shape.

Building for the People Who Matter

There is a distinct kind of craftsmanship in building software for your own home.

When you build an enterprise tool, you measure success with DAUs, retention graphs, and conversion funnels. When you build software for your family, the feedback loop is immediate and unfiltered. If a reminder fires late, or if a calendar conflict gets missed, you hear about it over breakfast.

Sadie is proving that local AI is ready for prime time. By pairing a capable home server with open-source software and tight local integrations, you can build an assistant that is smarter, faster, and infinitely more respectful of your family’s privacy than anything you can buy off a retail shelf.

The tools are already here. It is just a matter of orchestration.