Sectra
Sectra is a voice-first AI assistant Oliver is building from scratch — wake word, speech, vision, and computer control, running locally on his own machine.
80
Object classes detected live
0
Per-request API cost
4
Parametric CAD primitives
- Custom-trained "Hey Sectra" wake word — not a generic pre-trained phrase
- Hunted a real speech-to-text hallucination bug down to the exact VAD/Whisper interaction causing it, then verified the fix with a synthesized round-trip test
- Live camera object detection (COCO-SSD, 80 classes) running fully client-side
- Generates parametric 3D models with geometry checked against closed-form volume math
- Computer control is real but permission-scoped — consequential actions confirm first
- Runs on local models via Ollama end to end — no per-request API cost
Problem
Sectra replaces an assistant Oliver built earlier and never actually used. The bar this time was explicit: "this needs to be so good that I genuinely use it" — which meant fixing root causes as they surfaced instead of shipping around them.
Approach
When speech-to-text started hallucinating words that were never said, the fix was diagnosing the exact interaction between VAD silence-padding and Whisper's decoder, not swapping models and hoping. The build follows one rule throughout: no faked capabilities. If a feature isn't wired up yet — PCB design is still deferred — the interface says so honestly instead of showing a mockup that only looks like it works.
Outcome
Live object detection running in the renderer, parametric CAD validated against exact geometric formulas, and tool-calling that lets the assistant actually act on the computer, gated behind a confirmation step for anything consequential.
Next
AI Agent Supervisor