FUZZY INPUT
Led UX research and prototyping for an LLM-disambiguated in-cabin intent UI — from fleet telemetry across 21M+ daily transactions, through wizard-of-Oz studies in a cabin seat buck, to a functional Android prototype and a final demo on production head-unit hardware to leadership.
Conner Ward led UX research and prototyping for FuzzyInput, an LLM-disambiguated in-cabin intent UI at Mercedes-Benz, grounding its design in fleet telemetry across 21M+ daily voice transactions.
FuzzyInput treats every spoken utterance as fuzzy by design, resolving it in a three-region widget (what was heard, the smallest sufficient control, and a deferred follow-up). Conner ran the telemetry analysis and competitor audit, built the Figma framework, ran wizard-of-Oz cabin-buck studies with gaze tracking, and shipped a functional Android prototype that ran on production head-unit hardware against live cabin services in a demo to leadership. The work was done at Mercedes-Benz R&D North America's Sunnyvale AI Labs over roughly 14 months in 2024–2025.
Role
UX engineer / researcher — individual contributor with cross-team scope across design, ML, HU platform, and hardware.
Timeframe
~14 months · 2024–2025.
Collaborators
Design (Carlsbad), ML / ASR (Sunnyvale + Sindelfingen), HU component library (Germany + India), hardware (Germany rotator-knob team).
Scope
Telemetry analysis · competitor audit · Figma framework + design review · cabin-buck studies w/ gaze tracking · Android prototype · gRPC bridge into production HU services · executive in-vehicle demo.
What shipped
The R1/R2/R3 fuzzy-input widget framework — running on production HU hardware against live cabin services. The first end-to-end LLM-disambiguated intent surface inside the Mercedes HU contract.
Every utterance is fuzzy
Problem. In-cabin voice never reaches the cleanliness assumed by command-driven UI. Supplier ASR mishears under cabin noise, models run on intermittent connectivity, and the HU's component library was built for menu-driven flows, not free-form intent. Existing "AI" surfaces in luxury vehicles fail by either competing with primary task UI or by treating ambiguity as a failure mode.
Approach. Treat every utterance as fuzzy by design. Build a disambiguation surface — three regions, one shared interaction grammar — that a driver can confirm, correct, defer, or swipe past in a single half-glance gesture. Ground the widget vocabulary in fleet telemetry, not designer intuition. Validate in a cabin-buck with wizard-of-Oz ASR before committing engineering. Ship on real hardware with a gRPC bridge into the production HU stack.
The Framework
Three-region widget — R1 utterance, R2 affordance, R3 deferred action — adopted as the design contract for fuzzy-intent surfaces. Survived translation through 4+ downstream supplier teams.
Every fuzzy utterance lands in the same three-region card, with the same vocabulary regardless of domain. R1 shows what was heard with parsed entities inline-highlighted, so the driver sees what was understood without reading a paragraph. R2 is the smallest sufficient affordance for the inferred intent — slider, toggle, card, dial — drawn from the existing HU component library so the visual grammar is already learned. R3 is the deferred follow-up: surface the next likely action without forcing commitment.
The full action vocabulary collapsed to three verbs — jump, do, chat — applied to every parsed intent. Small surface, learned once, recognizable at speed.
Telemetry-Driven Scope
Cut the widget's first-class vocabulary from a proposed 30+ intents down to 5, on evidence. Stopped a much larger component-library scope discussion in one chart.
Pulled aggregate skill-action telemetry across the fleet and let the long tail decide which intents the fuzzy widget had to handle natively versus punt to chat. SeatHeater, SeatVentilation, AmbientLight, Camera/Parking, and SeatMassage dominate by an order of magnitude — those plus a navigation path became the widget's primary vocabulary. Everything else routes to the chat fallback because nobody uses it enough to justify a custom interaction.
Competitor & Indirect UX Audit
Direct automaker surfaces confirmed the gap — command-driven, not fuzzy-aware. Indirect mobile patterns (Siri Shortcuts, iOS notifications, social-feed swipe) supplied the gesture vocabulary drivers already carry — folded directly into R3.
Direct competitors mostly confirmed the gap. The actionable patterns came from indirect surfaces — mobile assistants and social UI — because that's where users have already trained the gesture grammar the driver carries into the cabin. The EVO map critique is the load-bearing example of the failure mode we needed to avoid: an AI surface that competes with primary task UI fails no matter how good the model is.
Figma Prototyping
Single living widget framework with variants covering 80%+ of fleet-observed intents. Reviewed by the design lead and adopted as the in-cabin contract for downstream platform teams.
Worked the framework end-to-end with the design team. Every variant scored against four heuristics: recognition-over-recall, consistency with existing HU grammar, error prevention (where cost-of-error is glance time, not a dialog), and minimalist aesthetic. Variants that scored badly on glance time got cut even when they tested well on a desktop.
Wizard-of-Oz · In-Cabin Buck
High-fidelity from session one, in a real seat buck under driving load with gaze tracking. The methodology call was validated by the design lead — glance-time data became the primary cut criterion for downstream Figma variants.
The common wisdom that you should test low-fi paper prototypes first is wrong for an established luxury HU context. Participants cannot evaluate a fuzzy-input widget against a hand-drawn surface because they cannot calibrate "would I trust this in a real cabin." Every study from session one used full-fidelity Figma click-throughs with a wizard-of-Oz AI backend (Whisper for ASR, a researcher driving the right state behind the curtain) so the participant was responding to something that looked and felt like the production target.
For the distractible-context tests, subjects sat in the cabin seat buck — full dashboard surfaces, target-vehicle seating geometry, projected forward-scene driving sim — with gaze tracking on and a secondary task running. The Figma click-through was driven from a researcher tablet so we could swap intents on the fly. Gaze data + secondary-task performance produced the glance-time numbers that design heuristics alone couldn't.
Functional Prototype
Validated the framework on real ASR — no wizard — with the cabin-canonical input device (physical rotator knob). Composed from production HU library primitives, so what tested on the bench was visually 1:1 with what would ship.
Ported the validated Figma flow onto an Android tablet running the real ASR pipeline. Bridged in a physical rotator knob — the cabin-canonical confirm/scroll/back affordance — over network from a hardware rig the Germany team had built. Having the knob on the bench meant we could test the widget against the real motor pattern, not a touch surrogate.
Rather than build a parallel UI stack, the prototype reaches into the production HU component library and composes the fuzzy widget from the same primitives — same slider, same card, same toggles. What tested on the bench was visually 1:1 with what would ship.
Production HU Demo
The first end-to-end LLM-disambiguated fuzzy-intent surface running inside the Mercedes HU contract — on production hardware, in a real vehicle, against live cabin services. Made the case to leadership for finishing the component library.
Migrated the functional prototype onto actual head-unit hardware for a demo to leadership. Manual Figma → Android translation (no shared design-token pipeline yet), fuzzy widget running inside an Android container, gRPC bridges into production HU services so the widget could actually toggle ambient light, set climate, and queue music against the real car.
Same Framework, New Substrate
Side experiment: same fuzzy-input + deferred-action grammar, but R2 is generated on the fly by an LLM with tool-use over an MCP server backed by an arbitrary app surface. Suggests the widget vocabulary is a UI-level abstraction, not an automotive-specific one — it transfers cleanly to any context where a user emits fuzzy intent and the system can hold tools.
Naming Fuzzy
A one-eyed felt monster named Fuzzy gave a dry research program a face — concept directions generated as a contact sheet with ChatGPT image generation in an afternoon, then the chosen direction fabricated as a physical plush.
The core idea — treat every utterance as fuzzy by design — wanted a mascot, both to make the framework memorable internally and to keep "fuzzy" front-of-mind in reviews. Rather than commission an illustrator for an artifact that didn't need to be precious, I used ChatGPT's image generation as a divergence engine: one prompt for a friendly one-eyed fuzzy monster, then iterate on tone, horns/no-horns, flat-vs-glossy, and logo-vs-character lockups. The grid below is the spread that came out of that session — each tile a different direction on the same character, generated and culled in a single sitting.
The friendly one-eyed direction won — approachable, a little goofy, and legible at favicon size. From there it jumped substrate: the chosen character was fabricated as a real blue felt plush with striped horns, the version shown animating in this grid. A digital concept sheet became a physical object that sat on the desk through the rest of the project.
What Held Up
What held up. The R1/R2/R3 framework, the deferred-action grammar, the telemetry-driven scope cut, and the high-fidelity wizard-of-Oz study methodology all survived the translation through multiple international supplier teams. Those are the pieces I'd bring forward unchanged.
What drifted. Pieces that depended on shared philosophy rather than shared spec. When design intent travels through two-line tickets and weekly syncs, the surface decision (use a slider here) arrives without the underlying constraint (because the driver cannot read a list at speed). Predictably, downstream reinterpretation goes maximalist — implementers add the picker, the swatches, the labels, the help icon, all the things the original design deliberately omitted, because absent the constraint they don't know which omissions were load-bearing.
Get the design philosophy itself onto the same wire as the design assets — a single living doc, versioned, that travels with every Figma file and every spec, so the receiving team can answer "why isn't there a picker here" without booking a sync. The widget vocabulary is small enough to fit on one page; the reason it doesn't is organizational, not design-driven.
FAQ
What is the FuzzyInput project?
FuzzyInput is a Mercedes-Benz in-cabin research project: an LLM-disambiguated intent UI that treats every spoken utterance as fuzzy by design and resolves it in a three-region widget — what was heard, the smallest sufficient control, and a deferred follow-up action. It was built at Mercedes-Benz R&D North America's Sunnyvale AI Labs (2024–2025).
What was Conner Ward's role on FuzzyInput?
Conner Ward led the UX research and prototyping for Mercedes-Benz FuzzyInput as a UX engineer / researcher with cross-team scope. That spanned fleet telemetry analysis across 21M+ daily transactions, a competitor audit, the Figma widget framework, wizard-of-Oz cabin-buck studies with gaze tracking, a functional Android prototype, and a final in-vehicle demo on production head-unit hardware to leadership.
What was the outcome and tech used?
FuzzyInput's R1/R2/R3 three-region widget became the design contract for fuzzy-intent surfaces at Mercedes-Benz and ran end-to-end on production head-unit hardware against live cabin services. The stack included fleet telemetry analysis, LLM-based intent disambiguation, real ASR (Whisper for the wizard-of-Oz studies), a Figma prototype, and an Android prototype bridged into production head-unit services over gRPC, with a physical rotator-knob input.