irene cheung

Product Designer

The capture dashboard behind Meta's Codec Avatars

Redesigning the internal tool that orchestrates 300+ camera and sensor streams, as the team's only designer and a daily operator of the system.

Internal ToolsHardware R&DShippedNDA Protected

Summary

Category

Internal Research Dashboard

Org

R&D Engineering & Spatial Computing

Product

Meta Reality Labs, Codec Avatars

Role

Only designer on a five-person project team

Core Challenge

  • 01 Real-time data with near-zero tolerance for latency
  • 02 Years of inherited design debt
  • 03 Ambiguous error-handling protocols
  • 04 Delayed sensor capture schedules

Impact

  • 01 Delivered a UI specification and multi-state verification guidelines for the engineering team
  • 02 Rapidly prototyped and user-tested a complex troubleshooting flow within one day

Hired to run the captures, I became the team's only designer and redesigned the dashboard I used every day.

The redesigned flows let operators resolve pipeline failures themselves instead of stopping a session to pull in engineers.

Tool

Figma

Design System, Prototyping

Miro

Card-Sorting, UX research synthesis

System Impact

Codec Avatars

Quest Pro

Stakeholder

  • TPM
  • Engineers
  • System Integrators
  • Research Operators

Background

Codec Avatars represent Meta's long-term research project to create photorealistic, real-time digital identities that use neural networks to mirror human expression and presence in VR.

Problem Space

One missed failure could cost the capture, or the whole day's schedule.

  1. 01 The old dashboard showed nearly all 300+ streams at once, with equal visual weight
  2. 02 Nothing told the operator what mattered right now
  3. 03 Camera lag, sync drift, and system-health errors surfaced late
  4. 04 A mid-session failure could invalidate the capture, and cascade into the day's back-to-back schedule

I knew exactly where it hurt, because I ran these sessions every day.

Comparison of Codec Avatar renders: relit 3D face captures above matching source photos of the same six people

Codec Avatar renders above the source photos of the same people. (source)

For a bit of context

This is the exact physical infrastructure I designed for, and operated daily. A capture session inside this dome runs 300+ synchronized camera and IMU streams at once.

Two fisheye views inside the MUGSY capture dome, showing rings of cameras surrounding an empty operator chair

Inside the MUGSY capture dome.

Grayscale camera-array capture grid showing a subject's face from many synchronized angles
Color camera-array capture grid showing a subject's face and shoulders from many synchronized angles

A subject seen by the synchronized camera array.

One dashboard had to work for three very different capture rigs.

Each rig produced different data and failed in different ways. A normal reading on one rig could mean trouble on another. I needed one interface that stayed legible across all three, without asking operators to hold three mental models.

How I ran it

Audit the existing dashboard → shadow and interview operators (myself included) → clickable prototypes → engineering review → operator testing → ship in increments

These are the questions I ask myself as I design. There are no standard procedures, always case by case. Indeed, in some work environments, like a leaner team, things get shipped without user testing. But in the case of Meta, the dashboard was not only used by the lab I worked for, it impacted multiple locations. I need to be able to challenge my design well before it breaks.

Pain point spotted in the labIs it repetitive?NOYESWas the one-timeimpact severe?YESNOTreat it as a risk:design a guardrailLog it and watchfor recurrenceIs it related tothe metrics?NOA metric is missing.Fix the measurement firstYESDoes it support thePM or engineer's claim?YESAttach the evidenceto the claimNOCan I reproduce it?NOKeep collectingevidenceYESShow it in the listener's language:metrics for the PM,feasibility for the engineerDESIGN PHASE

Scroll sideways to see the whole diagram.

I design technical tools in three layers, from the backend up

This project taught me to think in layers. The dashboard itself stays under NDA, so what I can show is the thinking: how I broke an unfamiliar, deeply technical system into pieces until I could see where design would help.

I was new to R&D when I started. Building this framework is how I stopped being new.

Data

What do the sensors see and feel? And what do they miss?

Telemetry dataEdge casesLatency
Logic

What does the system make of it all? What matters, and what's a no-no?

System statesGuardrailConfidence level
UX

And what does the operator get? What's happening, and what do I do next?

Error handlingExplainabilityPredictabilityAbility to overrideStakeholders' mental model

Becoming a better designer

Design became decisions the team could act on.

PMs, engineers, and designers speak different love languages. So I stopped presenting "UI" and started speaking the listener’s.

TPM
Design changes translated into the metrics they already tracked: capture throughput, failed sessions, time lost to errors.
Engineers
Feasibility first. No static handoffs. I brought clickable prototypes and iterated with them inside their constraints.

Read more

Tanmigo interface showing a tango video paired with an annotation panel for tagging embrace, footwork, and axis

tanmigo

A 0-to-1 video annotation platform for dancers

Bridging the gap between video discovery and physical practice with a seamless, one-stop annotation experience, designed and built in code with an engineer.

TypeScriptFront-End DevelopmentLive Product
Claude Code chat interface mid-conversation, showing a Bash tool call and a permission prompt to allow running git status

Claude Code

Developer experience in Claude Code

Restoring a sense of agency in AI coding tools, tested with a working prototype.

Agentic Developer ExperienceConceptPrototype