2025-05-26 · 41 citations · club pick

EgoZero: Robot Learning from Smart Glasses

Vincent Liu, Ademi Adeniji, Haotian Zhan, Siddhant Haldar, Raunaq Bhirangi, Pieter Abbeel, Lerrel Pinto

No peer-reviewed venue on record yet. 41 citations, 1 of them influential, as of the last refresh.

Abstract

Despite recent progress in general purpose robotics, robot policies still lag far behind basic human capabilities in the real world. Humans interact constantly with the physical world, yet this rich data resource remains largely untapped in robot learning. We propose EgoZero, a minimal system that learns robust manipulation policies from human demonstrations captured with Project Aria smart glasses, $\textbf{and zero robot data}$. EgoZero enables: (1) extraction of complete, robot-executable actions from in-the-wild, egocentric, human demonstrations, (2) compression of human visual observations into morphology-agnostic state representations, and (3) closed-loop policy learning that generalizes morphologically, spatially, and semantically. We deploy EgoZero policies on a gripper Franka Panda robot and demonstrate zero-shot transfer with 70% success rate over 7 manipulation tasks and only 20 minutes of data collection per task. Our results suggest that in-the-wild human data can serve as a scalable foundation for real-world robot learning - paving the way toward a future of abundant, diverse, and naturalistic training data for robots. Code and videos are available at https://egozero-robot.github.io.

Ten-minute slide kit

Six slides is the whole talk: what was broken, what people tried, what these authors did, what the numbers say, where it falls over, and the sentence people should remember.

SLIDE 1

The problem

Robots face significant challenges in replicating human generality and dexterity in the physical world. While deep learning has fueled progress in domains like language \[1, [2\]](#page-10-0), vision \[3, 4, 5, 6, [7\]](#page-10-0), <sup>∗</sup>Correspondence to Vincent Liu: vincent.liu15@gmail.com speech \[8, 9, [10\]](#page-10-0), and complex games \[11, [12\]](#page-10-0), these successes rely on internet-scale datasets that are tightly aligned with downstream applications. In robotics, collecting similarly…

SLIDE 2

What came before

Imitation learning has emerged as a powerful paradigm in robotics, enabling robots to acquire complex skills by learning directly from real-world demonstrations [\[28\]](#page-12-0). By observing and replicating expert behavior, robots can bypass the need for hand-engineered solutions to manipulation tasks, making this approach particularly conducive to domains with high-dimensional state and action spaces \[29, [30\]](#page-12-0). Teleoperation is one of the most widely used methods for imitation learning from…

SLIDE 3

The method

Not recoverable from the parsed text. Read this section in the paper yourself.

SLIDE 4

What they measured

In this section, we compare EGOZERO with baselines adapted from related works and ablate some of EGOZERO's core components. From these comparisons, we demonstrate how our specific design choices make zero-shot in-the-wild transfer possible. We also explore the generalization properties that emerge from EGOZERO's unified state-action representation space. We evaluate EGOZERO on a Franka Panda gripper

SLIDE 5

Where it breaks

The largest source of error during inference comes from the correspondence model DIFT [\[61\]](#page-14-0). Correspondence encodes pose by ordering the state space, making policy learning sample efficient \[27, [16\]](#page-11-0). At larger data scale, pose information can be learned directly from dense unordered geometric information (i.e. using grounded segmentation models

SLIDE 6

One-line takeaway

EgoZero is a minimal system that learns robust manipulation policies from human demonstrations captured with Project Aria smart glasses, and suggests that in-the-wild human data can serve as a scalable foundation for real-world robot learning.

Assembled from the paper's own PDF, parsed with its layout intact, 54,532 characters of it, then split on the paper's own section headings. Extractive, not generated: every sentence here is lifted from the paper. Check it before you present it.

Figures worth putting on a slide

Figure 1: EGOZERO trains policies in a unified state-action space defined as egocentric 3D points. Unlike previous methods which leverage multi-camera calibration and depth sensors
Figure 2: Our 7 tasks. Top: open oven door, put bread on plate, sweep board with broom, erase board. Bottom: sort fruit, fold towel, and insert book in shelf. See Appendix [A](#pag
Figure 3: Distribution of bread keypoints for "Put bread in plate" task. The columns are projections of the 3D space onto each 2D plane. The policy generalizes to object poses far
Figure 4: Object semantic generalization. Human demonstrations are done with only black ovens (top). The policy transfers zero-shot to the robot with the same oven (middle) and als
Figure 6: Put bread on plate.
Figure 7: Sweep board with broom.

Presented at

Saturday, June 20, 2026

Saturday Robotics, session 13

Robotics & World Models Reading Club 13: HumanEgo: Train Robot Policy from 30 min Egocentric Videos — SF 0620

San Francisco, CA