2025-07-16 · 131 citations · club pick

EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos

Ruihan Yang, Qinxi Yu, Yecheng Wu, Rui Yan, Borui Li, An-Chieh Cheng, Xueyan Zou, Yunhao Fang, Xuxin Cheng, Ri-Zhao Qiu, Hongxu Yin, Sifei Liu, Song Han, Yao Lu, Xiaolong Wang

arXiv abstract PDF HTML Code DOI

No peer-reviewed venue on record yet. 131 citations, 3 of them influential, as of the last refresh.

Abstract

Real robot data collection for imitation learning has led to significant advancements in robotic manipulation. However, the requirement for robot hardware in the process fundamentally constrains the scale of the data. In this paper, we explore training Vision-Language-Action (VLA) models using egocentric human videos. The benefit of using human videos is not only for their scale but more importantly for the richness of scenes and tasks. With a VLA trained on human video that predicts human wrist and hand actions, we can perform Inverse Kinematics and retargeting to convert the human actions to robot actions. We fine-tune the model using a few robot manipulation demonstrations to obtain the robot policy, namely EgoVLA. We propose a simulation benchmark called Ego Humanoid Manipulation Benchmark, where we design diverse bimanual manipulation tasks with demonstrations. We fine-tune and evaluate EgoVLA with Ego Humanoid Manipulation Benchmark and show significant improvements over baselines and ablate the importance of human data. Videos can be found on our website: https://rchalyang.github.io/EgoVLA

arXiv comment: More videos can be found on our website: https://rchalyang.github.io/EgoVLA

Ten-minute slide kit

Six slides is the whole talk: what was broken, what people tried, what these authors did, what the numbers say, where it falls over, and the sentence people should remember.

SLIDE 1

The problem

There has been a vast advancement in robotic manipulation in the last few years, thanks to large-scale real robot data collection \[1, [2\]](#page-8-0). Compared to approaches that leverage simulation, directly performing supervised learning with real robot data avoids the Sim2Real domain gap and easily increases the task complexity. To efficiently collect complex robot manipulation data, multiple teleoperation tools with joint mapping \[3, 4, [5\]](#page-8-0), exoskeleton \[6, 7, [8\]](#page-8-0), and VR devices…

SLIDE 2

What came before

Dexterous Manipulation: Research in dexterous manipulation has progressed from control-based methods \[19, 20, 21, 22, 23, 24, [25\]](#page-9-0) to learning-driven approaches \[26, [27\]](#page-9-0). While early work emphasized precision, generalization across diverse scenarios remained limited. Learning-based methods introduced pose vector generation \[28, 29, [30\]](#page-10-0), intermediate representations \[31, [32\]](#page-10-0), and contact maps \[33, [34\]](#page-10-0), but large-scale dexterous…

SLIDE 3

The method

We build EgoVLA on top of a vision-language model to leverage strong visual and semantic reasoning. Specifically, we use *NVILA-2B*[\[67\]](#page-12-0) as the backbone for its robust vision-language understanding and compact size, enabling both intention inference and efficient fine-tuning. As shown in Fig.2, EgoVLA takes as input current and historical egocentric visual observations, language instructions, action query tokens, and human proprioception. These inputs are encoded by the VLM backbone and further…

SLIDE 4

What they measured

Beyond data scarcity, a major challenge in learning-based robotics is the lack of scalable, robust, and reproducible evaluation. Real-world evaluation is often costly, time-consuming, and raises concerns around safety and reproducibility—barriers that disproportionately affect resource-constrained settings such as academic labs. Recent work [\[69\]](#page-12-0) has shown that simulation-based evaluations are highly correlated with real-world performance, supporting their use as a reliable proxy. To enable…

SLIDE 5

Where it breaks

Our pretraining framework requires human data with hand and wrist pose annotations, which may limit data availability. However, the increasing accessibility of high-fidelity AR/VR devices (e.g., Quest 3, Vision Pro, Aria Glasses) is expected to ease this constraint. Additionally, although EgoVLA is pretrained with a unified action space, it cannot be directly deployed for manipulation without further fine-tuning on a moderate amount of robot

SLIDE 6

One-line takeaway

This paper explores training Vision-Language-Action (VLA) models using egocentric human videos using EgoVLA and fine-tune and evaluate EgoVLA with Ego Humanoid Manipulation Benchmark and show significant improvements over baselines and ablate the importance of human data.

Assembled from the paper's own PDF, parsed with its layout intact, 76,513 characters of it, then split on the paper's own section headings. Extractive, not generated: every sentence here is lifted from the paper. Check it before you present it.

Figures worth putting on a slide

Figure 1: EgoVLA. Our vision-language-action model learns manipulation skills from egocentric human videos and transfers them to a bimanual humanoid robot. The top row illustrates
Figure 2: EgoVLA takes visual history, language instruction, and action query token as input. The latent features are converted to human action with the action head. We use the wri
Figure 3: Human Data Following insights from language model and vision-language model training, we emphasize the importance of dataset structure in driving model performance. We co
Figure 4: Unified Action Space: MANO hand parameters are used as a shared action space for humans and robots. For robot hands, during training, optimized mano parameters produce th
Figure 5: Task Visualization. All simulated tasks with predicted wrist trajs from EgoVLA.
Figure 6: Visual Instruction Following. Top: original HOI4D samples with corresponding language instructions. Red lines indicate ground-truth human wrist trajectories, and green li

Presented at

Saturday, June 20, 2026

Saturday Robotics, session 13

Robotics & World Models Reading Club 13: HumanEgo: Train Robot Policy from 30 min Egocentric Videos — SF 0620

San Francisco, CA