Haoru Xue | ่ๆตฉๅ
I am a PhD student at Berkeley AI Research (BAIR), UC Berkeley advised by Prof. Shankar Sastry. I also work with Prof. Trevor Darrell and Guanya Shi. I am affiliated with the EMBER Center.
I am interning at NVIDIA GEAR Lab supervised by Dr. Jim Fan and Prof. Yuke Zhu.
Previously I was a MS Robotics student at CMU LeCAR lab and DRIVE lab, working with Prof. Guanya Shi and John M. Dolan. In 2023 I was a visiting student with Prof. Francesco Borrelli at MPC lab, UC Berkeley.
I’m interested in foundemental robot learning problems, a.k.a. receipe for physical AGI: scalable priors, long-horizon reasonings, dexterous and agile motor skills.
From 2021 to 2024 I led AI Racing Tech ๐๏ธ, a multi-million $ autonomous racing research project in UCB, CMU and UCSD across 5 labs. I deployed real-world robot learning on F1-level self-driving race cars at 160 MPH. We are the top U.S. team in the Indy Autonomous Challenge. Learn more in my playground.
Latest
[12/2025] ๐ฎ New blog: Teleoperation-as-a-Service (TaaS)
[12/2025, NVIDIA] ๐จ New paper: DoorMan. Follow the release on X!
[11/2025, NVIDIA] ๐จ New paper: VIRAL. Follow the release on X!
[06/2025, UCB] ๐จ Thrilled to announce LeVERB. Follow the release on X!
[05/2024, NVIDIA] ๐ฆพ I am interning at NVIDIA GEAR Lab.
[05/2025, ICRA] ๐ฅณ DIAL-MPC is selected as Best Paper Finalist at ICRA 2025!
See earlier
[04/2025, ICML] ๐ One paper accepted at ICML 2025.
[02/2025] ๐ฎ New blog: Pathways to Data Scaling Law of Robotics Foundation Model
[01/2025, ICRA] ๐ Three papers accepted at ICRA 2025.
[10/2024, UCB] ๐จ Thrilled to announce AnyCar. Follow the release on X!
[09/2024, UCB] ๐จ Thrilled to announce DIAL-MPC โ๏ธ. Follow the release on X!
[08/2024, UCB] ๐ I joined UC Berkeley EECS PhD program (AI Robotics).
[05/2024, ICRA] (presentation | slides) Oral on “Learning MPC with Error Dynamics Regression for Autonomous Racing“.
[04/2024, CMU] (presentation | slides) Master thesis talk on “Optimal Control and Robot Learning on Agile Safety-Critical Systems“.
[04/2024, Stanford] Talk on “Towards Foundational Control Models for Agile Locomotion” at Stanford Intelligent Systems Laboratory (SISL).
[01/2024, UCB] Talk on “Safe Online Adaptation for Robots with Extreme Dynamics” at DARPA ANSR program.
[10/2023, UCB] Talk on “Learning MPC with Error Dynamics Regression for Autonomous Racing” at Model Predictive Control (MPC) Lab.
Research
Ultimately, I want to build generalist robots that will fundamentally change the human society. Recently, these are my research interests:
- Enabling long-horizon reasoning and generalization on VLM for robotics
- Receipe for RL post-training on VLAs
- Bridging high-level vision-language planning and low-level whole-body motor skills on loco-manipulation tasks.
Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer
Haoru Xue*, Tairan He*, Zi Wang*, Qingwei Ben, Wenli Xiao, Zhengyi Luo, Ye Yuan, Xingye Da, Fernando Castaรฑeda, Shankar Sastry, Changliu Liu, Guanya Shi, Linxi “Jim” Fan, Yuke Zhu
Dec 2025 | website | arxiv | social media
DoorMan proposes a teacher-student-bootstrap framework for challenging humanoid loco-manipulation tasks such as door opening. Trained as an RGB policy purely in simulation, it is up to 31.7% times faster than human in the real world.
VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation
Tairan He*, Zi Wang*,Haoru Xue*, Qingwei Ben*, Zhengyi Luo, Wenli Xiao, Ye Yuan, Xingye Da, Fernando Castaรฑeda, Shankar Sastry, Changliu Liu, Guanya Shi, Linxi “Jim” Fan, Yuke Zhu
Nov 2025 | website | arxiv | social media
VIRAL investigatesthe scaling law of visual sim-to-real. We find the right recipe to enjoy the free lunch of simulation: zero-shot, robust, continuous real-world deployment.
Self-Improving Vision-Language-Action Models with Data Generation via Residual RL
Wenli Xiao*, Haotian Lin*, Andy Peng, Haoru Xue, Tairan He, Yuqi Xie, Fengyuan Hu, Jimmy Wu, Zhengyi Luo, Linxi “Jim” Fan, Guanya Shi, Yuke Zhu
Oct 2025 | website | arxiv | social media
Probe, Learn Distill (PLD) is a recipe for VLA RL post-training using real-world data, letting robots discover, recover, and distill their own data flywheel.
LeVERB: Humanoid Whole-Body Control with Latent Vision-Language Instruction
Haoru Xue*, Xiaoyu Huang*, Dantong Niu*, Qiayuan Liao*, Thomas Kragerud, Jan Tommy Gravdahl, Xue Bin Peng, Guanya Shi, Trevor Darrell, Koushil Sreenath, Shankar Sastry
June 2025 | website | arxiv | social media
LeVERB is the first latent whole-body humanoid VLA. We introduce a latent vocabulary as an interface between vision-language and whole-body action to enable expressive task specification and interpolatable execution.
Full-Order Sampling-Based MPC for Torque-Level Locomotion Control via Diffusion-Style Annealing
Haoru Xue*, Chaoyi Pan*, Zeji Yi, Guannan Qu, Guanya Shi
ICRA 2025 Best Paper Finalist (Top 1%)
website | arxiv | code | social media
DIAL-MPC โ๏ธ : diffusion + sampling-based MPC can achieve adaptive and training-free whole-body control on legged systems using RL-style rewards. Deployed in real on agile quadrupedal torque control.
Pre-training Auto-regressive Robotic Models with 4D Representations
Dantong Niu*, Yuvan Sharma*, Haoru Xue, Giscard Biamby, Junyi Zhang, Ziteng Ji, Trevor Darrell, Roei Herzig
ICML 2025 | website | arxiv | social media
ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a robotics model that has stronger spatial and temporal understandings.
AnyCar to Anywhere: Learning Universal Dynamics Model for Agile and Adaptive Mobility
Wenli Xiao*, Haoru Xue*, Tony Tao, Dvij Kalaria, John M. Dolan, Guanya Shi
ICRA 2025 | website | arxiv | code | social media | IEEE Spectrum
AnyCar ๐๏ธ ๐ ๐ ๐ป ๐: a generalist dynamics model ๐ built with transformer + large-scale sim pre-training + small-scale real fine-tuning. Achieves agile and adaptive control on a family of wheeled embodiments (few or zero shot) and outperforms specialist policies.
Learning Model Predictive Control with Error Dynamics Regression for Autonomous Racing
Haoru Xue*, Edward L. Zhu, John M. Dolan, Francesco Borrelli
ICRA 2024 | website | arxiv | video | code | social media | talk
Use MPC + robot learning to explore optimal policy and dynamics model online safely. Perform Sim2Real and learn the handling limit of extreme driving like a professional race car driver. Deployed on a full-size race car!
Segment Anything Model for Road Network Graph Extraction
Congrui Hetang*, Haoru Xue, Cindy Le, Tianwei Yue, Wenping Wang, Yihui He
CVPR 2024 Workshop | arxiv | code
We propose a SAM-based pipeline for large-scale road network extraction from aerial image. Comparable accuracy with SOTA is achieved while being 40 times faster.
Best Paper (2nd Workshop on Scene Graphs and Graph Representation Learning)
Education
UC Berkeley EECS
PhD (AI Robotics) 2024
CMU Robotics Institute
MS Robotics 2022 – 2024
Prof. Guanya Shi, John Dolan
UC San Diego
Electrical Eng. 2018 – 2021
Experience
LeCAR Lab, CMU Robotics Institute
Visiting Researcher
May – Aug. 2024
Advisor: Prof. Guanya Shi
MPC Lab, UC Berkeley
Visiting Researcher
Apr. – Aug. 2023
Advisor: Prof. Francesco Borrelli
AI Racing Tech
Lead Graduate Researcher
Dec. 2019 – May 2024
Advisors: Prof. Shankar Sastry, John Dolan; Dr. Allen Yang, Jack Silberman
Awards & Honors
- [2025] Best Paper Finalist, ICRA 2025
- [2024] UC Berkeley EECS department scholarship recipient
- [2023] 3rd Place, Indy Autonomous Challenge at Las Vegas CES
- [2022] 2rd Place, Indy Autonomous Challenge at Texas Motor Speedway
- [2021] Henry G. Booker Memorial Honors Award, UC San Diego ECE Department