Haoru Xue

I am an incoming PhD student at Berkeley AI Research (BAIR), UC Berkeley. I am currently visiting LeCAR lab advised by Professor Guanya Shi at CMU.

Previously I was a MS Robotics student at CMU DRIVE Lab, working with Professor John M. Dolan and Guanya Shi. In 2023 I was a visiting student with Professor Francesco Borrelli at MPC lab, UC Berkeley.

I’m interested in humanoid, quaduped, and vehicle robots that can do agile and dexterous loco-manipulation. I also want to strategically interface them with foundation models for robotics to perform long-horizon tasks.

I also lead AI Racing Tech, a multi-million $ autonomous racing research project in UCB, CMU and UCSD across 5 labs with over 20 active researchers. I deploy real-world robot learning on extremely agile self-driving race cars at 200 MPH. We are the top U.S. team in the Indy Autonomous Challenge. Learn more about this endeavor in my playground.

Latest

[05/16/2024, ICRA] (presentation | slides) Oral on “Learning MPC with Error Dynamics Regression for Autonomous Racing“. (10:30-12:00 at AX – F206)

[04/24/2024, CMU] (presentation | slides) Master thesis talk on “Optimal Control and Robot Learning on Agile Safety-Critical Systems“.

[04/22/2024, Stanford] Talk on “Towards Foundational Control Models for Agile Locomotion” at Stanford Intelligent Systems Laboratory (SISL).

[01/26/2024, UC Berkeley] Talk on “Safe Online Adaptation for Robots with Extreme Dynamics” at DARPA ANSR program.

[10/09/2023, UC Berkeley] Talk on “Learning MPC with Error Dynamics Regression for Autonomous Racing” at Model Predictive Control (MPC) Lab.

[10/03/2023, IROS] Oral on “Spline-Based Trajectory Optimization for Autonomous Racing“

Research

Ultimately, I want to build genearlist robots that will fundamentally change the human society. Recently, these are my research interests on humanoids, quadupeds, and vehicles:

  • The Interface between high-level LLM/VLM/Diffusion planning and low-level RL policy is crutial for enabling both long-horizon and dexterous/agile loco-manipulation.
  • Agility and Adaptability goes hand-in-hand towards foundational control models that few-shot adapts to controling unseen robots.

Learning Model Predictive Control with Error Dynamics Regression for Autonomous Racing
Haoru Xue*,ย Edward L. Zhu,ย John M. Dolan, Francesco Borrelli

ICRA 2024 |ย website | arxivย |ย videoย |ย code | social mediaย |ย talk

Use MPC + robot learning to explore optimal policy and dynamics model online safely. Perform Sim2Real and learn the handling limit of extreme driving like a professional race car driver. Deployed on a full-size race car!

WROOM: An Autonomous Driving Approach for Off-Road Navigation
Dvij Kalaria*, Shreya Sharma, Sarthak Bhagat, Haoru Xue, John M. Dolan

ICRA 2024 Workshop | arxiv | website | code

WROOM brings a gym environment for training off-road driving RL policy. We use PPO + CBF to train an end-to-end agent to safely navigate in the real world.

Segment Anything Model for Road Network Graph Extraction
Congrui Hetang*, Haoru Xue, Cindy Le, Tianwei Yue, Wenping Wang, Yihui He

CVPR 2024 Workshop | arxiv | code

We propose a SAM-based pipeline for large-scale road network extraction from aerial image. Comparable accuracy with SOTA is achieved while being 40 times faster.

Best Paper (2nd Workshop on Scene Graphs and Graph Representation Learning)

Spline-Based Minimum-Curvature Trajectory Optimization for Autonomous Racing
Haoru Xue*, Tianwei Yue, John M. Dolan

arxiv | code

Education

UC Berkeley EECS
PhD (AI Robotics) 2024

CMU Robotics Institute
MS Robotics 2022 – 2024

Prof. John Dolan, Guanya Shi

UC San Diego
Electrical Eng. 2018 – 2021

Experience

LeCAR Lab, CMU Robotics Institute
Visiting Researcher
May – Aug. 2024

Advisor: Prof. Guanya Shi

MPC Lab, UC Berkeley
Visiting Researcher
Apr. – Aug. 2023

Advisor: Prof. Francesco Borrelli

AI Racing Tech
Lead Graduate Researcher
Dec. 2019 – Present

Advisor: Prof. Shankar Sastry, John Dolan; Dr. Allen Yang, Jack Silberman

Autoware Foundation
Software Engineer
Dec. 2021 – Aug. 2022

Awards