Haoru Xue
I am an incoming PhD student at Berkeley AI Research (BAIR), UC Berkeley. I am currently visiting LeCAR lab advised by Professor Guanya Shi at CMU.
Previously I was a MS Robotics student at CMU DRIVE Lab, working with Professor John M. Dolan and Guanya Shi. In 2023 I was a visiting student with Professor Francesco Borrelli at MPC lab, UC Berkeley.
I’m interested in humanoid, quaduped, and vehicle robots that can do agile and dexterous loco-manipulation. I also want to strategically interface them with foundation models for robotics to perform long-horizon tasks.
I also lead AI Racing Tech, a multi-million $ autonomous racing research project in UCB, CMU and UCSD across 5 labs with over 20 active researchers. I deploy real-world robot learning on extremely agile self-driving race cars at 200 MPH. We are the top U.S. team in the Indy Autonomous Challenge. Learn more about this endeavor in my playground.
Latest
[05/16/2024, ICRA] (presentation | slides) Oral on “Learning MPC with Error Dynamics Regression for Autonomous Racing“. (10:30-12:00 at AX – F206)
[04/24/2024, CMU] (presentation | slides) Master thesis talk on “Optimal Control and Robot Learning on Agile Safety-Critical Systems“.
[04/22/2024, Stanford] Talk on “Towards Foundational Control Models for Agile Locomotion” at Stanford Intelligent Systems Laboratory (SISL).
[01/26/2024, UC Berkeley] Talk on “Safe Online Adaptation for Robots with Extreme Dynamics” at DARPA ANSR program.
[10/09/2023, UC Berkeley] Talk on “Learning MPC with Error Dynamics Regression for Autonomous Racing” at Model Predictive Control (MPC) Lab.
[10/03/2023, IROS] Oral on “Spline-Based Trajectory Optimization for Autonomous Racing“
Research
Ultimately, I want to build genearlist robots that will fundamentally change the human society. Recently, these are my research interests on humanoids, quadupeds, and vehicles:
- The Interface between high-level LLM/VLM/Diffusion planning and low-level RL policy is crutial for enabling both long-horizon and dexterous/agile loco-manipulation.
- Agility and Adaptability goes hand-in-hand towards foundational control models that few-shot adapts to controling unseen robots.
![](https://haoruxue.github.io/wp-content/uploads/2024/04/lmpc_cover.gif)
Learning Model Predictive Control with Error Dynamics Regression for Autonomous Racing
Haoru Xue*,ย Edward L. Zhu,ย John M. Dolan, Francesco Borrelli
ICRA 2024 |ย website | arxivย |ย videoย |ย code | social mediaย |ย talk
Use MPC + robot learning to explore optimal policy and dynamics model online safely. Perform Sim2Real and learn the handling limit of extreme driving like a professional race car driver. Deployed on a full-size race car!
![](https://haoruxue.github.io/wp-content/uploads/2024/04/gail.gif)
WROOM: An Autonomous Driving Approach for Off-Road Navigation
Dvij Kalaria*, Shreya Sharma, Sarthak Bhagat, Haoru Xue, John M. Dolan
ICRA 2024 Workshop | arxiv | website | code
WROOM brings a gym environment for training off-road driving RL policy. We use PPO + CBF to train an end-to-end agent to safely navigate in the real world.
![](https://haoruxue.github.io/wp-content/uploads/2024/04/sam_road_segmentation-1024x685.png)
Segment Anything Model for Road Network Graph Extraction
Congrui Hetang*, Haoru Xue, Cindy Le, Tianwei Yue, Wenping Wang, Yihui He
CVPR 2024 Workshop | arxiv | code
We propose a SAM-based pipeline for large-scale road network extraction from aerial image. Comparable accuracy with SOTA is achieved while being 40 times faster.
Best Paper (2nd Workshop on Scene Graphs and Graph Representation Learning)
![](https://haoruxue.github.io/wp-content/uploads/2024/04/iros_late_braking-Page-1.drawio-1024x577.png)
Spline-Based Minimum-Curvature Trajectory Optimization for Autonomous Racing
Haoru Xue*, Tianwei Yue, John M. Dolan
arxiv | code
Education
![](https://haoruxue.github.io/wp-content/uploads/2024/04/bair-1024x1024.png)
UC Berkeley EECS
PhD (AI Robotics) 2024
![](https://haoruxue.github.io/wp-content/uploads/2024/04/cmu-wordmark-square-w-on-r-1-1024x1024.png)
CMU Robotics Institute
MS Robotics 2022 – 2024
Prof. John Dolan, Guanya Shi
![](https://haoruxue.github.io/wp-content/uploads/2024/04/ucsd_square-1024x1024.png)
UC San Diego
Electrical Eng. 2018 – 2021
Experience
![](https://haoruxue.github.io/wp-content/uploads/2024/07/ri-mark-white-on-red-800x800-socialmedia.png)
LeCAR Lab, CMU Robotics Institute
Visiting Researcher
May – Aug. 2024
Advisor: Prof. Guanya Shi
![](https://haoruxue.github.io/wp-content/uploads/2024/04/UCBerkeley_wordmark_white.png)
MPC Lab, UC Berkeley
Visiting Researcher
Apr. – Aug. 2023
Advisor: Prof. Francesco Borrelli
![](https://haoruxue.github.io/wp-content/uploads/2024/04/art_logo_dark-1024x1024.png)
AI Racing Tech
Lead Graduate Researcher
Dec. 2019 – Present
Advisor: Prof. Shankar Sastry, John Dolan; Dr. Allen Yang, Jack Silberman
Awards
- [2023] 3rd Place, Indy Autonomous Challenge at Las Vegas CES
- [2022] 2rd Place, Indy Autonomous Challenge at Texas Motor Speedway
- [2021] Henry G. Booker Memorial Honors Award, UC San Diego ECE Department