Tongzhou Mu 木同舟

Ph.D. candidate, Department of Computer Science and Engineering, University of California San Diego Email: t3mu [@] ucsd [DOT] edu Google Scholar / DBLP / Twitter / Github / LinkedIn

I am currently working on learning from demonstrations. Email me if you are interested in collaboration!

About me

I am a Ph.D. candidate in the Department of Computer Science and Engineering at the University of California San Diego, and I am fortunate to be advised by Prof. Hao Su. My long-term research goal is to build a decision-making framework with strong generalizability. Specifically, I am interested in Reinforcement Learning / Imitation Learning, Concept Discovery and Reasoning, and Robotics / Embodied AI.

Prior to starting as a Ph.D. student, I received my B.Eng. and M.Sc. degrees in computer science from Zhejiang University, and UC San Diego in 2017 and 2019, respectively. I have also spent time at Microsoft Research Asia, Intel AI, Wormpex AI , Amazon Alexa AI, and NVIDIA Robotics Lab.

I am the major developer of the 1st version of ManiSkill benchmark, and the lead organzier of the ManiSkill Challenge 2021 and the Generalizable Policy Learning in the Physical World Workshop. I also actively contribute to the development of the subsequent generations of ManiSkill, e.g., ManiSkill 2 and ManiSkill 3.

Research interests

My research interests include

• Reinforcement Learning / Imitation Learning

  • Sturctured Policies and Neural Programs

  • Learning from Demonstrations

  • Curriculum in RL

• Concept Discovery

  • Emerged Concepts from Interactions

  • Reasoning over Concepts

• Robotics / Embodied AI

  • Planning and Control for Manipulation

  • Building Full-Physical Tasks in Simulator

Publications & Preprints

Papers sorted by years. Representative papers are highlighted.

2023

Learning Reusable Dense Rewards for Multi-Stage Tasks Tongzhou Mu, Minghua Liu, Hao Su Submitted to ICLR 2024 [Project Page]

On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline Nicklas Hansen*, Zhecheng Yuan*, Yanjie Ze*, Tongzhou Mu* , Aravind Rajeswaran+, Hao Su+, Huazhe Xu+, Xiaolong Wang+ International Conference on Machine Learning (ICML) 2023 Pre-training Robot Learning Workshop at CoRL 2022

Abstract-to-Executable Trajectory Translation for One-Shot Task Generalization Stone Tao, Xiaochen Li, Tongzhou Mu, Zhiao Huang, Yuzhe Qin, Hao Su International Conference on Machine Learning (ICML) 2023 Deep Rieinforcement Learning Workshop at NeurIPS 2022 [Project Page] [Video] [Code] [Slides] * Stone Tao and Xiaochen Li are undergruduate students I advised

Boosting Reinforcement Learning and Planning with Demonstrations: A Survey Tongzhou Mu, Hao Su ArXiv preprint [Slides]

ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills Jiayuan Gu, Fanbo Xiang, Xuanlin Li*, Zhan Ling*, Xiqiang Liu*, Tongzhou Mu* , Yihe Tang*, Stone Tao*, Xinyue Wei*, Yunchao Yao*, Xiaodi Yuan, Pengwei Xie, Zhiao Huang, Rui Chen, Hao Su * equally contributed authors are ordered by alphabets International Conference on Learning Representations (ICLR) 2023 [Code] [Project Page] [Challenge Website]

Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation Xiaoshuai Zhang, Rui Chen, Fanbo Xiang, Yuzhe Qin, Jiayuan Gu, Zhan Ling, Minghua Liu, Peiyu Zeng, Songfang Han, Zhiao Huang, Tongzhou Mu, Jing Xu, Hao Su IEEE Transactions on Robotics (T-RO) 2023

2022

Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning Tongzhou Mu, Kaixiang Lin, Feiyang Niu, Govind Thattai Transactions on Machine Learning Research (TMLR) 2022 Elements of Reasoning: Objects, Structure, and Causality Workshop at ICLR 2022

2021

ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations Tongzhou Mu* , Zhan Ling*, Fanbo Xiang*, Derek Yang*, Xuanlin Li*, Stone Tao, Zhiao Huang, Zhiwei Jia, Hao Su Conference on Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track 2021 [Video] [Slides] [Poster] [Code] [Baselines] [Challenge Website]

2020

Refactoring Policy for Compositional Generalizability using Self-Supervised Object Proposals Tongzhou Mu* , Jiayuan Gu*, Zhiwei Jia, Hao Tang, Hao Su Conference on Neural Information Processing Systems (NeurIPS) 2020 [Project Page] [Poster] [Slides] [Code]

State Alignment-based Imitation Learning Fangchen Liu, Zhan Ling, Tongzhou Mu, Hao Su International Conference on Learning Representations (ICLR) 2020

Before 2019

Transfer Value or Policy? A Value-centric Framework Towards Transferrable Continuous Reinforcement Learning Xingchao Liu*, Tongzhou Mu* , Hao Su Deep Reinforcement Learning Workshop at NeurIPS 2018

Accelerated Doubly Stochastic Gradient Algorithm for Large-scale Empirical Risk Minimization Zebang Shen, Hui Qian, Tongzhou Mu, and Chao Zhang International Joint Conference on Artificial Intelligence (IJCAI) 2017

Adaptive Variance Reducing for Stochastic Gradient Descent Zebang Shen, Hui Qian, Tengfei Zhou, and Tongzhou Mu International Joint Conference on Artificial Intelligence (IJCAI) 2016

Talks

• Modeling the Physical World for Embodied AI: Environments and Algorithms
  Invited Talk at Microsoft Research Asia, Apr 2023
  

• Learning Neural Structrued Policy via Policy Refactorization
  Ph.D. Thesis Proposal at UC San Diego, Mar 2023
  

• Boosting Reinforcement Learning and Planning with Demonstrations
  Ph.D. Research Exam at UC San Diego, Jan 2023
  

• On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline
  Pre-training Robot Learning Workshop at CoRL 2022, Dec 2022
  

• Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
  Invited Talk at UC Berkeley Robot Learning Lab,    Nov 2021
  Invited Talk at Stanford Vision and Learning Lab,  Dec 2021

• Task-driven Entity Abstraction from Visual Observations
  Invited Talk at Qualcomm AI Lab, Mar 2020

Professional Services

• Workshop Organizer: ICLR 2022 Workshop “Generalizable Policy Learning in the Physical World” (Lead Organzier)

• Tutorial Organizer: CVPR 2022 Tutorial “Building and Working in Environments for Embodied AI”

• Challenge Organizer:

  • SAPIEN ManiSkill Challenge 2021 (Lead Organzier)

  • SAPIEN ManiSkill Challenge 2022

• Conference Reviewer:

  • Machine Learning: ICLR, NeurIPS, AAAI

  • Robotics: ICRA, IROS

  • Computer Vision: ICCV

• Journal Reviewer: RA-L

Teaching

• Teaching Assistant: CSE 152 Introduction to Computer Vision at UC San Diego, Fall 2018

• Consultant Volunteer: CSE291-J00 Deep Learning Lab (Computer Vision) at UC San Diego, Fall 2020

• Guest Lecturer: CSE291-A00 Machine Learning for Robotics at UC San Diego, Winter 2023

Awards

• ACM-ICPC (International Collegiate Programming Contest) Asia Regional Contest Gold Medal, 2015

• China Computer Federation Elite Collegiate Award (top 108 in China), 2016

• Award of Excellence for Stars of Tomorrow Internship Program, Microsoft Research Asia, 2017