Hao Dong

董豪 北京大学 助理教授 博士生导师 国家级青年人才计划 北大博雅青年学者

hao.dong@pku.edu.cn

Google Scholar / Github


About Me

I am an Assistant Professor at School of Computer Science, Peking University, where I lead PKU-Agibot Lab. My current research focuses on embodied AI, large models, reinforcement learning and computer vision. Our goal is to find the scaling law to create a cost-effective and autonomous robot system. Our work has been recognized as a Best Application Paper Finalist at IROS.

Additionally, I am fortunate to serve as an Area Chair or Senior Program Committee member for CVPR, NeurIPS and AAAI conferences, and as the Associate Editor of ICRA and Machine Intelligence Research. I received the MIR Outstanding Associate Editor Award. Also, I have been involved in open source AI system for a long time, I have led several open source projects, such as Polar Research Station, TensorLayer GitHub Stars and OpenMLsys GitHub Stars , and have won the Best Open Source Software Award at ACM Multimedia, as well as the OpenI Outstanding Project Award twice.
more Before joining PKU, I obtained my Ph.D. degree from Imperial College London under the supervision of Yike Guo. Prior to my Ph.D., I received a MSc degree with distinction from Imperial, and a first-class BEng degree from the University of Central Lancashire. Furthermore, I have founded a startup focused on AI-driven hardware between 2012 and 2015.

News

  • NEW 近期中文报告:
  • [2024/10] NEW SCANet is recognized as a Best Application Paper Finalist at IROS 2024.
  • [2024/09] Two papers get accepted to NeurIPS 2024
  • [2024/09] The world's first general navigation large model that unifies visual-language navigation, object navigation as well as demand-driven navigation into one single framework: InstructNav
  • [2024/09] Three papers get accepted to CoRL 2024: Generic Instruction Navigation, Interactive Correction for Manipulation, Articulation-Aware VLM
  • [2024/09] One paper gets accepted to Nature Machine Intelligence
  • [2024/09] One paper gets accepted to RAL
  • [2024/08] Call for Papers: Special Issues on Embodied AI in Journal of Field Robotics
  • [2024/07] Two papers get accepted to ECCV 2024: Omni6DPose, Grasping
  • show more
  • [2024/06] Three papers get accepted to IROS 2024:Pre-grasping, ManipVQA, Lego Assembly
  • [2024/06] CVPR 2024 Embodied AI Workshop PRS Challenge: Human-centered In-building Embodied Delivery
  • [2024/05] Two papers get accepted to RSS 2024
  • [2024/04] Our RGB-based object grasping paper is accepted to RAL 2024
  • [2024/02] Three papers get accepted to CVPR 2024
  • [2024/01] Five papers get accepted to ICRA 2024
  • [2024/01] I received the MIR Outstanding Associate Editor Award
  • [2024/01] Two papers get accepted to ICLR 2024: SparseDFF and PerSAM
  • [2023/12] One paper gets accepted to PAMI and two papers for AAAI 2024 Bi-DexHands, MUTR and FractureAssembly
  • [2023/09] Five NeurPS 2023 submissions are all accepted: Demand-driven Navigation, GenPose, GraspGF, EnvAwareAfford and Where2Explore
  • [2023/09] I will serve as an associate editor of ICRA
  • [2023/08] One paper gets accepted to SIGGRAPH Asia, and two papers for BMVC
  • [2023/07] Two papers get accepted to ICCV 2023: DefoAfford and 3D Shape Assembly
  • [2023/06] I will serve as an AC of CVPR 2024
  • [2023/06] I will serve as a SPC of AAAI 2024
  • [2023/04] Our visual-audio navigation gets accepted to RAL
  • [2023/03] I will serve as an AC of NeurIPS 2023
  • [2023/02] Three paper get accepted to CVPR 2023
  • ...

    PKU-Agibot Lab

    Our lab welcomes research interns, masters, PhD candidates and postdocs. The current research interests include:
  • grasping and manipulation
  • task planning
  • navigation
  • safety and interpretability in robotics
    For more information, please contact Hao Dong at hao.dong (a) pku.edu.cn

  • Services

  • Area Chair: NeurIPS (2023, 2024), CVPR (2023, 2024)
  • Senior Program Committee: AAAI (2023, 2024)
  • Associate Editor: ICRA, Machine Intelligence Research

  • Courses

  • Foundamentals of AI (Spring Term 2023 - )
  • Introduction to Computing (A) (Fall Term 2022 - )
  • previous courses
  • Deep Generative Models (Spring Term 2020 - 2022)
  • Introduction to Computing (B) (Fall Term 2020 - 2021)
  • Study and Practice on Topics of Frontier Computing (I) (Autumn Term 2019)
  • Introduction to Deep Learning (Turing Class) (Summer Term 2019)

  • Books
    Deep Reinforcement Learning: Fundamentals, Research and Applications
    Hao Dong, Zihan Ding, Shanghang Zhang Eds.
    Springer Nature 2020 ISBN 978-981-15-4094-3
    --- A Selection of the High-impact Publications in CS by Chinese Researchers from Springer Nature
    Chinese version 深度强化学习:基础、研究与应用 董豪、丁子涵、仉尚航 等著(简体中文译本 Simplified Chinese)
    电子工业出版社 2021 ISBN 978-7-121-41188-5
    新一代AI霸主 - 深度強化學習 董豪、丁子涵、仉尚航 等著(繁體中文譯本 Traditional Chinese)
    深智數位 2022 ISBN 978-986-0776-82-9
    [Free Open Source Book] [Springer ] [Broadview] [繁体版本] [京东]
    Machine Learning System: Design and Implementation
    Luo Mai, Hao Dong Eds. Springer Nature 2024 coming soon
    Chinese version 机器学习系统:设计与实现 麦络、董豪 等著
    清华大学出版社 Tsinghua University Press 2023 ISBN 978-7-302-63007-4
    [OpenMLsys GithubGitHub Stars] [English Open Source Book (coming soon)] [Chinese Open Source Book] [京东]
    Papers
    ( show recent selected / show more )
    Canonical Representation and Force-Based Pretraining of 3D Tactile for Dexterous Visuo-Tactile Policy Learning
    Tianhao Wu, Jinzhou Li, Jiyao Zhang, Mingdong Wu, Hao Dong
    arXiv 2024
    [Paper] [Webpage]
    SpatialBot: Precise Spatial Understanding with Vision Language Models
    Wenxiao Cai, Yaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, Bo Zhao
    arXiv 2024
    [Paper] [Code] [机器之心]
    Efficient and Scalable Reinforcement Learning for Large-scale Network Control
    Chengdong Ma, Aming Li, Yali Du, Hao Dong, Yaodong Yang
    Nature Machine Intelligence (NMI) 2024
    [Paper] [新华网] [科技日报]
    GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation
    Haoran Lu, Yitong Li, Ruihai Wu, Sijie Li, Ziyu Zhu, Chuanruo Ning, Yan Shen, Longzan Luo, Yuanpei Chen, Hao Dong
    Neural Information Processing System (NeurIPS) 2024
    [Paper] [Webpage] [Code] [Docs]
    MO-DDN: A Coarse-to-Fine Attribute-based Exploration Agent for Multi-Object Demand-driven Navigation
    Hongcheng Wang, Peiqi Liu, Wenzhe Cai, Mingdong Wu, Zhengyu Qian, Hao Dong
    Neural Information Processing System (NeurIPS) 2024
    [Paper] [Webpage] [Code]
    Human-centered In-building Embodied Delivery Benchmark
    Zhuoquan Xu, Yang Liu, Xiaoqi Li, Jiyao Zhang, Hao Dong
    arXiv 2024
    [Paper] [Webpage]
    UniDexFPM: Universal Dexterous Functional Pre-grasp Manipulation via Diffusion Policy
    Tianhao Wu, YunChong Gan, Mingdong Wu, Jingbo Cheng, Yaodong Yang, Yixin Zhu, Hao Dong
    arXiv 2024
    [Paper] [Webpage]
    GFPack++: Improving 2D Irregular Packing by Learning Gradient Field with Attention
    Tianyang Xue, Lin Lu, Yang Liu, Mingdong Wu, Hao Dong, Yanbin Zhang, Renmin Han, Baoquan Chen
    arXiv 2024
    [Paper]
    InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment
    --- The world's first general navigation large model that unifies visual-language navigation, object navigation as well as demand-driven navigation into one single framework.
    Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, Hao Dong
    Conference on Robot Learning (CoRL) 2024
    [Paper] [Webpage] [Code] [量子位]
    AIC-MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation
    --- The first automatic system for low-level end-effector action correction in manipulation tasks.
    Chuyan Xiong, Chengyu Shen, Xiaoqi Li, Kaichen Zhou, Jiaming Liu, Ruiping Wang, Hao Dong
    Conference on Robot Learning (CoRL) 2024
    [Paper] [Webpage]
    A3VLM: Actionable Articulation-Aware Vision Language Model
    Siyuan Huang, Haonan Chang, Yuhan Liu, Yimeng Zhu, Hao Dong, Peng Gao, Abdeslam Boularias, Hongsheng Li
    Conference on Robot Learning (CoRL) 2024
    [Paper] [Code] [OpenGVLab摘要]
    NaturalVLM: Leveraging Fine-grained Natural Language for Affordance-Guided Visual Manipulation
    Ran Xu, Yan Shen, Xiaoqi Li, Ruihai Wu, Hao Dong
    IEEE Robotics and Automation Letters (RAL) 2024
    [Paper] [Webpage]
    UniDoorManip: Learning Universal Door Manipulation Policy over Large-scale and Diverse Door Manipulation Environments
    Yu Li*, Xiaojie Zhang*, Ruihai Wu*, Zilong Zhang, Yiran Geng, Hao Dong, Zhaofeng He
    arXiv 2024
    [Paper] [Webpage] [量子位]
    Omni6DPose: A Benchmark and Model for Universal 6D Object Pose Estimation and Tracking
    --- The largest-scale benchmark for universal 6D object pose estimation.
    Jiyao Zhang, Weiyao Huang, Bo Peng, Mingdong Wu, Fei Hu, Zijian Chen, Bo Zhao, Hao Dong
    European Conference on Computer Vision (ECCV) 2024
    [Paper] [Webpage] [Code] [计算机视觉工坊]
    Local Occupancy-Enhanced Object Grasping with Multiple Triplanar Projection
    Kangqi Ma, Hao Dong, Yadong Mu
    European Conference on Computer Vision (ECCV) 2024
    [Paper]
    PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environments
    Kairui Ding, Boyuan Chen, Ruihai Wu, Yuyang Li, Zongzheng Zhang, Huan-ang Gao, Siqi Li, Yixin Zhu, Guyue Zhou, Hao Dong, Hao Zhao
    International Conference on Intelligent Robots and Systems (IROS) 2024 (Oral)
    [Paper] [Webpage]
    ManipVQA: Injecting Robotic Affordance and Physically Grounded Information into Multi-Modal Large Language Models
    Siyuan Huang, Iaroslav Ponomarenko, Zhengkai Jiang, Xiaoqi Li, Xiaobin Hu, Peng Gao, Hongsheng Li, Hao Dong
    International Conference on Intelligent Robots and Systems (IROS) 2024 (Oral)
    [Paper] [Code]
    SCANet: Correcting LEGO Assembly Errors with Self-Correct Assembly Network
    --- Best Application Paper Finalist (4/3645)
    Yuxuan Wan, Kaichen Zhou, Jinhong Chen, Hao Dong
    International Conference on Intelligent Robots and Systems (IROS) 2024 (Oral)
    [Paper] [Webpage] [Code]
    Broadcasting Support Relations Recursively from Local Dynamics for Object Retrieval in Clutters
    Yitong Li*, Ruihai Wu*, Haoran Lu, Chuanruo Ning, Yan Shen, Guanqi Zhan, Hao Dong
    Robotics: Science and Systems (RSS) 2024
    [Paper] [Webpage] [Code]
    MPI: Learning Manipulation by Predicting Interaction
    Jia Zeng, Qingwen Bu, Bangjun Wang, Wenke Xia, Li Chen, Hao Dong, Haoming Song, Dong Wang, Di Hu, Ping Luo, Heming Cui, Bin Zhao, Xuelong Li, Yu Qiao, Hongyang Li
    Robotics: Science and Systems (RSS) 2024
    [Paper] [Webpage] [Code]
    A Survey of Reasoning with Foundation Models
    Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, Yue Wu, Wenhai Wang, Junsong Chen, Zhangyue Yin, Xiaozhe Ren, Jie Fu, Junxian He, Wu Yuan, Qi Liu, Xihui Liu, Yu Li, Hao Dong, Yu Cheng, Ming Zhang, Pheng Ann Heng, Jifeng Dai, Ping Luo, Jingdong Wang, Ji-Rong Wen, Xipeng Qiu, Yike Guo, Hui Xiong, Qun Liu, Zhenguo Li
    arXiv 2023
    [Paper] [Github] GitHub Stars
    LVDiffusor: Distilling Functional Rearrangement Priors from Large Models into Diffusor
    Yiming Zeng*, Mingdong Wu*, Long Yang, Jiyao Zhang, Hao Ding, Hui Cheng, Hao Dong
    IEEE Robotics and Automation Letters (RAL) 2024
    [Paper] [Webpage]
    Pattern4Ego: Learning Egocentric Video Representation Using Cross-Video Activity Patterns
    Ruihai Wu, Yourong Zhang, Yu Qi, Andy Guanhong Chen, Hao Dong
    International Conference on Multimedia Retrieval (ICMR) 2024
    [Paper]
    ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation
    Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yuxing Long, Yan Shen, Renrui Zhang, Jiaming Liu, Hao Dong
    Conference on Computer Vision and Pattern Recognition (CVPR) 2024
    [Paper] [Webpage] [Code] [量子位] [强化学习技术前沿] [集智书童]
    UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence
    Ruihai Wu, Haoran Lu, Yiyan Wang, Yubo Wang, Hao Dong
    --- The world's first work of category-level garment manipulation with only few-shot demonstrations
    Conference on Computer Vision and Pattern Recognition (CVPR) 2024
    [Paper] [Webpage] [Code]
    No Time to Train: Empowering Non-Parametric Networks for Few-shot 3D Scene Segmentation
    Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyu Guo, Jiaming Liu, Han Xiao, Chaoyou Fu, Hao Dong, Peng Gao
    Conference on Computer Vision and Pattern Recognition (CVPR) 2024 (Highlight)
    [Paper] [Code] [公众号]
    ImageManip: Image-based Robotic Manipulation with Affordance-guided Next View Selection
    Xiaoqi Li, Yanzi Wang, Yan Shen, Haoran Lu, Qianxu Wang, Ponomarenko Iaroslav, Boshi An, Jiaming Liu, Hao Dong
    arXiv 2023
    [Paper] [Webpage]
    RGBManip: Monocular Image-based Robotic Manipulation through Active Object Pose Estimation
    Boshi An, Yiran Geng, Kai Chen, Xiaoqi Li, Qi Dou, Hao Dong
    International Conference on Robotics and Automation (ICRA) 2024
    [Paper] [Webpage] [Code] [北大]
    Articulated Object Manipulation with Coarse-to-fine Affordance for Mitigating the Effect of Point Cloud Noise
    Suhan Ling, Yian Wang, Shiguang Wu, Yuzheng Zhuang, Tianyi Xu, Yu Li, Chang Liu, Hao Dong
    International Conference on Robotics and Automation (ICRA) 2024
    [Paper] [Webpage] [Code]
    RoboKeyGen: Robot Pose and Joint Angles Estimation via Diffusion-based 3D Keypoint Generation
    Yang Tian, Jiyao Zhang, Guowei Huang, Bin Wang, Ping Wang, Jiangmiao Pang, Hao Dong
    International Conference on Robotics and Automation (ICRA) 2024
    [Paper] [Webpage] [Code]
    Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions
    --- The world's first visual language navigation large model system deployed in real world
    Yuxing Long, Xiaoqi Li, Wenzhe Cai, Hao Dong
    International Conference on Robotics and Automation (ICRA) 2024
    [Paper] [Webpage] [Code] [量子位]
    PixNav: Bridging Zero-shot Object Navigation and Foundation Models through Pixel-guided Navigation Skill
    --- The world's first purely visual-based object goal navigation large model
    Wenzhe Cai, Siyuan Huang, Guangran Cheng, Yuxing Long, Peng Gao, Changyin Sun, Hao Dong
    International Conference on Robotics and Automation (ICRA) 2024
    [Paper] [Webpage] [Code] [北大]
    RGBGrasp: Image-based Object Grasping by Capturing Multiple Views during Robot Arm Movement with Neural Radiance Field
    Chang Liu, Kejian Shi, Kaichen Zhou, Haoxiao Wang, Jiyao Zhang, Hao Dong
    IEEE Robotics and Automation Letters (RAL) 2024
    [Paper] [Webpage]
    SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation
    Qianxu Wang, Haotong Zhang, Congyue Deng, Yang You, Hao Dong, Yixin Zhu, Leonidas Guibas
    International Conference on Learning Representations (ICLR) 2024
    [Paper] [Webpage] [Code]
    PerSAM: Personalize Segment Anything Model with One Shot
    Renrui Zhang, Zhengkai Jiang, Ziyu Guo, Shilin Yan, Junting Pan, Hao Dong, Peng Gao, Hongsheng Li
    International Conference on Learning Representations (ICLR) 2024
    [Paper] [Webpage] [Code] [AIWalker]
    Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers
    Ruiyuan Zhang, Jiaxiang Liu, Zexi Li, Hao Dong, Jie Fu, Chao Wu
    AAAI Conference on Artificial Intelligence 2024
    [Paper] [Code]
    Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation
    Shilin Yan, Renrui Zhang, Ziyu Guo, Wenchao Chen, Wei Zhang, Hongyang Li, Yu Qiao, Hao Dong, Zhongjiang He, Peng Gao
    AAAI Conference on Artificial Intelligence 2024
    [Paper] [Code]
    Bi-DexHands: Towards Human-Level Bimanual Dexterous Manipulation
    Yuanpei Chen, Yiran Geng, Fangwei Zhong, Jiaming Ji, Jiechuang Jiang, Zongqing Lu, Hao Dong, Yaodong Yang
    --- The world's first bimanual dexterous manipulation benchmark (in simulation)
    IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 2023
    [Paper] [Webpage] [Code]
    Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model
    Siyuan Huang, Zhengkai Jiang, Hao Dong, Yu Qiao, Peng Gao, Hongsheng Li
    arXiv 2023
    [Paper] [Code] [机器人3D感知] [CSDN]
    Mixup-Augmented Meta-Learning for Sample-Efficient Fine-Tuning of Protein Simulators
    Jingbang Chen, Yian Wang, Xingwei Qu, Shuangjia Zheng, Yaodong Yang, Hao Dong, Jie Fu
    arXiv 2023
    [Paper] [Code]
    Posterior Instance Injection Detector for Arbitrary-Oriented Object Detection From Optical Remote-Sensing Imagery
    Tong Zhang, Yin Zhuang, He Chen, Guanqun Wang, Lihui Ge, Liang Chen, Hao Dong, Lianlin Li
    Remote Sensing 2023
    [Paper]
    Plan4MC: Skill Reinforcement Learning and Planning for Open-World Minecraft Tasks
    Haoqi Yuan, Chi Zhang, Hongcheng Wang, Feiyang Xie, Penglin Cai, Hao Dong, Zongqing Lu
    Neural Information Processing Systems (NeurIPS) FMDM Workshop 2023
    [Paper] [Webpage] [Code] [机器之心]
    Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation
    ---The world's first human demand-driven navigation model
    Hongcheng Wang, Andy Guan Hong Chen, Xiaoqi Li, Mingdong Wu, Hao Dong
    Neural Information Processing Systems (NeurIPS) 2023
    [Paper] [Webpage] [Video] [Code] [BAAI]
    GenPose: Generative Category-level Object Pose Estimation via Diffusion Models
    --- The next-generation category-level 6D object pose paradigm: generative pose estimation
    Jiyao Zhang, Mingdong Wu, Hao Dong
    Neural Information Processing Systems (NeurIPS) 2023
    [Paper] [Webpage] [Code] [北大]
    Learning Environment-aware Affordance for 3D Articulated Object Manipulation under Occlusions
    --- The world's first work of affordance learning with environment constraints
    Ruihai Wu, Kai Cheng, Yan Zhao, Chuanruo Ning, Guanqi Zhan, Hao Dong
    Neural Information Processing Systems (NeurIPS) 2023
    [Paper] [Webpage] [Code] [AIR学术] [AIR论坛]
    GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Grasping
    Tianhao Wu, Mingdong Wu, Jiyao Zhang, Yunchong Gan, Hao Dong
    Neural Information Processing Systems (NeurIPS) 2023
    [Paper] [Webpage] [Code] [新智元]
    Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects
    --- The world's first work of few-shot exploration for object manipulation with novel geometries
    Chuanruo Ning, Ruihai Wu, Haoran Lu, Kaichun Mo, Hao Dong
    Neural Information Processing Systems (NeurIPS) 2023
    [Paper] [Webpage] [Code]
    Learning Gradient Fields for Scalable and Generalizable Irregular Packing
    Tianyang Xue, Mingdong Wu, Lin Lu, Haoxuan Wang, Hao Dong, Baoquan Chen
    SIGGRAPH Asia 2023
    [Paper] [Webpage] [Code]
    Learning Part Motion of Articulated Objects Using Spatially Continuous Neural Implicit Representations
    Yushi Du, Ruihai Wu, Yan Shen, Hao Dong
    British Machine Vision Conference (BMVC) 2023
    [Paper] [Webpage] [Code]
    Score-PA: Score-based 3D Part Assembly
    Junfeng Cheng, Mingdong Wu, Ruiyuan Zhang, Guanqi Zhan, Chao Wu, Hao Dong
    British Machine Vision Conference (BMVC) 2023 (Oral)
    [Paper] [Code]
    MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library
    Siyi Hu, Yifan Zhong, Minquan Gao, Weixun Wang, Hao Dong, Xiaodan Liang, Zhihui Li, Xiaojun Chang, Yaodong Yang
    Journal of Machine Learning Research 2023
    [Paper] [Documentation] [Code]
    DefoAfford: Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation
    Ruihai Wu, Chuanruo Ning, Hao Dong
    International Conference on Computer Vision (ICCV) 2023
    [Paper] [Webpage] [Code] [将门创投] [AIR学术] [AIR论坛]
    Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assembly
    Ruihai Wu, Chenrui Tie, Yushi Du, Yan Zhao, Hao Dong
    International Conference on Computer Vision (ICCV) 2023
    [Paper] [Webpage] [Code]
    Learning a Universal Human Prior for Dexterous Manipulation from Human Preference
    Zihan Ding, Yuanpei Chen, Allen Z. Ren, Shixiang Shane Gu, Hao Dong, Chi Jin
    RSS Workshop on Learning Dexterous Manipulation 2023
    [Paper]
    Learning Semantic-Agnostic and Spatial-Aware Representation for Generalizable Visual-Audio Navigation
    Hongcheng Wang, Yuxuan Wang, Fangwei Zhong, Mingdong Wu, Jianwei Zhang, Yizhou Wang, Hao Dong
    IEEE Robotics and Automation Letters (RAL) 2023
    [Paper] [Webpage] [Code] [CFCS]
    SGTAPose: Robot Structure Prior Guided Temporal Attention for Camera-to-Robot Pose Estimation from Image Sequence
    Yang Tian, Jiyao Zhang, Zekai Yin, Hao Dong
    Conference on Computer Vision and Pattern Recognition (CVPR) 2023
    [Paper] [Webpage] [Code]
    GFPose: Learning Gradient Field for Multi-Hypothesis 3D Human Pose Estimation
    Hai Ci, Mingdong Wu, Wentao Zhu, Xiaoxuan Ma, Hao Dong, Fangwei Zhong, Yizhou Wang
    Conference on Computer Vision and Pattern Recognition (CVPR) 2023
    [Paper] [Webpage] [Code] [CFCS]
    PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations
    Haoran Geng, Ziming Li, Yiran Geng, Jiayi Chen, Hao Dong, He Wang
    Conference on Computer Vision and Pattern Recognition (CVPR) 2023
    [Paper] [Webpage] [Code]
    ReBNN: Resilient Binary Neural Network
    Sheng Xu, Yanjing Li, Teli Ma, Mingbao Lin, Hao Dong, Baochang Zhang, Peng Gao, Jinhu Lu
    AAAI Conference on Artificial Intelligence 2023 (Oral)
    [Paper] [Code]
    RLAfford: End-to-End Affordance Learning for Robotic Manipulation
    Yiran Geng, Boshi An, Haoran Geng, Yuanpei Chen, Yaodong Yang, Hao Dong
    International Conference on Robotics and Automation (ICRA) 2023
    [Paper] [Webpage] [Code] [CFCS] [AIR学术] [AIR论坛]
    DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Object Manipulation
    Yan Zhao, Ruihai Wu, Zhehuan Chen, Yourong Zhang, Qingnan Fan, Kaichun Mo, Hao Dong
    International Conference on Learning Representations (ICLR) 2023
    [Paper] [Webpage] [Code] [AIR学术] [AIR论坛]
    Object-Centric Masked Image Modeling-Based Self-Supervised Pretraining for Remote Sensing Object Detection
    Tong Zhang, Yin Zhuang, He Chen, Liang Chen, Guanqun Wang, Peng Gao, Hao Dong
    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2023
    [Paper]
    P2FEViT: Plug-and-Play CNN Feature Embedded Hybrid Vision Transformer for Remote Sensing Image Classification
    Guanqun Wang, He Chen, Liang Chen, Yin Zhuang, Shanghang Zhang, Tong Zhang, Hao Dong, Peng Gao
    Remote Sensing 2023
    [Paper] [Code]
    Intelligent Indoor Metasurface Robotics
    ---Journal cover: a new robot concept of robot percepton and privacy
    Hanting Zhao, Shengguo Hu, Hongrui Zhang, Zhuo Wang, Hao Dong, Philipp del Hougne, Tie Jun Cui, Lianlin Li
    National Science Review (NSR) 2022
    [Paper] [Journal Cover] [中国科学杂志社]
    MyoChallenge 2022: Learning Contact-rich Manipulation using a Musculoskeletal Hand
    ---First Place in NeurIPS 2022 Challenge Track (1st in 340 submissions from 40 teams)
    Vittorio Caggiano, Guillaume Durandau, Huwawei Wang, Alberto Chiappa, Alexander Mathis, Pablo Tano, Nisheet Patel, Alexandre Pouget, Pierre Schumacher, Georg Martius, Daniel Haeufle, Yiran Geng, Boshi An, Yifan Zhong, Jiaming Ji, Yuanpei Chen, Hao Dong, Yaodong Yang, Rahul Siripurapu, Luis Eduardo Ferro Diez, Michael Kopp, Vihang Patil, Sepp Hochreiter, Yuval Tassa, Josh Merel, Randy Schultheis, Seungmoon Song, Massimo Sartori, Vikash Kumar
    Proceedings of the NeurIPS 2022 Competitions Track, Proceedings of Machine Learning Research
    [Paper] [Challenge Page] [Code] [Award] [Slide] [Talk] [Media(BIGAI)] [Media(CFCS)] [Media(PKU-EECS)] [Media(IAI)] [Media(PKU)] [Media(China Youth Daily)]
    Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARL
    Jakub Grudzien Kuba, Xidong Feng, Shiyao Ding, Hao Dong, Jun Wang, Yaodong Yang
    arXiv 2022
    [Paper] [Code]
    GraspARL: Dynamic Grasping via Adversarial Reinforcement Learning
    Tianhao Wu, Fangwei Zhong, Yiran Geng, Hongchen Wang, Yongjian Zhu, Yizhou Wang, Hao Dong
    arXiv 2022
    [Paper]
    RoboAssembly: Learning Generalizable Furniture Assembly Policy in a Novel Multi-robot Contact-rich Simulation Environment
    Mingxin Yu*, Lin Shao*, Zhehuan Chen, Tianhao Wu, Qingnan Fan, Kaichun Mo, Hao Dong
    arXiv 2022
    [Paper] [Webpage]
    TarGF: Learning Target Gradient Field to Rearrange Objects without Explicit Goal Specification
    Mingdong Wu, Fangwei Zhong, Yulong Xia, Hao Dong
    Neural Information Processing Systems (NeurIPS) 2022
    [Paper] [Webpage] [Code]
    Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
    Yuanpei Chen, Tianhao Wu, Shengjie Wang, Xidong Feng, Jiechuang Jiang, Stephen Marcus McAleer, Hao Dong, Zongqing Lu, Song-Chun Zhu, Yaodong Yang
    Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks 2022
    [Paper] [Webpage] [Code]
    AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions
    Yian Wang*, Ruihai Wu*, Kaichun Mo*, Jiaqi Ke, Qingnan Fan, Leonidas Guibas, Hao Dong
    --- The world's first work of active exploration for object manipulation with invisible dynamics and kinematics
    European Conference on Computer Vision (ECCV) 2022
    [Paper] [Webpage] [Code] [CFCS] [AIR学术] [AIR论坛]
    DREDS: Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects
    Qiyu Dai*, Jiyao Zhang*, Qiwei Li, Tianhao Wu, Hao Dong, Ziyuan Liu, Ping Tan, He Wang
    European Conference on Computer Vision (ECCV) 2022
    [Paper] [Webpage] [Code]
    Scalable Model-based Policy Optimization for Decentralized Networked Systems
    Yali Du, Chengdong Ma, Yuchen Liu, Runji Lin, Hao Dong, Jun Wang, Yaodong Yang
    International Conference on Intelligent Robots and Systems (IROS) 2022
    [Paper] [Code]
    VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D Articulated Objects
    Ruihai Wu, Yan Zhao, Kaichun Mo, Zizheng Guo, Yian Wang, Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, Hao Dong
    International Conference on Learning Representations (ICLR) 2022
    [Paper] [Code] [Webpage] [Youtube] [Bilibili] [CFCS] [AIR学术] [AIR论坛]
    Consecutive Pre-Training: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain
    Tong Zhang, Peng Gao, Hao Dong, Yin Zhuang, Guanqun Wang, Wei Zhang, He Chen
    Remote Sensing 2022
    [Paper] [Code]
    Hierarchical Disentangling Network for Building Extraction from Very High Resolution Optical Remote Sensing Imagery
    Jianhao Li, Yin Zhuang, Shan Dong, Peng Gao, Hao Dong, He Chen, Liang Chen, Lianlin Li
    Remote Sensing 2022
    [Paper] [Code]
    Adaptive Local Context Embedding for Small Vehicle Detection from Aerial Optical Remote Sensing Images
    Shanjunyu Liu, Yin Zhuang, Hao Dong, Peng Gao, Guanqun Wang, Tong Zhang, Liang Chen, He Chen, Lianlin Li
    IEEE International Geoscience and Remote Sensing Symposium (IGRASS) 2022
    [Paper]
    DMotion: Robotic Visuomotor Control with Unsupervised Forward Model Learned from Videos
    ---The first attempt to learn the forward model unsupervisedly via motion disentanglement
    Haoqi Yuan, Ruihai Wu, Andrew Zhao, Haipeng Zhang, Zihan Ding, Hao Dong
    International Conference on Intelligent Robots and Systems (IROS) 2021
    [Paper] [Webpage] [Code] [CFCS]
    End-to-End Object Detection with Adaptive Clustering Transformer
    Minghang Zheng, Peng Gao, Xiaogang Wang, Hongsheng Li, Hao Dong
    British Machine Vision Conference (BMVC) 2021 (Oral)
    [Paper] [Code] [集智书童]
    Contrastive Multimodal Fusion with TupleInfoNCE
    Yunze Liu, Qingnan Fan, Shanghang Zhang, Hao Dong, Thomas Funkhouser, Li Yi
    International Conference on Computer Vision (ICCV) 2021
    [Paper] [Code] [Code]
    P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding
    Yunze Liu, Li Yi, Shanghang Zhang, Qingnan Fan, Thomas Funkhouser, Hao Dong
    arXiv 2012.13089
    [Paper] [Code]
    Fast and Flexible Human Pose Estimation with HyperPose
    Yixiao Guo*, Jialei Liu*, Guo Li*, Luo Mai, Hao Dong
    ACM Multimedia (MM) Open Source 2021
    [Paper] [Code]
    Efficient Reinforcement Learning Development with RLzoo
    Zihan Ding, Tianyang Yu, Yanhua Huang, Hongming Zhang, Luo Mai, Hao Dong
    ACM Multimedia (MM) Open Source 2021
    [Paper] [Code] [机器之心]
    Edge-Enhanced Dual Discriminator Generative Adversarial Network for Fast MRI with Parallel Imaging Using Multi-view Information
    Jiahao Huang, Weiping Ding, Jun Lv, Jingwen Yang, Hao Dong, Javier Del Ser, Jun Xia, Tiaojuan Ren, Stephen Wong, Guang Yang
    Applied Intelligence 2021
    [Paper]
    Generative 3D Part Assembly via Dynamic Graph Learning
    ---The world's first 3D part assemble model without external guidance
    Jialei Huang*, Guanqi Zhan*, Qingnan Fan, Kaichun Mo, Lin Shao, Baoquan Chen, Leonidas Guibas, Hao Dong
    Neural Information Processing Systems (NeurIPS) 2020
    [Paper] [Code] [Webpage] ( [机器之心]/ [AI科技评论] )
    ACL-GAN: Unpaired Image-to-Image Translation using Adversarial Consistency Loss
    Yihao Zhao, Ruihai Wu, Hao Dong
    European Conference on Computer Vision (ECCV) 2020
    [Paper] [Code] [Webpage] [CFCS]
    Lyapunov-Based Reinforcement Learning for Decentralized Multi-Agent Control
    Qingrui Zhang, Hao Dong and Wei Pan
    International Conference on Distributed Artificial Intelligence (DAI) 2020 (Oral)
    [Paper]
    Role-Wise Data Augmentation for Knowledge Distillation
    Jie Fu, Xue Geng, Zhijian Duan, Bohan Zhuang, Xingdi Yuan, Adam Trischler, Jie Lin, Chris Pal, Hao Dong
    arXiv-2004.08861 2020
    [Paper] [Code]
    DLGAN: Disentangling Label-Specific Fine-Grained Features for Image Manipulation
    Guanqi Zhan, Yihao Zhao, Bingchan Zhao, Haoqi Yuan, Baoquan Chen, Hao Dong
    arXiv:1911.09943 2019
    [Paper]
    An Artificial Intelligence Based Data-driven Approach for Design Ideation
    Liuqing Chen, Pan Wang, Hao Dong, Feng Shi, Ji Han, Yike Guo, Peter RN Childs, Jun Xiao, Chao Wu
    Journal of Visual Communication and Image Representation 2019
    [Paper]
    SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks
    Simiao Yu, Hao Dong, Felix Liang, Yuanhan Mo, Chao Wu, Yike Guo
    International Conference on Image Processing (ICIP) 2019 (Oral)
    [Paper]
    Conditional Image Synthesis Using Stacked Auxiliary Classifier Generative Adversarial Networks
    Zhongwei Yao, Hao Dong, Pan Wang, Chao Wu, Yike Guo
    Future of Information and Communications Conference (FICC) 2018
    [Paper]
    Generative Creativity: Adversarial Learning for Bionic Design
    Simiao Yu, Hao Dong, Pan Wang, Chao Wu, Yike Guo
    International Conference on Artificial Neural Networks (ICANN) Munich, Germany, 2019
    [Paper]
    Text-to-Image Synthesis via Visual-Memory Creative Adversarial Network
    Shengyu Zhang, Hao Dong, Wei Hu, Yike Guo, Chao Wu, Di Xie, Fei Wu
    Pacific Rim Conference on Multimedia (PCM) 2018
    [Paper]
    Dropping Activation Outputs with Localized First-layer Deep Network for Enhancing User Privacy and Data Security
    Hao Dong, Chao Wu, Wei Zhen, Yike Guo
    IEEE Trans. on Inform. Forensics and Security (TIFS) 2018
    [Paper]
    Towards Desynchronisation Detection in Biosignals
    Akara Supratak, Steffen Schneider, Hao Dong, Ling Li, Yike Guo
    Neural Inform. Process. Systems (NeurIPS) Time Series Workshop 2017
    [Paper] [Webpage]
    SisGAN: Semantic Image Synthesis via Adversarial Learning
    --- The world's first work for manipulating image using natural language (text-guided image manipulation)
    Hao Dong*, Simiao Yu*, Chao Wu, Yike Guo
    International Conference on Computer Vision (ICCV) 2017
    [Paper]
    TensorLayer: A Versatile Library for Efficient Deep Learning Development
    ---Winner of the Best Open Source Software Award
    Hao Dong, Akara Supratak, Luo Mai, Fangde Liu, Axel Oehmichen, Simiao Yu, Yike Guo
    ACM Multimedia (MM) Open Source 2017
    [Paper] [Code] [Organisation] [Documentation]
    DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction
    Guang Yang*, Simiao Yu*, Hao Dong, Greg Slabaugh, Pier Luigi Dragotti, Xujiong Ye, Fangde Liu, Simon Arridge, Jennifer Keegan, Yike Guo, David Firmin
    IEEE Trans. Med. Imag. (TMI) 2017
    [Paper] [Code]
    Deep De-Aliasing for Fast Compressive Sensing MRI
    Simiao Yu*, Hao Dong*, Guang Yang, Greg Slabaugh, Pier Luigi Dragotti, Xujiong Ye, Fangde Liu, Simon Arridge, Jennifer Keegan, David Firmin, Yike Guo
    arXiv:1705.07137 2017
    [Paper]
    I2T2I: Learning Text to Image Synthesis with Textual Data Augmentation
    Hao Dong, Jingqing Zhang, Douglas McIlwraith, Yike Guo
    International Conference on Image Processing (ICIP) 2017 (Oral)
    [Paper] [Code]
    Unsupervised Image-to-Image Translation with Generative Adversarial Networks
    Hao Dong, Paarth Neekhara, Chao Wu, Yike Guo
    arXiv:1701.02676 2017
    [Paper] [Code]
    DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG
    Akara Supratak, Hao Dong, Chao Wu, Yike Guo
    IEEE Trans. on Neural Systems and Rehabilitation Eng. (TNSRE) 2017
    [Paper] [Code]
    Mixed Neural Network Approach for Temporal Sleep Stage Classification
    Hao Dong, Akara Supratak, Wei Pan, Chao Wu, Paul M Matthews, Yike Guo
    IEEE Trans. on Neural Systems and Rehabilitation Eng. (TNSRE) 2017
    [Paper]
    Automatic Brain Tumor Detection and Segmentation Using U-Net Based Fully Convolutional Networks
    Hao Dong, Guang Yang, Fangde Liu, Yuanhan Mo, Yike Guo
    Medical Image Understanding and Analysis (MIUA) 2017 (Oral)
    [Paper]
    TensorDB: Database Infrastructure for Continuous Machine Learning
    Fangde Liu, Axel Oehmichen, Jingqing Zhang, Kai Sun, Hao Dong, Yuanman Mo, Yike Guo
    International Conference Artificial Intelligence (ICAI) 2017
    [Paper]
    DropNeuron: Simplifying the Structure of Deep Neural Networks
    Wei Pan, Hao Dong, Yike Guo
    arXiv:1606.07326 2016
    [Paper] [Code]