Publications

* denotes equal contribution and α-β order denotes alphabetical authorship ordering

Conference Publications

• DPO Meets PPO: Reinforced Token Optimization for RLHF
  Han Zhong*, Zikang Shan*, Guhao Feng*, Wei Xiong*, Xinle Cheng, Li Zhao, Di He, Jiang Bian, Liwei Wang
  International Conference on Machine Learning (ICML) 2025
  ICML 2024 Workshop on Models of Human Feedback for AI Alignment
  

• BRiTE: Bootstrapping Reinforced Thinking Process to Enhance Language Model Reasoning
  Han Zhong, Yutong Yin, Shenao Zhang, Xiaojun Xu, Yuanxin Liu, Yifei Zuo, Zhihan Liu, Boyi Liu, Sirui Zheng, Hongyi Guo, Liwei Wang, Mingyi Hong, Zhaoran Wang
  International Conference on Machine Learning (ICML) 2025
  

• The Sample Complexity of Online Strategic Decision Making with Information Asymmetry and Knowledge Transportability
  Jiachen Hu, Rui Ai, Han Zhong, Xiaoyu Chen, Liwei Wang, Zhaoran Wang, Zhuoran Yang
  International Conference on Machine Learning (ICML) 2025
  

• Self-Exploring Language Models: Active Preference Elicitation for Online Alignment
  Shenao Zhang, Donghan Yu, Hiteshi Sharma, Han Zhong, Zhihan Liu, Ziyi Yang, Shuohang Wang, Hany Hassan, Zhaoran Wang
  Transactions on Machine Learning Research (TMLR) 2025
  

• Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity
  (α-β order) Guhao Feng, Han Zhong
  Conference on Neural Information Processing Systems (NeurIPS) 2024
  ICML Workshop on Aligning Reinforcement Learning Experimentalists and Theorists (ARLET) 2024
  

• Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithm
  Miao Lu*, Han Zhong*, Tong Zhang, Jose Blanchet
  Conference on Neural Information Processing Systems (NeurIPS) 2024
  ICML Workshop on Aligning Reinforcement Learning Experimentalists and Theorists (ARLET) 2024
  

• Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regret
  Han Zhong*, Jiachen Hu*, Yecheng Xue, Tongyang Li, Liwei Wang
  International Conference on Machine Learning (ICML) 2024
  

• Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint
  Wei Xiong*, Hanze Dong*, Chenlu Ye*, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, Tong Zhang
  International Conference on Machine Learning (ICML) 2024
  ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models
  

• Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
  Rui Yang*, Xiaoman Pan*, Feng Luo*, Shuang Qiu*, Han Zhong, Dong Yu, Jianshu Chen
  International Conference on Machine Learning (ICML) 2024
  

• Combinatorial Multivariant Multi-Armed Bandits with Applications to Episodic Reinforcement Learning and Beyond
  Xutong Liu, Siwei Wang, Jinhang Zuo, Han Zhong, Xuchuang Wang, Zhiyong Wang, Shuai Li, Mohammad Hajiesmaili, John C.S. Lui, Wei Chen
  International Conference on Machine Learning (ICML) 2024
  

• A3S: A General Active Clustering Method with Pairwise Constraints
  Xun Deng, Junlong Liu, Han Zhong, Fuli Feng, Chen Shen, Xiangnan He, Jieping Ye, Zheng Wang
  International Conference on Machine Learning (ICML) 2024
  

• Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
  Rui Yang*, Han Zhong*, Jiawei Xu*, Amy Zhang, Chongjie Zhang, Lei Han, Tong Zhang
  International Conference on Learning Representations (ICLR) 2024
  

• Sample-efficient Learning of Infinite-horizon Average-reward MDPs with General Function Approximation
  Jianliang He, Han Zhong, Zhuoran Yang
  International Conference on Learning Representations (ICLR) 2024
  

• Horizon-Free and Instance-Dependent Regret Bounds for Reinforcement Learning with General Function Approximation
  Jiayi Huang, Han Zhong, Liwei Wang, Lin F. Yang
  International Conference on Artificial Intelligence and Statistics (AISTATS) 2024
  

• A Theoretical Analysis of Optimistic Proximal Policy Optimization in Linear Markov Decision Processes
  Han Zhong, Tong Zhang
  Conference on Neural Information Processing Systems (NeurIPS) 2023
  

• Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coverage
  (α-β order) Jose Blanchet, Miao Lu, Tong Zhang, Han Zhong
  Conference on Neural Information Processing Systems (NeurIPS) 2023 (short version)
  

• Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Exploration
  Zhihan Liu*, Miao Lu*, Wei Xiong*, Han Zhong, Hao Hu, Shenao Zhang, Sirui Zheng, Zhuoran Yang, Zhaoran Wang
  Conference on Neural Information Processing Systems (NeurIPS) 2023
  

• Tackling Heavy-Tailed Rewards in Reinforcement Learning with Function Approximation: Minimax Optimal and Instance-Dependent Regret Bounds
  Jiayi Huang, Han Zhong, Liwei Wang, Lin F. Yang
  Conference on Neural Information Processing Systems (NeurIPS) 2023
  

• A Reduction-based Framework for Sequential Decision Making with Delayed Feedback
  Yunchang Yang*, Han Zhong*, Tianhao Wu*, Bin Liu, Liwei Wang, Simon S. Du
  Conference on Neural Information Processing Systems (NeurIPS) 2023
  

• Posterior Sampling for Competitive RL: Function Approximation and Partial Observation
  Shuang Qiu*, Ziyu Dai*, Han Zhong, Zhaoran Wang, Zhuoran Yang, Tong Zhang
  Conference on Neural Information Processing Systems (NeurIPS) 2023
  

• Provable Sim-to-real Transfer in Continuous Domain with Partial Observations
  Jiachen Hu*, Han Zhong*, Chi Jin, Liwei Wang
  International Conference on Learning Representations (ICLR) 2023
  

• Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
  Wei Xiong*, Han Zhong*, Chengshuai Shi, Cong Shen, Liwei Wang, Tong Zhang
  International Conference on Learning Representations (ICLR) 2023
  

• Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power
  Binghui Li*, Jikai Jin*, Han Zhong, John E. Hopcroft, Liwei Wang
  Conference on Neural Information Processing Systems (NeurIPS) 2022
  

• Pessimistic Minimax Value Iteration: Provably Efficient Equilibrium Learning from Offline Datasets
  Han Zhong*, Wei Xiong*, Jiyuan Tan*, Liwei Wang, Tong Zhang, Zhaoran Wang, Zhuoran Yang
  International Conference on Machine Learning (ICML) 2022
  ICLR 2022 Workshop on Gamification and Multiagent Solutions
  

• Nearly Optimal Policy Optimization with Stable at Any Time Guarantee
  Tianhao Wu*, Yunchang Yang*, Han Zhong*, Liwei Wang, Simon S. Du, Jiantao Jiao
  International Conference on Machine Learning (ICML) 2022
  

• Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation
  Xiaoyu Chen*, Han Zhong*, Zhuoran Yang, Zhaoran Wang, Liwei Wang
  International Conference on Machine Learning (ICML) 2022
  

• A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Games
  Wei Xiong, Han Zhong, Chengshuai Shi, Cong Shen, Tong Zhang
  International Conference on Machine Learning (ICML) 2022
  ICLR 2022 Workshop on Gamification and Multiagent Solutions
  

• A Reduction-Based Framework for Conservative Bandits and Reinforcement Learning
  Yunchang Yang*, Tianhao Wu*, Han Zhong*, Evrard Garcelon, Matteo Pirotta, Alessandro Lazaric, Liwei Wang, Simon S. Du
  International Conference on Learning Representations (ICLR) 2022
  

• Breaking the Moments Condition Barrier: No-Regret Algorithm for Bandits with Super Heavy-Tailed Payoffs
  Han Zhong, Jiayi Huang, Lin F. Yang, Liwei Wang
  Conference on Neural Information Processing Systems (NeurIPS) 2021

Journal Publications

• Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopically Rational Followers?
  Han Zhong, Zhuoran Yang, Zhaoran Wang, Michael I. Jordan
  Journal of Machine Learning Research (JMLR) 2023
  ICLR 2022 Workshop on Gamification and Multiagent Solutions

Preprints

• Learning an Optimal Assortment Policy under Observational Data
  Yuxuan Han, Han Zhong, Miao Lu, Jose Blanchet, Zhengyuan Zhou
  

• Less is More: Improving LLM Alignment via Preference Data Selection
  Xun Deng, Han Zhong, Rui Ai, Fuli Feng, Zheng Wang, Xiangnan He
  

• GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond
  Han Zhong*, Wei Xiong*, Sirui Zheng, Liwei Wang, Zhaoran Wang, Zhuoran Yang, Tong Zhang
  

• Optimistic Policy Optimization is Provably Efficient in Non-stationary MDPs
  Han Zhong, Zhongren Chen, Zhuoran Yang, Zhaoran Wang, Csaba Szepesvári
  

• Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds Globally Optimal Policy
  Han Zhong, Xun Deng, Ethan X. Fang, Zhuoran Yang, Zhaoran Wang, Runze Li