Beyond Single-Task Reinforcement Learning: Robustness, Generalization, and Fast Adaptation
Prof. Ping-Chun Hsieh
Computer Science Dept., National Yang Ming Chiao Tung University, Taiwan
Prof. Yu-Shuen Wang
Computer Science Dept., National Yang Ming Chiao Tung University, Taiwan
Abstract
Reinforcement learning (RL) has historically been developed under a single-task assumption: an agent is trained and evaluated within the same Markov Decision Process (MDP). This paradigm enabled major advances in theory and practice, but it is misaligned with real-world deployments where agents must generalize across environments, adapt to new tasks, transfer knowledge across domains, and handle uncertainty or shifting objectives. In recent years, several research directions have emerged to address these gaps, including robust RL (uncertain dynamics), meta RL (fast adaptation under a task distribution), cross-domain RL (mismatched state/action spaces), multi-objective RL (vector rewards and unknown preferences), and reward-free RL (learning reusable representations without rewards). These areas are often studied in isolation, with distinct terminologies and assumptions, making it difficult for researchers and practitioners to build a unified mental model. This tutorial aims to provide a unified, structured view. We frame these settings as instances of generalization across MDPs, highlight their shared core ideas (representation learning, uncertainty handling, adaptation, compositionality), and clarify key differences (what shifts, what is observed, what is optimized). We will cover representative algorithmic techniques, theoretical lenses, and practical considerations, and discuss how these directions connect to modern application demands (robust deployment, transfer, continual updates, multi-criteria decision making). This allows the community to better keep pace with the rapid expansion of the RL landscape.
Biography
Dr. Ping-Chun Hsieh is an Associate Professor of the Department of Computer Science at National Yang Ming Chiao Tung University (NYCU). He received his B.S. and M.S. in Electrical Engineering from National Taiwan University in 2011 and 2013, respectively, and his Ph.D. degree in Electrical and Computer Engineering from Texas A&M University in 2018. His research interests include reinforcement learning, multi-armed bandits, Bayesian optimization, and wireless networks. His research received the Best Paper Awards from ACM MobiHoc 2020 and ACM MobiHoc 2017. He is a recipient of P.-C. Huang Junior Chair Professorship from NYCU, NSTC 2030 Emerging Young Scholar Program, MOST Young Scholar Fellowship, NYCU CS Outstanding Young Scholar Award, Pulsaris AI Chair Professorship, and the Government Scholarship to Study Abroad from the Ministry of Education, Taiwan. Dr. Hsieh is a Fellow of the Higher Education Academy (FHEA).
Prof. Yu-Shuen Wang is a professor of the Department of Computer Science at National Yang Ming Chiao Tung University. He received his PhD degree from the Visual System Laboratory, National Cheng Kung University, Tainan, Taiwan, ROC, in 2010. Currently, He leads the Computer Graphics and Visualization Lab at the Institute of Data Science and Engineering. His research interests include Computer Graphics, Computer Vision, Machine Learning, and Reinforcement Learning. He was honored with the prestigious Wu Da-Yu Memorial Award and the NCTU EECS Outstanding Young Scholar Award.
