Wednesday / CS Seminar

February 21st / 12:00 PM

In Person / Mudd 3514

Speaker

Yuan Cao, University of Hong Kong

Talk Title

Understanding Deep Learning Through Phenomena Discovery and Explanation

Abstract

Deep learning has achieved great success in many applications. However, the success of deep learning has not been well understood in theory. In this talk, I will discuss some recent efforts to bridge the gap between theory and practice through phenomenon discovery and explanation. In the first part of this talk, I will discuss the phenomenon of "benign overfitting" in deep learning, and present our recent results characterizing benign and harmful overfitting in training convolutional neural networks (CNNs). In the second part of the talk, I will discuss the recently discovered phenomenon on the generalization gap between Adam and stochastic gradient descent in image classification tasks. I will present an intuitive explanation for this generalization gap and provide a rigorous theoretical guarantee to support the explanation. Overall, this talk will provide insights into the "feature learning" procedure of neural networks, and how it is related to various interesting phenomena in deep learning.

Biography

Yuan Cao is an assistant professor in the Department of Statistics and Actuarial Science and Department of Mathematics at the University of Hong Kong. Before joining HKU, he was a postdoctoral scholar at UCLA. He received his B.S. from Fudan University and Ph.D. from Princeton University. Yuan's research interests include deep learning theory, non-convex optimization, and high-dimensional statistics. He has published research papers in top machine learning journals (JMLR, ML, TMLR) and conferences (NeurIPS, ICML, ICLR, COLT, AAAI, IJCAI, etc), including a spotlight presentation in NeurIPS 2019, a long talk in ICML 2021, and an oral presentation in NeurIPS 2022.

Research Interests/Area

deep learning theory, non-convex optimization, high-dimensional statistics