Mathematical theory of deep learning

Abstract

Deep learning has been widely applied and brought breakthroughs in dealing with big data from speech recognition, computer vision, natural language processing, and many other domains.  It is based on deep neural networks, often with structures designed for various purposes. Compared with its success in practical applications, it is not well understood in theory.  A theoretical foundation is desired for understanding modelling and generalization abilities of deep learning models with network architectures and structures. This walk will start with deep convolutional neural networks (CNNs) which are induced by convolutions. The convolutional architecture gives essential differences between deep CNNs and the classical fully connected neural networks. We describe a mathematical theory for deep CNNs associated with the rectified linear unit activation. In particular, we discuss approximation and learning abilities of deep CNNs and some other related deep learning models.

Presenter

Ding-Xuan Zhou is Professor and Head of School of Mathematics and Statistics, University of Sydney. Before moving to Australia, he was a Chair Professor in the School of Data Science and Department of Mathematics at City University of Hong Kong, serving also as Director of the Liu Bie Ju Centre for Mathematical Sciences (2019-22), Associate Dean of School of Data Science (2018-22), and Head of Department of Mathematics (2006-12). His recent research is focused on theory of machine learning and deep neural networks, which has attracted two ARC DPs. He is an Editor-in-Chief of the journals ``Analysis and Application'' and “Mathematical Foundations of Computing” and serves editorial boards of more than ten journals. He was rated in 2014-2017 by Thomson Reuters/Clarivate Analytics as a highly-cited researcher.