Theoretical Insights And Algorithm Design For Auc Optimisation And Contrastive Representation Learning

Abstract

This talk covers two recent developments in machine learning that address challenges beyond standard accuracy-based objectives. In the first part, we focus on AUC maximisation, which is more suitable than accuracy for learning with imbalanced data. We present novel min-max reformulations and efficient stochastic primal-dual algorithms that decouple the pairwise structure of AUC, enabling scalable optimisation in both streaming and deep learning settings.

In the second part, we turn to contrastive representation learning (CRL), a foundational approach in self-supervised learning. We develop new generalisation error bounds for CRL using Rademacher complexity and vector-contraction inequalities, addressing key gaps between theory and empirical success. Our results show that, under suitable conditions, the number of negative samples has only a mild effect on generalisation, and we also quantify how contrastive representations support downstream classification tasks.

Together, these results offer theoretical insights and algorithmic tools for learning in settings where accuracy alone is not enough—whether due to imbalanced data or the need for strong representations from unlabelled data.

Biography

Yiming Ying is a Professor at the School of Mathematics and Statistics at the University of Sydney. Previously, he was a tenured Professor in the Department of Mathematics and Statistics and an affiliated faculty member in the Department of Computer Science at SUNY Albany. His research interests span statistical learning theory, trustworthy machine learning, and optimization. He has received the University at Albany’s Presidential Award for Excellence in Research and Creative Activities (2022) and the SUNY Chancellor’s Award for Excellence in Scholarship and Creative Activities (2023). His research has been supported by various agencies including the Australian Research Council (ARC), NSF, IBM, the UK Engineering and Physical Sciences Research Council (EPSRC), and the NHS Foundation Trust.