Bayesian Additive Regression Tree Copula Processes For Scalable Distributional Prediction

Abstract

We show how to construct the implied copula process of response values from a Bayesian additive regression tree (BART) model with prior on the leaf node variances. This copula process, defined on the covariate space, can be paired with any marginal distribution for the dependent variable to construct a flexible distributional BART model. Bayesian inference is performed via Markov chain Monte Carlo on an augmented posterior, where we show that key sampling steps can be realised as those of Chipman et al. (2010), preserving scalability and computational efficiency even though the copula process is high-dimensional. The posterior predictive distribution from the copula process model is derived in closed form as the push-forward of the posterior predictive distribution of the underlying BART model with an optimal transport map. Under suitable conditions, we establish posterior consistency for the regression function and posterior means and prove convergence in distribution of the predictive process and conditional expectation. Simulation studies demonstrate improved accuracy of distributional predictions compared to the original BART model and leading benchmarks. Applications to five real datasets with 506 to 515,345 observations and 8 to 90 covariates further highlight the efficacy and scalability of our proposed BART copula process model.

Short bio

Michael Smith has held the Chair of Management in Econometrics at MBS since 2007. Michael completed his PhD at the Australian Graduate School of Management at UNSW. Prior to joining MBS, he held positions at Monash University and the University of Sydney. He has also held visiting positions at Ludwig Maximilians University in Munich, the Wharton School, McCombs School of Business (UT Austin), London Business School and UCL. Past major awards include an Alexander von Humboldt fellowship and an ARC Future Fellowship. Michael’s research focuses on developing Bayesian methods for the analysis of large and complex datasets that arise in business, economics and elsewhere. On the applied side has worked on marketing models, neuroimaging, and macroeconomic and business forecasting. He has a long-standing interest in the electricity markets, including the modelling and forecasting of demand and spot prices.