On Singular Bayesian Inference of Underdetermined Quantities

EasyChair Preprint 13763

Abstract

When the quantities of interest remain underdetermined a posteriori, we would like to draw inferences for at least one particular solution. Can we do that in a Bayesian way? What is a probability distribution over an underdetermined quantity? How to get a posterior for one particular solution from a posterior for infinitely many underdetermined solutions? Guided by invariant underdetermined ill-posed inverse problems, we find that a probability distribution over an underdetermined quantity is non-absolutely continuous, partially improper wrt the initial reference measure but proper wrt its restriction to its support. Thus, it is necessary and sufficient to choose the prior restricted reference measure to assign partially improper priors by e.g. maximum entropy and the posterior restricted reference measure to obtain the proper posterior for one particular solution. We can then work with underdetermined models such as Hoeffding-Sobol expansions seamlessly, especially to effectively counter the curse of dimensionality within nonparametric inverse problems. We demonstrate Singular Bayesian Inference (SBI) at work in an advanced Bayesian Optimization application: dynamic pricing. Such a nice generalization of Bayesian-maxentropic inference could motivate many theoretical and practical developments.

Keyphrases: Bayesian inference, Improper probability distributions, Invariance, MaxEnt, inverse problems

@booklet{EasyChair:13763,
  author    = {Fabrice Pautot},
  title     = {On Singular Bayesian Inference of Underdetermined Quantities},
  howpublished = {EasyChair Preprint 13763},
  year      = {EasyChair, 2024}}