Bayesian Inference for Non Linear Models

Inference, Estimation and Learning, University of Michigan

Problem Description

Implementation and Analysis of the following Algorithms for Inference on parameters of Satellite Dynamics model, SIR Disease Model and a Banana Distribution:

Metropolis Hastings Markov Chain Monte Carlo
Adaptive Metropolis Monte Carlo
Delayed Rejection Monte Carlo
Delayed Rejection Adaptive Metropolis Monte Carlo(DRAM)
Laplace Approximation

Satellite Dynamics

Disease Dynamics

Project overview

Banana Distribution: Implement the above mentioned algorithms and use a Banana distribution to check the implementation.

Satellite Dynamics: The goal of learning is to learn some control parameters and some products of inertia.

SIR Model : Goal is to learn the parameters of a benchmark 3 state model for the spread of disease (SIR model)

Key achievements

Implemented the listed algorithms and verified implementation validity
Analyzed the results using various heuristics including:
- - Mixing plots
  - Acceptance ratios
  - Autocorrelation
  - Marginals
Modify the dynamics to arrive at and study the case of non-identifiable parameters

Algorithms Used

The algorithms (shown here) used have been implemented from scratch

Metropolis-Hastings Markov Chain Monte Carlo

Adaptive Metropolis

Delayed Rejection

DRAM

Results - Satellite Dynamics and Disease Dynamics - Posterior Predictives

The posterior predictives for the learnt parameters is shown here. For the Disease Dynamics, the occurence of Non Identifiable parameters in a system is also explored, where we see it's effects on the marginals.

The presence of Non Identifiable parameters can be inferred through the emergence of an asymptotic distribution for the marginals

Analyses

Analysis on the behaviors of the algorithms was performed using various heuristics. An illustration of the heurisitics used is shown here for the case of the banana distribution. Similar Studies was done for Satellite Dyamics and Disease Dynamics Inferences.

Banana Distribution - Mixing Plots and Acceptance Ratios

Banana Distribution - Autocorrelations

Banana Distribution - Marginal Plots