Full Bayesian Parameter Estimation
Introduction The prevoius post discussed Maximum Likelihood Estimation (MLE) and Maximum A Posteriori (MAP) Estimation , focusing on the two properties of parameter estimation methods: whether an estimation method is a point or density estimate and whether an estimation method uses prior knowledge. While MLE does not use prior knowledge, MAP integrates prior knowledge into parameter estimation. Both parameter estimation methods are a point estimate. The current post focuses on Bayesian parameter estimation. Unlike the previous approaches, the Bayesian estimation is a density estimation method that produces a full distribution of parameter values as its outcome. As the name of the "Bayesian estimate" suggests, the formula is based on the Bayes theorem, which solves a conditional probability \( P(\theta|D) \). The mathematical definition of full Bayesian parameter estimation is below: \[ P(\theta | D) = \frac{P(D | \theta)...