Parameter Estimation - Grid-Based Approach

introduction

Approximate posterior distribution $P_r$The simplest numerical approach to$($$\theta$$|$$d$$)$ is to base all computations on an array of (not necessarily normalized)density values​​evaluated on a regular grid. Integrals over distributions are approximated as simple sums, and computing other standard derived quantities is fairly simple.

Pros and cons of the grid approach

Advantages of grid-based methods

1. Accuracy/speed, since regular spacing of evaluations means minimal redundancy - the estimation error of a standard derivation decreases with increasing sample size;
2. Repeatability , because (assuming only that the grid parameters are held fixed, the posterior density can be directly estimated) the result will be the same if recomputed;
3. Simple, because the algorithm for building a regular grid is not complicated;
4. There is no normalization requirement, since a grid of unnormalized posterior values ​​can be used.

Disadvantages, or limitations, of grid-based methods

1. Requires some extrinsic knowledge of the range of parameter values ​​for which the posterior is important (while the prior range may be sufficient, it is usually not enough);
2. If there is a posteriori error, the underlying "systematic" error is periodic, or varies on a scale commensurate with the spacing between grid points;
3. This approach is only suitable for low-dimensional problems (any problem with more than 3-4 parameters is computationally infeasible);
4. Writing code to handle grids of arbitrary dimensions can be unwieldy.

The algorithms required to generate the posterior sample arrays and process them into useful output are very simple, certainly much simpler than their purely mathematical expressions, especially in multidimensional environments. It is much more natural to describe them in terms of algorithms. Here the binary posterior distribution $P_r(\theta |d)=P_r(x,y|d)$ , where$N_p=2，i=(x,y)$ ; the generalization to higher (or lower) dimensional problems is conceptually straightforward.

mesh generation

For grid-based methods, for the distribution $P_r(x,y|d)$ (ie, the ability to assess) is not a sufficient starting point. Some information is needed to decide the range of parameter values ​​to consider: it must be possible to determine$x_{min}、x_{max}、y_{min}Japanese{\_}_{max}$P_r $P_r(x_{min}\le x\le x_{max}，y_{min}\le y\le y_{max}|d)$ $\ll$ 1(immediately,$x_{min}、x_{max}、y_{min}Japanese{\_}_{max}$The bounded region contains almost all probabilities). However, simply making this range arbitrarily large is not enough, since most , if not all, of the grid points will fall in low probability regions . There are general algorithms for doing this (sampling methods), but to have to resort to these would defeat the purpose of grid-based methods to a large extent. So here assume $x_{min}、x_{max}、y_{min}Japanese{\_}_{max}$Reasonable values ​​for are known from some external information, although it cannot be overemphasized that if such constraints are not readily available, then grid-based methods can immediately become useless.

The next decision to make is the resolution of the grid, given by the number of columns $N_c$and the number of rows $N_r$definition. This choice is subject to a trade-off between accuracy and speed.

The minimum value of each dimension is 10; for the posterior distribution often encountered in real problems, any value greater than $10^{A\; value\; of\; 2}$ is usually unnecessary. As with product lines, this is subject to a certain amount of trial and error.

Therefore, the grid covers $x_{\min }\le x\le x_{\max }$和$y_{\min }\le y\le y_{\max }$range with $N_c\times N_r$An area is $\Delta x\_\times$A cell array of$\Delta$$y$$\Delta x\_=\left(x_{\max }-x_{\min }\right)/2$ ,$y\_=\left(y_{\max }-y_{\min }\right)/2$ . From this point, follow the algorithm specified below:

1. For each column combination, c ( ∈ { 1 , 2 , … , N c } ) c\left(\in\left\{1,2, \ldots, N_{\mathrm{c}}\right\ }\right)c(∈{ 1,2,…,Nc​} ) and line combinations,r ( ∈ { 1 , 2 , … , N r } ) r\left(\in\left\{1,2, \ldots, N_{\mathrm{r}}\right\}\ right)r(∈{ 1,2,…,Nr​} ) to calculate,

$\left(x_c,y_r\right)=\left[x_{\min }+\frac{c-1/2}{N_c}\left(x_{\max }-x_{\min }\right),y_{\min }+\frac{r-1/2}{N_r}\left(y_{\max }-y_{\min }\right)\right],$

where the points lying in the middle of each grid cell are selected.

2. For each element in the array, compute the unnormalized posterior probability,

$p_{c,r}^{\prime }=\mathrm{Pr}(x,y)\mathrm{Pr}(d)\mid x,y).$

3. Numerical normalization of posterior samples by calculation

$p_{c,r}=\frac{p_{c,r}^{\prime }}{{\sum }_{c=1}^{N_{\mathrm{c}}}{\sum }_{r=1}^{N_{\mathrm{r}}}{p}_{c,r}^{\prime }}.$

Although this step is sometimes unnecessary, it is numerically cheap and simplifies subsequent analysis

A piecewise constant approximation to the posterior probability is now provided by

$\begin{array}{c}\mathrm{Pr}(x,y\mid \mathbf{d})\simeq \frac{1}{\left(x_{\max }-x_{\min }\right)\left(y_{\max }-y_{\min }\right)}{\sum }_{c=1}^{N_{\mathrm{c}}}{\sum }_{r=1}^{N_{\mathrm{r}}}\\ Th\left[x-\left(x_c-\Delta x/2\right)\right]Th\left[\left(x_c+\Delta x/2\right)-x\right]Th\left[y-\left(y_c-y/2\right)\right]Th\left[\left(y_c+y/2\right)-y\right]p_{c,r},\end{array}$

Zero outside the area covered by the grid. More complex interpolation schemes can also be used to get from $p_{c,r}$to a distribution defined for all x and y, but the point is that the continuous function $P_r(x_c,y_r|d)$ is now encoded (albeit approximately) in the finite set of numbers$p_{c,r}$middle.

references

https://docslib.org/doc/8720468/parameter-estimation-daniel-mortlock-mortlock-ic-ac-uk-last-modi-ed-september-12-2013

Feroz, F., Hobson, M. P., and Bridges, M. (2009). MULTINEST: an efficient and robust Bayesian inference tool for cosmology and particle physics. Monthly Notices of the Royal Astronomical Society, 398, 1601–1614.

Gelman, A. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–511.

Skilling, J. (2004). Nested sampling. In AIP Conference Proceedings of the 24th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, volume 735 of Lecture Notes in Physics, Berlin Springer Verlag, pages 395–405.