Statistics: Saddlepoint Approximation

**bearkiller** · #1 (**permalink**) 02-06-2010

Introduction: Saddlepoint approximation is used to approximate the density function of some distributions. It is an interesting approximation not only because of its efficiency but also because its motivation comes from some basic ideas which astonish me.

Laplace approximation:
Now, we want to approximate $f(x)$ . Denote $h(x) = \log f(x)$ and use Taylor expansion around $x_0$ we have $f(x) = \exp \{ h(x_0) +(x-x_0)h'(x_0) + \frac{(x-x_0)^2}{2}h''(x_0) \}$
If we chose $x_0 = \hat{x}$ which is a maximum, we have $\int{f(x)dx} = \exp\{h(\hat{x})\}\int{\exp\{-\frac{(x-\hat{x})^2}{\frac{2}{-h''(\hat{x})}}\}dx} = \exp\{ h(\hat{x}) \}\{ \frac{2\pi}{-h''(\hat{x})} \}^{1/2}$
Note that we don't need to compute anything because we have the formula of normal distribution which has mean $\hat{x}$ and variance $1/-h''(\hat{x})$

Saddlepoint approximation
Now, we want to approximate the density function $f(x)$ . Let $\phi(t)$ be the moment generating function and is defined as $\phi(t) = \int_{-\infty}^{\infty}{\exp\{ tx \}f(x)dx}$ . We actually can obtain $f(x)$ from $\phi(x)$ by using inverse formula $f(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty}{\phi(it)\exp\{itx\}dt}$
Denote $K(t) = \log \phi(t)$ and change variable, we have $f(x) = \frac{1}{2\pi i}\int_{\tau-i\infty}^{\tau+i\infty}{\exp\{K(t) - tx\}dt}$
There is a theorem in complex analysis that the integral is the same over all paths that are parallel to the imaginary axis in the neighbourhood of zero where $\phi(t)$ exists. So, we can chose $\tau$ .
Now, using previous idea, we chose $\tau = \hat{t}$ which is the solution of $K'(t)=x$ and get $f(x) =\exp\{ K(\hat{t})-\hat{t}x \}\{ \frac{1}{2\pi K''(\hat{t})} \}^{1/2}$
Note that here we need $K''(\hat{t}(x))>0$ ( maybe because the integral in complex case is different from real case). Furthermore, $\hat{t}(x)$ is a constant in the real direction and is a minimum in the imaginary direction. That's why this method is called saddlepoint approximation.

Noncentral chi-squared The density function of noncentral chi-squared is
$f(x) = \sum_{k=0}^\infty{\frac{x^{p/2+k-1}e^{-x/2}}{\Gamma(p/2+k)2^{p/2+k}}\frac{\lambda^k e^{-\lambda}}{k!}}$ where $p$ is the degree of freedom and $\lambda$ is the noncentrality parameter.
Clearly, it's impossible for us to get the exact value of this density. However, the moment generating function of this distribution has a nice form $\phi(t)=\frac{e^{2\lambda t/(1-2t)}}{(1-2t)^{p/2}}$ and that's how Saddlepoint approximation will give a hand in here. Remember that we have to renormalize the approximation because we are approximating the density function.

Reference: Gousti and Casella, "Explaining the Saddlepoint Approximation", The American Statistician, Vol. 5, No. 3, 1999, pp. 216-224