Cramer-Rao Lower Bound

Resources: page=35

---

Classical Estimation of a deterministic (fixed, though unknown) parameter.

Assume that our observations, $x$, are distributed according to the pdf $p(x;\theta )$, and also assume that the pdf satisfies the following regularity condition

$$E\left\{\frac{\partial }{\partial \theta }\log (p(x;\theta ))\right\}=0,\forall \theta$$

Then the variance of any Minimum Variance Unbiased Estimator $\hat{\theta }$ of the paramter $\theta$ will be bounded blow by

$$Var(\hat{\theta })\ge \frac{1}{-E\left[\frac{{\partial }^2}{\partial {\theta }^2}\log (p(x;\theta ))\right]}=\frac{1}{E\left[{\left(\frac{\partial }{\partial \theta }\log (p(x;\theta )\right)}^2\right]}$$

---

While in vector form we have

$$var({\theta }_i)\ge [I^{-1}(\Theta ){]}_{i{i}^{\prime }}$$

Where $I$ is the Fisher Matrix

---

Efficient Estimator

Example

Check the regularity condition

$$\begin{array}{rl}& \mathbb{E}\left\{\frac{\partial }{\partial \theta }\log (p(x;\theta ))\right\}\\ & {\int }_{{\mathbb{R}}^N}p(x;\theta )\frac{\partial }{\partial \theta }\log (p(x;\theta ))dx\\ & \dots ?\end{array}$$