Sample Variance Unbiased for SRS

Note

For a SRSWOR on a finite population, the sample variance \(\hat\sigma^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X})^2\) is a biased estimator for population variance \(\sigma^2\).

Expanding the square:

\[ \begin{align} \hat\sigma^2 &= \frac{1}{n}\sum_{i=1}^n (X_i - \bar{X})^2\\ &= \frac{1}{n} \left[\sum_{i=1}^{n} X_i^2 - 2 \bar{X} \sum_{i=1}^{n} X_i + n\bar{X}^2\right]\\ &= \frac{1}{n} \left[\sum_{i=1}^{n} X_i^2 - 2 n \bar{X}^2 + n\bar{X}^2\right]\\ &= \frac{1}{n} \sum_{i=1}^{n} X_i^2 - \bar{X}^2 \\ \end{align} \]

Thus, we know that:

\[ \begin{align} \mathbb{E}(\hat\sigma^2) &= \frac{1}{n} \left[\sum_{i=1}^{n} \mathbb{E}[X_i^2] - \mathbb{E}[\bar{X}^2] \right]\\ &= \frac{1}{n} \left[\sum_{i=1}^{n} \left(\text{Var}(X_i) + (\mathbb{E}[X_i])^2 \right) - \left(\text{Var}(\bar{X}) + (\mathbb{E}[\bar{X}])^2\right)\right] \end{align} \] The above uses the fact that the variance shortcut \(\text{Var}(Y) = \mathbb{E}[Y^2] - (\mathbb{E}[Y])^2\) can be rearranged to obtain: \(\mathbb{E}[Y^2] = \text{Var}(Y) + (\mathbb{E}[Y])^2\). Subbing the results obtained in lecture 9 for \(\color{red}{\text{Var}(\bar{X})}\), we get

\[ \begin{align} &= \frac{1}{n} \sum_{i=1}^{n} \left[ \sigma^2 + \mu^2 - {\color{red} \frac{\sigma^2}{n} \left( 1 - {\frac{n - 1}{N - 1}} \right)} - \mu^2 \right]\\ &= \frac{1}{n} \left[ n\sigma^2 + \cancel{n\mu^2} - \sigma^2 \left( 1 - {\frac{n - 1}{N - 1}} \right) - \cancel{n\mu^2} \right]\\ &= \sigma^2 - \frac{\sigma^2}{n} \left( 1 - {\frac{n - 1}{N - 1}} \right) \\ &= \sigma^2 - \frac{\sigma^2}{n} \left( {\frac{N- 1 - n + 1}{N - 1}} \right) \\ &= \sigma^2 \left( \frac{n-1}{n} \right) \left( {\frac{N}{N - 1}} \right) \end{align} \]