# Estimation -- dealing with uncertainty! {#estimation}

```{r echo = FALSE}

H.Means <- c(200, 210)
H.SDs <- c(6, 6)
mystery <- 215

# Descriptive statistics for height at site I
Height.mean <- H.Means[1]
Height.sd <- H.SDs[1]
nValue <- length(martian$Height[martian$Site == "Site I"])

# SEM and 95% CI for height at site I
Height.SEM <- (Height.sd/sqrt(nValue))
Height.95CI <- c(Height.mean - 1.96*Height.SEM, Height.mean + 1.96*Height.SEM)

# # Values for one-sample t-tests
# distance <- 5 # Set distance away from x-bar for tests
# HypoValue <- Height.mean - distance # Set less than
# HypoValue2 <- Height.mean + distance # Set greater than.
# 
# # One-sample t-tests
# tValue <- (Height.mean - HypoValue)/Height.SEM
# tValue2 <- (Height.mean - HypoValue2)/Height.SEM
```

So far, we only discussed the sample in descriptive statistics and only the population in probability distributions. The major challenge in Statistics is to connect these two parts together. That is, given what we know about the sample, what can we say about the population? This is the subject of _statistical inference_:

<dl>
  <dt>Statistical inference</dt>
  <dd>the process of drawing conclusions about populations, i.e. scientific truths, from sample data.</dd>
</dl>

Why is this such a challenge? Simply put, the crux of the whole matter is that the sample is usually much smaller than the population. This means that it doesn't contain all the information available in the population. Any conclusions drawn from the sample will probably not exactly hold for the entire population so it's important to keep the following two questions in mind:

<dl>
  <dt>Two key questions in inferential statistics:</dt>
  <dd>
1. How confident can I be that the results I got from my sample can be generalised to the population, i.e. how much _noise_ do I have in my sample?
2. In case I see an interesting effect, is it real (due to some systematic pattern) or just due to chance, i.e. is there a real _signal_ among the noise?
  </dd>
</dl>

These two questions relate to the two major topics of inferential statistics: estimation and hypothesis testing. In estimation we measure the noise inherent in our sample, and in hypothesis testing we compare the noise to the signal by means of a _signal:noise_ ratio.

## Estimation in a nutshell

We'll return to our Martian dataset to explain the general concept of estimation. The histogram of our 10 site I Martian heights has a shape that seems to be compatible with a Normal distribution. If we want to formalise this assumption, we'd write

$$ X \sim N(\mu, \sigma^2),$$

where $X$ is the variable in our population, i.e. 'height', and '$\sim N(\mu, \sigma^2)$' means that we assume a Normal distribution with mean $\mu$ and variance $\sigma^2$.

As we saw in Chapter \@ref(cha:understandingprobability), once we accept the model of a Normal distribution, $\mu$ and $\sigma^2$ are the only unknown features of the height in our Martians population. Unfortunately, we don't know $\mu$ and $\sigma^2$, and all the information we have is contained in our sample.

It seems we have to accept the fact there is no way for us to know the exact population parameters, so we need to use our sample to get a rough idea about their values. For example, we could use the known sample mean of the Martian height ($\bar{x} = `r H.Means[1]`$ cm) instead of the unknown population mean $\mu$ for any further statements.

When we do this, the mean $\bar{x}$ is not only a descriptive statistic for the location of our sample, but now it's also an _estimate_ of the unknown population parameter $\mu$. While this seems to be obvious and only a matter of wording, it actually has some interesting conceptual implications! Once we realise that the sample that gives rise to our value $\bar{x}$ was drawn randomly and that if we took another sample, we'd probably calculate a different value, we really would like to answer 3 important questions:

<dl>
  <dt>Three key questions in estimation</dt>
  <dd>
1. Is there a systematic error in using $\bar{X}$ instead of $\mu$? i.e. Is $\bar{X}$ an unbiased estimator of $\mu$?
2. How variable is $\bar{X}$? Q2: i.e. How much does $\bar{X}$ depend on our given sample? How much would it vary if we had, by chance, taken a _different_ sample from the same population?
3. Is $\bar{X}$ the best estimator of $\mu$? i.e. Is there some other location statistic that has better properties than $\bar{X}$, with respect to Q1 and Q2?
  </dd>
</dl>

These questions are what estimation is all about! Estimation is about knowing the properties of the descriptive statistics we calculate from the sample when we use them to infer population characterisics. In particular we want to assess the uncertainty that is associated with the conclusions obtained from the data at hand. This in turn helps us to judge the reliability of our findings!

To answer the above questions there is an important concept we have to understand. If you learn only one thing from this workshop, then let it be that ... *descriptive statistics have distributions* in inference!

## Descriptive statistics have distributions

Descriptive statistics have distributions!? Let's use a simulation to see what we mean. Assume that the Martian height in our population at site I follows a Normal distribution with $\mu=`r H.Means[1]`$ cm and $\sigma=`r H.SDs[1]`$ cm.^[Recall that Roman letters refer to the sample - $\bar{x}$ is the sample mean - and Greek letters refer to the population - $\mu$ is the population mean.] Now imagine that instead of our single sample we could take _many_ samples, e.g. $m$ samples in total, and that for each sample we calculate the sample mean such that we'd have $m$ sample means: $\bar{x}_1, \bar{x}_2, \ldots, \bar{x}_m$.

Video: What happens when we take multiple, independent simple random samples from the same population?


```{r eval = FALSE, echo = FALSE}

set.seed(20140309)
df <- data.frame(obs = rnorm(500000, H.Means[1], H.SDs[1]))
df.original <- df
m <- seq(-4,4,1)
xValues <- c(H.Means[1]+(m*H.SDs[1]))
# xLabels <- paste(xValues)
sign <- c(rep("-",4), "", rep("+", 4))

xLabels <- paste0(xValues,
                 "\n$\\mu$",
                 sign,
                 abs(m),
                 "$\\sigma$")

xLabels[5] <- paste0(xValues[5],
                 "\n$\\mu$")

basePlot <-
  ggplot(df, aes ( x = obs )) +
  geom_histogram(aes(y = ..density..), binwidth = 2, fill = MidYellow) +
  scale_y_continuous(name = "Frequency", expand = c(0,0)) +
  scale_x_continuous("Height at site I (cm)",
                     breaks = xValues,
                     labels = xLabels,
                     limits = range(xValues),
                     expand = c(0,0)) +
  ggtitle("Population Distribution\nN(200,36)") +
  theme(panel.background = element_blank(),
        panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        axis.ticks.x = element_line(colour = "black"),
        axis.ticks.y = element_line(colour = "black"),
        axis.text.x = element_blank(),
        axis.title.x = element_blank(),
        axis.text.y = element_text(colour = "black"),
        plot.title = element_text(lineheight=.8, size = 11)
  )

# basePlot

basePlot +
  stat_function(fun = dnorm, args = list(mean = mean(df$obs), sd = sd(df$obs)), col = DarkBlue) +
  geom_vline(xintercept = mean(df$obs), col = DarkBlue, size = 1)  +
  geom_vline(xintercept = mean(df$obs) + sd(df$obs), col = DarkBlue, size = 0.5, linetype = 2)  +
  geom_vline(xintercept = mean(df$obs) - sd(df$obs), col = DarkBlue, size = 0.5, linetype = 2)

#######################################
#### Sampling Distribution ############
#######################################

# Function to get a random sample of a certain size a given number of times.
make_rando <- function(sample.size = 10, seed = randomness, use.seed = TRUE, iter = 8) {
  if (use.seed == TRUE) {
    for (i in 1:length(seed)) {
      set.seed(seed[i])
      index <- sample(length(tina$x), sample.size, replace = FALSE)
      tina[[paste("sample",i, sep = "")]] <- MidYellow # yellow
      tina[[paste("sample",i, sep = "")]][index] <- MidRed # red
      tina[[paste("sample",i, sep = "")]] <- factor(tina[[paste("sample",i, sep = "")]],
                                                    levels = c(MidRed, MidYellow))
    }
  } else {
    for (i in 1:iter) {
      index <- sample(length(tina$x), sample.size, replace = FALSE)
      tina[[paste("sample",i, sep = "")]] <- MidYellow # yellow
      tina[[paste("sample",i, sep = "")]][index] <- MidRed # red
      tina[[paste("sample",i, sep = "")]] <- factor(tina[[paste("sample",i, sep = "")]],
                                                    levels = c(MidRed, MidYellow))
    }
  }
  return(tina)
}

set.seed(20150804)
randomness <- round(rnorm(30, 2346434, 232454))

tina <- data.frame(x = df$obs)

# Take samples using default seed by setting use.seed = F and iter to a number of iterations.
#tina <- make_rando(sample.size = 10, use.seed = FALSE, iter =  5)
tina <- make_rando(sample.size = 10, use.seed = TRUE, iter =  30)

# Calculate the mean for each sample group:
#tina.mean <- apply(tina[2:ncol(tina)], 2, function(x) {mean(tina$x[x == TRUE])})
tina.mean <- apply(tina[2:ncol(tina)], 2, function(x) {mean(tina$x[x == MidRed])})
tina.mean <- data.frame(mean = tina.mean)
tina.mean$mean <- round(tina.mean$mean, 1)

tina.cumul <- data.frame(value = tina.mean$mean,
                         pop = "Small Population Variance",
                         n = 10)

# After making video, show the distribution of the sample means:
base_sample_means <- ggplot(tina.mean, aes (x = mean)) +
  geom_histogram(aes(y = ..density..), binwidth = 1, fill = MidRed) +
  scale_y_continuous("Frequency",
                     # breaks = seq(0, 0.3, 0.15),
                     expand = c(0,0)) +
  scale_x_continuous("Height at site I (cm)",
                     breaks = round(xValues,2),
                     labels = xLabels,
                     limits = range(xValues),
                     expand = c(0,0)) +
  ggtitle("Distribution of sample means\n(30 samples of 10 observations each)") +
  theme(panel.background = element_blank(),
        panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        axis.ticks.x = element_line(colour = "black"),
        axis.ticks.y = element_line(colour = "black"),
        axis.text.x = element_blank(),
        axis.title.x = element_blank(),
        axis.text.y = element_text(colour = "black"),
        plot.title = element_text(lineheight=.8, size = 11)
  )

base_sample_means

# Take samples using default seed by setting use.seed = F and iter to a number of iterations.
tina <- make_rando(sample.size = 30, use.seed = TRUE, iter =  30)

# Calculate the mean for each sample group:
#tina.mean <- apply(tina[2:ncol(tina)], 2, function(x) {mean(tina$x[x == TRUE])})
tina.mean <- apply(tina[2:ncol(tina)], 2, function(x) {mean(tina$x[x == MidRed])})
tina.mean <- data.frame(mean = tina.mean)
tina.mean$mean <- round(tina.mean$mean, 1)


tina.cumul <- rbind(tina.cumul,
                    data.frame(value = tina.mean$mean,
                         pop = "Small Population Variance",
                         n = 30))

# After making video, show the distribution of the sample means:
base_sample_means <- ggplot(tina.mean, aes (x = mean)) +
  geom_histogram(aes(y = ..density..), binwidth = 1, fill = MidRed) +
  scale_y_continuous("Frequency",
                     breaks = seq(0, 0.3, 0.15),
                     expand = c(0,0)) +
  scale_x_continuous("Height at site I (cm)",
                     breaks = round(xValues,2),
                     labels = xLabels,
                     limits = range(xValues),
                     expand = c(0,0)) +
  ggtitle("(30 samples of 30 observations each)") +
  theme(panel.background = element_blank(),
        panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        axis.ticks.x = element_line(colour = "black"),
        axis.ticks.y = element_line(colour = "black"),
        axis.text.x = element_text(colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.title = element_text(lineheight=.8, size = 11)
  )

base_sample_means

```


```{r 3010PopSam, fig.cap='Large sample sizes.', out.width='100%', fig.align='center', echo=FALSE}
knitr::include_graphics("figures/SamplingOriginal.png")
```

Taking 30 random samples from a population with known parameters. _top_: The population distribution. We generated a population with a known mean of $200.00$cm and a standard deviation of $6.00$cm. _middle_: The distribution of 30 sample means taken from this population, each containing 10 observations. _bottom_: The distribution of 30 sample means taken from this population, each containing 30 observations. In both the middle ($n=10$) and bottom ($n=30$) plots, the mean of the sample means is centered on the original population mean. A higher $n$ results in a narrower distribution of sample means (bottom) compared to smaller $n$ values (middle).

Each time we draw a sample we observe a different $\bar{x}$! E.g. a sample that _by chance_ contains the tallest Martians will probably provide a much higher $\bar{x}$ than a sample that _by chance_ contains the shortest Martians. What does this mean? To use a bit of theoretical Stats jargon, you just saw that.

*The sample mean is a random variable that has a distribution!*

So when you want to use the sample mean $\bar{X}$ to _infer_ the unknown population mean $\mu$ it's actually the properties of this random variable that we are interested in. Although it may sound strange at first, this is actually _the_ intellectual quantum leap you'll have to manage when you want to understand inferential statistics. The rest will fall into place from this! As this is such an important point, so let's paraphrase what we mean here:

> Just as height, $X$, is a random variable that takes different values $x_1$, $x_2$, $\ldots$, $x_n$ for $n$ observations in your single sample, the sample mean, $\bar{X}$, is a random variable for which you can observe different values $\bar{x}_1$, $\bar{x}_2$, $\ldots$ once the sample of Martians is drawn from the population.^[You may be wondering about the meaning behind upper and lower-case letters. The random variable $\bar{X}$ is called an _estimator_ whereas the individual values it takes once your sample is observed are called the _estimates_. The estimates are fixed values and are denoted with small letters whereas random variables are commonly denoted with capital letters. This is important if you write about statistics, but we won't get hung up on this here.]

With these new insights under our belt, we are now equipped to start talking about uncertainty! Let's tackle the three questions we presented in section \@ref(EstimQuest).

## Question 1 -- Is there systematic error?

Although in reality we usually only have a single sample to draw conclusions from, we just saw that our $\bar{x}$ is just _one of many_ $\bar{x}$ values we could have obtained. Nonetheless, we're still going to use this as an estimate of the underlying population mean $\mu$. From the video and Figure \@ref(fig:3010PopSam), we also learned that our single $\bar{x}$ is drawn from a distribution of $\bar{X}$, so in order to answer our three questions in \@ref(EstimQuest), let's describe the shape of the distribution.

```{exercise, name="Describing the distribution of sample means."}
After watching the video, and studying Figure \@ref(fig:3010PopSam), how would you describe the distribution of the sample means?
```

The distribution of $\bar{X}$ is centered around the population mean $\mu$, so although the individual values $\bar{x}_1$, $\bar{x}_2$, $\ldots$ are usually not exactly equal to $\mu$, the results are fine on average. In other words, the sample mean $\bar{X}$ is an _unbiased estimator_ for the underlying population mean which can be written as

$$\mu_{\bar{X}} = \mu,$$

where $\mu_{\bar{X}}$ denotes the mean of the distribution of the sample mean $\bar{X}$. This is actually a very desirable property because it means that we do not make a systematic error when we use the mean of our particular sample as an estimate of $\mu$.

## Question 2 -- The variance

It's very good to know that there's no systematic error in using $\bar{X}$, but we have seen that the particular values we get from different samples vary quite a bit. It would therefore be good to quantify this variation since it determines the precision of the sample mean as an estimator for $\mu$.


**Exercise**
Which factors do you think influence the variance of the $\bar{X}$?


```{r eval = FALSE, echo=FALSE}


set.seed(20140309)
df <- data.frame(obs = rnorm(500000, H.Means[1], H.SDs[1]*2))

m <- seq(-4,4,1)
xValues <- c(H.Means[1]+(m*H.SDs[1]))
# xLabels <- paste(xValues)
sign <- c(rep("-",4), "", rep("+", 4))
m2 <- c(2,0,1,0,0,0,1,0,2)

xLabels <- paste0(xValues,
                 "\n$\\mu$",
                 sign,
                 m2,
                 "$\\sigma$")

xLabels[5] <- paste0(xValues[5],
                 "\n$\\mu$")

xLabels[c(2,4,6,8)] <- paste0(xValues[c(2,4,6,8)])





# xValues <- c(H.Means[1]+(m*H.SDs[1]))
# xLabels <- paste(xValues)

# xLabels <- paste(xValues,
#                  "\\mu",
#                  m,
#                  "\\sigma")

basePlot <-
  ggplot(df, aes ( x = obs )) +
  geom_histogram(aes(y = ..density..), binwidth = 2, fill = MidYellow) +
  scale_y_continuous(name = "Frequency", expand = c(0,0)) +
  scale_x_continuous("Height at site I (cm)",
                     breaks = xValues,
                     labels = xLabels,
                     limits = range(xValues),
                     expand = c(0,0)) +
  ggtitle("Population Distribution\nN(200,144)") +
  theme(panel.background = element_blank(),
        panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        axis.ticks.x = element_line(colour = "black"),
        axis.ticks.y = element_line(colour = "black"),
        axis.text.x = element_blank(),
        axis.title.x = element_blank(),
        axis.text.y = element_text(colour = "black"),
        plot.title = element_text(lineheight=.8, size = 11)
  )

# basePlot

basePlot +
  stat_function(fun = dnorm, args = list(mean = mean(df$obs), sd = sd(df$obs)), col = DarkBlue) +
  geom_vline(xintercept = mean(df$obs), col = DarkBlue, size = 1)  +
  geom_vline(xintercept = mean(df$obs) + sd(df$obs), col = DarkBlue, size = 0.5, linetype = 2)  +
  geom_vline(xintercept = mean(df$obs) - sd(df$obs), col = DarkBlue, size = 0.5, linetype = 2)

#######################################
#### Sampling Distribution ############
#######################################

# randomness <- round(rnorm(30, 2346434, 232454))
tina <- data.frame(x = df$obs)

# Take samples using default seed by setting use.seed = F and iter to a number of iterations.
#tina <- make_rando(sample.size = 10, use.seed = FALSE, iter =  5)
tina <- make_rando(sample.size = 10, use.seed = TRUE, iter =  30)

# Calculate the mean for each sample group:
#tina.mean <- apply(tina[2:ncol(tina)], 2, function(x) {mean(tina$x[x == TRUE])})
tina.mean <- apply(tina[2:ncol(tina)], 2, function(x) {mean(tina$x[x == MidRed])})
tina.mean <- data.frame(mean = tina.mean)
tina.mean$mean <- round(tina.mean$mean, 1)

tina.cumul <- rbind(tina.cumul,
                    data.frame(value = tina.mean$mean,
                         pop = "Large Population Variance",
                         n = 10))

# After making video, show the distribution of the sample means:
base_sample_means <- ggplot(tina.mean, aes (x = mean)) +
  geom_histogram(aes(y = ..density..), binwidth = 1, fill = MidRed) +
  scale_y_continuous("Frequency",
                     expand = c(0,0)) +
  scale_x_continuous("Height at site I",
                     breaks = round(xValues,2),
                     labels = xLabels,
                     limits = range(xValues),
                     expand = c(0,0)) +
  ggtitle("Distribution of 30 sample means\n(10 observations each)") +
  theme(panel.background = element_blank(),
        panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        axis.ticks.x = element_line(colour = "black"),
        axis.ticks.y = element_line(colour = "black"),
        axis.text.x = element_blank(),
        axis.title.x = element_blank(),
        axis.text.y = element_text(colour = "black"),
        plot.title = element_text(lineheight=.8, size = 11)
  )

base_sample_means

tina <- make_rando(sample.size = 30, use.seed = TRUE, iter =  30)

# Calculate the mean for each sample group:
#tina.mean <- apply(tina[2:ncol(tina)], 2, function(x) {mean(tina$x[x == TRUE])})
tina.mean <- apply(tina[2:ncol(tina)], 2, function(x) {mean(tina$x[x == MidRed])})
tina.mean <- data.frame(mean = tina.mean)
tina.mean$mean <- round(tina.mean$mean, 1)

tina.cumul <- rbind(tina.cumul,
                    data.frame(value = tina.mean$mean,
                         pop = "Large Population Variance",
                         n = 30))


# After making video, show the distribution of the sample means:
base_sample_means <- ggplot(tina.mean, aes (x = mean)) +
  geom_histogram(aes(y = ..density..), binwidth = 1, fill = MidRed) +
  scale_y_continuous("Frequency",
                     expand = c(0,0)) +
  scale_x_continuous("Height at site I (cm)",
                     breaks = round(xValues,2),
                     labels = xLabels,
                     limits = range(xValues),
                     expand = c(0,0)) +
  ggtitle("(30 observations each)") +
  theme(panel.background = element_blank(),
        panel.grid.major = element_blank(),
        panel.grid.minor = element_blank(),
        axis.ticks.x = element_line(colour = "black"),
        axis.ticks.y = element_line(colour = "black"),
        axis.text.x = element_text(colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.title = element_text(lineheight=.8, size = 11)
  )

base_sample_means

```

```{r 3010PopSamLarge, fig.cap='Large sample sizes.', out.width='100%', fig.align='center', echo=FALSE}
knitr::include_graphics("figures/SamplingLargeSam.png")
```
Taking 30 random samples from a population with known parameters. _top_: The population distribution. We generated a population with a known mean of $200.00$cm and a standard deviation of $12.00$cm, twice as that used in Figure \@ref(fig:3010PopSam). _middle_: The distribution of 30 sample means taken from this population, each containing 10 observations. _bottom_: The distribution of 30 sample means taken from this population, each containing 30 observations. As in Figure \@ref(fig:3010PopSam), the sample means $\bar{X}$are centered on the population mean $\mu$ and a larger $n$ results in a narrower $\sigma^2_{\bar{X}}$. However in addition to what we have already observed, here we notice that the larger population variance $\sigma$ causes $\sigma^2_{\bar{X}}$ to also be larger than in Figure \@ref(fig:3010PopSam).

**Video** How does the _sample size_ influence the distribution of the sample mean?


**Video** How does the _population variance_ influence the distribution of the sample mean?

From this, we can see that there are two factors, the sample size, $n$, and the population variance, $\sigma^2$, that influence the variance of $\bar{X}$ that we denote as $\sigma^2_{\bar{X}}$. This can be quantified nicely as
$$ \sigma^2_{\bar{X}} = \frac{\sigma^2}{n},$$
where $\sigma^2$ denotes the variance of the population, and $n$ denotes the sample size. This formula tells us two things

1. As the variance of the population increases, so does the numerator and thus $\sigma^2_{\bar{X}}$ gets larger. 
2. As the ^[The precision of the sample mean increases with the sample size!] sample size increases, so does the denominator and thus $\sigma^2_{\bar{X}}$ gets smaller.



What does this mean for us? The fact that $\sigma^2_{\bar{X}}$ is influenced by the sample size is good news for us, because while we can't do anything about the variance in the population itself, we _do_ usually have control over the sample size! Thus, if we want our sample mean to be a more precise estimate of the population mean we can take a larger sample. If the population variance is large, we'll always have a difficult job inferring something about the population from our sample, but even in this case, a larger sample size will help.

When you think about it, this actually makes a lot of sense. The bigger the sample size, the more information about the population is being compressed into the mean. Just like in any other context, the more information you have at hand, the better the prediction will be. This is summarised in Figure \@ref(fig:dartvariance).

```{r dartvariance, fig.cap='Comparing different types of error bars.', out.width='100%', fig.align='center', echo=FALSE}
knitr::include_graphics("figures/PrecisionGrid.png")
```



The precision of sample means, i.e. the precision with which we can estimate $\mu$, is inversely proportional to the underlying population variance and proportional to the sample size. Regardless of the underlying population variance, which we cannot control, larger sample sizes are always preferred.

```{r SamplingSimulation1, fig.cap='Sampling distributions', echo=FALSE, fig.width = large_width, fig.height = large_width*1.5, eval = FALSE, echo = FALSE}
simCLT(x = 500)
```

## Question 3 -- What's the best estimator of $\mu$?

By now we have seen that there are two requirements for a good estimator: 

1. Unbiased, and 
2. Small variance (i.e. high precision). 

The _really_ good estimators combine both of these proporties,^[When choosing a descriptive statistic to infer something about your population, consider if these requirements have been fulfilled. Many measurements have become standard because they do!] and the estimator, that combines these two requirements in the optimal way is called the _minimum-variance unbiased_ estimator! As you might have guessed, $\bar X$ is the minimum-variance unbiased estimator for $\mu$.
`
Let's see what this means. Although it may seem obvious to use $\bar{X}$ as the estimator for $\mu$, we're not restricted to that. We could have chosen _any_ measurement of the sample to act as an estimator, e.g. we could have chosen the first value in the sample -- regardless of the sample size.^[We could have also chosen to take the median, or possibly the mid-point between the minimum and maximum values, the list goes on and on.] Or, if we were really stubborn, we could even have used a fixed value, e.g. 203, regardless of the actual sample. 

Figure \@ref(fig:ThreeScenarios) shows the performance of these three estimators in the context of a bull's eye target for shooting practice. The entire target corresponds to the values in our population and the bull's eye is the true population mean $\mu$. The goal is to hit the bull's eye when we take a shot, i.e. draw a sample and calculate the estimate.

```{r ThreeScenarios, fig.cap='Comparing different types of error bars.', out.width='100%', fig.align='center', echo=FALSE}
knitr::include_graphics("figures/ThreeScenarios.png")
```
The bull's eye analogy to understand the miniumum-variance unbiased estimator. Of all the unbiased estimators that we could have chosen, it turns out that $\bar{X}$ has the smallest variance.

<dl>
  <dt>Scenario 1</dt>
  <dd>We fix the estimator to a specific value. Here, our variance will be zero, because no matter what the sample is, our estimator will always be the same. On the one hand, this is fantastic! We like a tiny variance. However, our estimator is likely to be very biased. It doesn't contain any information from the sample!</dd>
  
  <dt>Scenario 2</dt>
  <dd>We take the first observation of the sample to be the estimator. This has a higher variance than the fixed-number, but despite this we prefer this estimator over the fixed value because it is _unbiased_. As we learned this means that if we take many samples, we get it right on average.</dd>

  <dt>Scenario 3</dt>
  <dd>We take $\bar{X}$ as the estimator. Here, we can observe that the variance is lower then in the second scenario, _and_ it is still an unbiased estimator. This is the ideal situation, we've found a balance between have very small variance and being unbiases.</dd>
</dl>

