In previous article we discussed about variance and standard deviation, assuming we had a population of data, even though we referred to it as a sample. However, in practice, the formulas used to calculate variance and standard deviation will differ slightly depending on whether the dataset is a sample drawn from a larger population or if it represents the entire population itself.
Key Takeaway
- We compute population variance by summing the squared deviations of each observation from the population mean, adding them together, and then dividing by the sample size n.
- However, in case of sample variance we divide by n-1, where n is the number of observations in the sample.
- we divide by n−1 because of degrees of freedom, degree of freedom provides an unbiased estimate of the population variance when working with a sample.
- Degrees of freedom refer to the number of independent pieces of information or values that can vary freely in a dataset or calculation.
- In essence, n−1 accounts for the adjustment needed to ensure that our sample statistic, such as variance, provides an unbiased estimate of the population parameter, considering the limitations imposed by using sample data rather than the entire population dataset.
Variance of a Population
So, when discussing the variance for a population, we use the Greek letter sigma squared. This notation is a convention used for population parameters, where Greek letters are typically employed. To calculate the population variance, we subtract the population mean from each observation, square the result, sum all these squared deviations, and finally dividing by N.
Variance of a Sample
When calculating the variance for a sample, we follow a slightly different approach and use Latin or Roman letters to distinguish it from population variance. This distinction is important in statistics to indicate whether we are working with a sample or the entire population dataset.
Conceptually, the calculation for sample variance, is similar to that for population variance. We compute it by summing the squared deviations of each observation from the sample mean, adding them together, and then dividing by the sample size n, where n is the number of observations in the sample. However, a key difference is that instead of dividing by n (the total sample size), we divide by n−1. This adjustment accounts for the fact that using n−1 degrees of freedom provides an unbiased estimate of the population variance when working with a sample.
Why we divide by n-1?
The reason we divide by n−1 instead of n when calculating sample statistics like variance relates to the concept of degrees of freedom. Degrees of freedom refer to the number of independent pieces of information or values that can vary freely in a dataset or calculation.
When we gather a sample from a population, we use sample statistics to estimate population parameters. For instance, when calculating sample variance, we rely on the sample mean as part of the calculation. The degrees of freedom in this context, n−1, represents the number of independent observations that can vary freely after using one sample statistic (the mean) to estimate variance.
In essence, n−1 accounts for the adjustment needed to ensure that our sample statistic, such as variance, provides an unbiased estimate of the population parameter, considering the limitations imposed by using sample data rather than the entire population dataset.
The sample mean, is computed by summing all the observations in our sample and dividing by the sample size n. No other sample statistics are used in this calculation. The degrees of freedom in the context of sample variance is n, which is the sample size minus zero adjustments. This reflects the number of independent observations available in our sample.
When we calculate the sample variance, we use the sample mean X bar as part of the calculation. Because we rely on one sample statistic (the mean) to estimate another (the variance), the degrees of freedom become n−1. This adjustment ensures that the sample variance provides an unbiased estimate of the population variance, considering the constraints of using a sample rather than the entire population dataset.
Degrees of freedom
Degrees of freedom has to do with constraints. So, it’s really constraints on the possible values of a set of observations. Suppose we have a small data set that just consists of three numbers and we abstractly, we call these x1, x2, x3.
If you’re told that we have no constraints, then each number could be anything than first number could be one, the second number could be five, the next number could be a million. So, we have no constraints, if we have no constraints these values are free to vary, there are three independent pieces of information.
Imagine we have a dataset where the sample mean is fixed at three. This means there’s a constraint because the average of all values in the dataset must be exactly three. Let’s consider a small dataset with three numbers: suppose the first number is 1 and the second number is 3. Given that the sample mean is three, the third number must be 5. This is because the sum of 1, 3, and 5 divided by 3 equals the sample mean of three.
The issue here is that knowing the sample mean restricts the possible values of the remaining observations. In statistical terms, this constraint on the variability is reflected in the degrees of freedom associated with the sample variance. Degrees of freedom are reduced by one (n – 1) because the sample mean acts as a constraint on how freely the values can vary.
