Sampling Distributions
A sampling distribution is the distribution of a statistic (like the sample mean or sample proportion) obtained from many samples of the same size drawn from the same population. It is the foundation of inferential statistics.
This guide covers key definitions, the distribution of sample means and proportions, the Central Limit Theorem, standard error, worked examples, memory aids, and a practice quiz.
1Introduction
In statistics, we often want to know something about a large group (a population), but it is usually impossible or impractical to collect data from every single member. Instead, we take a smaller, representative group (a sample) and use information from that sample to make inferences about the population.
A sampling distribution helps us understand how much sample statistics vary from sample to sample and provides the foundation for inferential statistics, including confidence intervals and hypothesis testing.
Imagine you want to know the average height of all 10,000 students at a university. You cannot measure everyone, so you randomly select 50 students and calculate their mean height. If you repeated this process hundreds of times -- each time picking a new random group of 50 -- you would get hundreds of different sample means. The distribution of all those sample means is the sampling distribution.
Why It Matters
Quantifying Uncertainty
Sampling distributions allow us to quantify the uncertainty in our estimates and determine how reliable our sample statistics are.
Confidence Intervals
Understanding how sample statistics vary lets us construct intervals that capture the true population parameter with a known level of confidence.
Hypothesis Testing
We compare observed sample statistics to what we would expect under a null hypothesis using the sampling distribution.
Probability Statements
Without understanding sampling distributions, we cannot confidently make probability statements about population parameters based on sample data.
2Key Definitions
Population
The entire group about which we want to draw conclusions. Its characteristics are called parameters (e.g., μ, σ, p).
Sample
A subset of the population selected for study. Its characteristics are called statistics (e.g., x̅, s, p̂).
Sampling Distribution
The probability distribution of a statistic obtained from all possible samples of a specific size from a given population.
Central Limit Theorem (CLT)
For sufficiently large n, the sampling distribution of the sample mean will be approximately normal, regardless of the population distribution shape.
Standard Error
The standard deviation of a sampling distribution. For means: σ/√n. For proportions: √[p(1-p)/n]. It measures how much sample statistics typically vary.
Law of Large Numbers (LLN)
As n increases, the sample mean x̅ gets closer to the population mean μ. It guarantees convergence but says nothing about the shape of the distribution.
3Distribution of Sample Means (x̅)
When we take repeated samples of size n from a population and calculate the mean for each sample, these sample means form their own distribution called the sampling distribution of the sample mean.
Shape
Normal if population is normal (any n), or approximately normal if n ≥ 30 (by CLT).
Center (Mean)
μₓ̅ = μ
The mean of all sample means equals the population mean.
Spread (Standard Error)
σₓ̅ = σ / √n
Larger n means smaller standard error and more precision.
Conditions for Use
1. Randomness
The samples must be randomly selected from the population.
2. Independence (10% Condition)
When sampling without replacement, n ≤ 0.10N (sample size is no more than 10% of population).
3. Normality (CLT Condition)
The population distribution is normal, OR n ≥ 30.
If σ is unknown, we use the sample standard deviation s as an estimate, giving sₓ̅ = s / √n. In this case, especially for small n, we use a t-distribution instead of a z-distribution.
4Distribution of Sample Proportions (p̂)
When we take repeated samples of size n and calculate the proportion of "successes" for each sample, these sample proportions form the sampling distribution of the sample proportion.
Shape
Approximately normal if np ≥ 10 and n(1-p) ≥ 10.
Center (Mean)
μₚ̂ = p
The mean of all sample proportions equals the population proportion.
Spread (Standard Error)
σₚ̂ = √[p(1-p)/n]
Larger n means less variability in sample proportions.
Conditions for Use
1. Randomness
The samples must be randomly selected.
2. Independence (10% Condition)
When sampling without replacement, n ≤ 0.10N.
3. Success/Failure Condition
np ≥ 10 and n(1-p) ≥ 10. This ensures the distribution is sufficiently symmetric to approximate normality.
Notice the difference: for means, the normality condition is n ≥ 30. For proportions, the condition is np ≥ 10 AND n(1-p) ≥ 10. These are different conditions and should not be confused.
5Central Limit Theorem (CLT)
The Central Limit Theorem is arguably the most important theorem in statistics.
The Central Limit Theorem
Regardless of the shape of the original population distribution (skewed, uniform, bimodal, etc.), the sampling distribution of the sample mean (or sum, or proportion) will become approximately normal as the sample size n increases.
When Does It Apply?
For Sample Means
Generally when n ≥ 30. If the population is already normal, any sample size works.
For Sample Proportions
When np ≥ 10 and n(1-p) ≥ 10 (the success/failure condition).
The CLT allows us to use normal distribution theory (Z-scores, probability calculations) to make inferences about population parameters, even when we do not know the original population distribution, as long as our sample size is large enough. This is crucial because real-world population distributions are rarely perfectly normal.
Effect of Sample Size on Standard Error
Small Sample (e.g., n = 10)
Wider sampling distribution, larger standard error, less precise estimates.
Large Sample (e.g., n = 100)
Narrower sampling distribution, smaller standard error, more precise estimates.
6Worked Examples
Example 1: Sampling Distribution of Sample Means
A certain brand of light bulbs has a mean lifetime of μ = 800 hours with a standard deviation of σ = 60 hours. A quality control manager takes a random sample of n = 36 light bulbs. What is the probability that the sample mean lifetime is less than 785 hours?
Step 1: Check Conditions
Random sample: Given
Independence: Population >> 36 × 10 = 360
Normality: n = 36 ≥ 30 (CLT applies)
Step 2: Calculate Mean and Standard Error
μₓ̅ = μ = 800 hours
σₓ̅ = σ / √n = 60 / √36 = 60 / 6 = 10 hours
Step 3: Calculate Z-score
Z = (x̅ - μₓ̅) / σₓ̅ = (785 - 800) / 10 = -15 / 10 = -1.5
Step 4: Find Probability
P(Z < -1.5) = 0.0668
P(x̅ < 785) = 0.0668 (6.68%)
Example 2: Sampling Distribution of Sample Proportions
Suppose 60% of adults in a large city support a new public park initiative. A random sample of n = 150 adults is surveyed. What is the probability that the sample proportion is between 0.55 and 0.65?
Step 1: Check Conditions
Random sample: Given
Independence: Population >> 150 × 10 = 1500
Success/Failure: np = 150 × 0.60 = 90 ≥ 10
n(1-p) = 150 × 0.40 = 60 ≥ 10
Step 2: Calculate Mean and Standard Error
μₚ̂ = p = 0.60
σₚ̂ = √[p(1-p)/n] = √[0.60 × 0.40 / 150]
= √[0.24 / 150] = √0.0016 = 0.04
Step 3: Calculate Z-scores
For p̂ = 0.55: Z₁ = (0.55 - 0.60) / 0.04 = -1.25
For p̂ = 0.65: Z₂ = (0.65 - 0.60) / 0.04 = 1.25
Step 4: Find Probability
P(Z < 1.25) - P(Z < -1.25) = 0.8944 - 0.1056 = 0.7888
P(0.55 < p̂ < 0.65) = 0.7888 (78.88%)
7Key Formulas
For Sample Means
Z-Score for Sample Mean
Z = (x̅ - μ) / (σ / √n)
Standard Error of Mean
σₓ̅ = σ / √n
For Sample Proportions
Z-Score for Sample Proportion
Z = (p̂ - p) / √[p(1-p)/n]
Standard Error of Proportion
σₚ̂ = √[p(1-p)/n]
Means of Sampling Distributions
Mean of Sample Means
μₓ̅ = μ
Mean of Sample Proportions
μₚ̂ = p
8Memory Aids
"CLT: C-L-T, Normal, Normal, Normal!"
The Central Limit Theorem makes sampling distributions normal if n is large enough. Just remember CLT = normality for sampling distributions.
"Standard Error is the STD of the STATISTIC!"
This helps you remember that the standard error is the standard deviation of the sampling distribution, not of individual data points.
"Square Root N in the Denominator: Bigger N, Smaller Error!"
A larger sample size means more precision, less variability, and a narrower sampling distribution. The √n in the denominator guarantees this.
"Means use σ, Proportions use p(1-p)"
This helps distinguish the standard error formulas. For means, you use the population standard deviation. For proportions, you use p(1-p).
"Conditions, Conditions, Conditions!"
Always remember to check: Random, Independence (10% rule), and Normality (CLT rule or Success/Failure rule). Forgetting conditions is one of the most common exam mistakes.
9Common Mistakes
Confusing σ with Standard Error
The population standard deviation (σ) measures variability of individual data points. The standard error (σ/√n or √[p(1-p)/n]) measures variability of sample statistics. The standard error always has √n in the denominator.
Forgetting to Check Conditions
Failing to verify randomness, independence, and normality (CLT for means, success/failure for proportions) can lead to incorrect conclusions, especially with small sample sizes.
Using the Wrong Z-score Formula
Using the Z-score for an individual observation Z = (x - μ) / σ instead of the Z-score for a sample statistic Z = (statistic - mean of sampling dist) / standard error. Always divide by the standard error, not σ.
Misinterpreting "Sampling Distribution"
Thinking it is the distribution of a single sample, rather than the distribution of a statistic calculated from many samples. A sampling distribution is a theoretical concept about repeated sampling.
Not Understanding the Role of Sample Size
Not realizing that increasing n decreases the standard error and makes the sampling distribution narrower and more normal (due to the CLT).
Confusing LLN with CLT
The Law of Large Numbers says x̅ approaches μ as n increases. The CLT says the shape of the distribution of x̅ becomes normal as n increases. They are related but distinct concepts.
Incorrect Success/Failure Condition
For proportions, the condition is np ≥ 10 and n(1-p) ≥ 10, not just n ≥ 30. These are different from the CLT condition for means.
Quick Revision Summary
- ✓A sampling distribution is the distribution of a statistic from all possible samples of a given size.
- ✓The mean of sample means equals the population mean: μₓ̅ = μ.
- ✓The standard error of the mean is σ/√n -- it decreases as sample size increases.
- ✓The mean of sample proportions equals the population proportion: μₚ̂ = p.
- ✓The standard error of the proportion is √[p(1-p)/n].
- ✓The Central Limit Theorem says sampling distributions become approximately normal for large n, regardless of the population shape.
- ✓For means: normality requires n ≥ 30 (or normal population). For proportions: np ≥ 10 and n(1-p) ≥ 10.
- ✓Always check three conditions: Randomness, Independence (10% rule), and Normality.
- ✓Larger sample sizes produce narrower sampling distributions and more precise estimates.
Frequently Asked Questions
- Why can't I just use the population standard deviation (σ) instead of the standard error (σ/√n)?
- The population standard deviation σ measures the variability of individual data points. The standard error measures the variability of sample means (or other statistics). Sample means are less variable than individual data points because extreme values tend to average out across a sample. Dividing by √n accounts for this reduced variability.
- How large is "sufficiently large" for the Central Limit Theorem?
- For sample means, n ≥ 30 is a widely accepted rule of thumb. For sample proportions, the conditions np ≥ 10 and n(1-p) ≥ 10 are used. If the population is already normal, the CLT is not strictly needed for means; the sampling distribution is normal for any n.
- What if the population standard deviation (σ) is unknown?
- If σ is unknown, we use the sample standard deviation (s) as an estimate. When doing this for sample means, especially with small sample sizes (n < 30), we typically use a t-distribution instead of a z-distribution.
- What is the difference between a histogram of a sample and a histogram of a sampling distribution?
- A histogram of a sample shows the distribution of the individual data points collected in one sample. A histogram of a sampling distribution shows the distribution of a statistic (like the mean) calculated from many different samples of the same size.
- Why is the 10% condition important?
- The 10% condition (n ≤ 0.10N) ensures that observations within the sample are approximately independent when sampling without replacement. If you sample a significant portion of the population, removing an item changes the probabilities for subsequent selections, violating independence assumptions crucial for the formulas.
Practice Quiz
Test your knowledge — select the correct answer for each question.
1.Which of the following best describes a sampling distribution?
2.The mean of the sampling distribution of the sample mean is always equal to:
3.The standard error of the mean measures:
4.According to the Central Limit Theorem, if the sample size is sufficiently large, the sampling distribution of the sample mean will be approximately normal:
5.If the population standard deviation is 50 and the sample size is 100, what is the standard error of the mean?
6.For the sampling distribution of sample proportions to be approximately normal, which conditions must be met?
7.As the sample size (n) increases, the standard error of the mean:
8.A population has a mean of 100 and a standard deviation of 20. If a sample of size 64 is taken, what is the mean of the sampling distribution of the sample means?
9.Which theorem states that as the sample size increases, the sample mean will get closer to the population mean?
10.You are studying the proportion of students who prefer online learning. The true population proportion is p = 0.40. If you take a sample of n = 200 students, what is the standard error of the sample proportion?
Final Study Advice
- 1.Always check all three conditions (Random, Independence, Normality) before applying sampling distribution formulas -- this is a common source of lost points on exams.
- 2.Make sure you use the correct standard error formula: σ/√n for means, √[p(1-p)/n] for proportions.
- 3.Remember the distinction: n ≥ 30 for means (CLT), but np ≥ 10 and n(1-p) ≥ 10 for proportions (Success/Failure).
- 4.Practice distinguishing between questions about individual observations and questions about sample statistics -- they use different Z-score formulas.
- 5.Visualize how the sampling distribution gets narrower as n increases to build intuition for why larger samples give better estimates.