7.1 Significance test for the difference of two means

Achievable AP Statistics

7. Quantitative data

Our AP Statistics course is currently in development and is a work-in-progress.

Significance test for the difference of two means

6 min read

Font

Discuss

Feedback

Significance test for the difference of two means

A significance test for the difference of two means helps you decide whether an observed gap between two sample means reflects a real difference between the two population means, or whether it could reasonably be explained by natural sampling variation. This kind of test is common when comparing two groups — such as a treatment group versus a control group, two schools, or two product designs — and you want evidence about whether the underlying averages truly differ.

To run a significance test for the difference between two means, check the following:

You have two independent random samples (ideally simple random samples).
Each population is roughly normal, or both sample sizes are at least $30$ so the Central Limit Theorem applies.
If you’re sampling without replacement, each sample is less than $10%$ of its population.

If all three conditions are met, it’s reasonable to conduct a significance test for the difference between two means.

Hypotheses for comparing two means

Definitions

Null hypothesis ( $H_{0}$ ): Says there is no difference between the population means.

$H_{0} or equivalently, H_{0} : μ_{1} = μ_{2} : μ_{1} - μ_{2} = 0$

Alternative hypothesis ( $H_{a}$ ): Depends on the question being asked.

$H_{a} H_{a} H_{a} : μ_{1} - μ_{2} < 0 (left-tailed: μ_{1} < μ_{2}) : μ_{1} - μ_{2} > 0 (right-tailed: μ_{1} > μ_{2}) : μ_{1} - μ_{2} \neq = 0 (two-sided)$

Practice problem

Example:

A tech company’s human resources director claims that engineers with assigned mentors and engineers without mentors spend the same amount of time working at the company before receiving their first promotion, so the company should no longer have mentors for the engineers. One employee thinks that engineers without mentors spend more time at the company before getting their first promotion.

A statistician takes two independent random samples of engineers and records the time (in months) before the first promotion:

Group 1 (no mentor): $25$ , $28$ , $31$ , $26$ , $24$

Group 2 (mentor): $18$ , $22$ , $24$ , $20$ , $19$ , $21$

Let $μ_{1}$ be the population mean for the no-mentor group, and let $μ_{2}$ be the population mean for the mentor group.

Part a.

Is a significance test possible? What assumptions are we making?

To run the test, all three conditions for inference must be checked:

Independent random samples
Normality (or large samples)
10% condition (if sampling without replacement)

Solution:

(spoiler)

Independent random samples: The problem states the samples are independent and random, so this condition is met (no extra assumption needed).
Normality / large samples: The sample sizes are not at least $30$ (they are $5$ and $6$ ), so we must assume both populations are approximately normally distributed. If that assumption is false, the test results may not be reliable.
10% condition (when sampling without replacement): The problem doesn’t give the total number of engineers in each group, so we must assume each sample is less than $10%$ of its population. That means assuming there are at least $50$ engineers without mentors and at least $60$ engineers with mentors. If either population is smaller than that, the test conclusions may not be reliable.

Part b.

Restate the parameters and the hypotheses.

The employee’s claim is that the no-mentor group waits longer, so we use a right-tailed test.

Solution:

(spoiler)

$μ_{1}$ : the mean time to first promotion (in months) for engineers without a mentor
$μ_{2}$ : the mean time to first promotion (in months) for engineers with a mentor

$H_{0} H_{a} : μ_{1} = μ_{2} : μ_{1} > μ_{2}$

Part c.

Complete the test and interpret the P-value.

First, compute the sample means:

$\overset{x}{ˉ}_{1} = \frac{134}{5} = 26.8 months$

$\overset{x}{ˉ}_{2} = \frac{124}{6} \approx 20.67 months$

The difference of sample means:

$\overset{x}{ˉ}_{1} - \overset{x}{ˉ}_{2} \approx 6.13$

Next, find the sample standard deviations using a TI-84:

Enter the data into lists.
Press STAT, highlight CALC, choose 1: 1-Var Stats.
Type L1 (2nd function 1) and press ENTER.

This gives:

$s_{1} s_{2} \approx 2.77 \approx 2.16$

Compute the standard error of $\overset{x}{ˉ}_{1} - \overset{x}{ˉ}_{2}$ :

$SE = \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}} = \frac{7.70}{5} + \frac{4.67}{6} = 1.54 + 0.778 = 2.318 \approx 1.522$

Calculate the test statistic:

$t = \frac{( x ˉ _{1} - x ˉ _{2} ) - 0}{SE} = \frac{26.8 - 20.667}{1.522} = \frac{6.133}{1.522} \approx 4.03$

Compute the degrees of freedom using Welch’s approximation:

$df = \frac{( \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}} ) ^{2}}{\frac{( \frac{s _{1}^{2}}{n _{1}} ) ^{2}}{n _{1} - 1} + \frac{( \frac{s _{2}^{2}}{n _{2}} ) ^{2}}{n _{2} - 1}} = \frac{( \frac{7.70}{5} + \frac{4.67}{6} ) ^{2}}{\frac{( 1.54 ) ^{2}}{4} + \frac{( 0.778 ) ^{2}}{5}} = \frac{( 2.318 ) ^{2}}{\frac{2.3716}{4} + \frac{0.605}{5}} = \frac{5.372}{0.7139} \approx 7.53$

To find the p-value on a TI-84, use 2nd → VARS → 6: tcdf(lower, upper, degrees of freedom).

Because this is a right-tailed test:

$tcdf (4.03, 1E99, 7.53) \approx 0.0022$

Solution:

(spoiler)

The p-value is approximately $0.0022$ .

Interpretation: If the population means are actually the same, then there is about a $0.22%$ chance of getting a sample difference as large as or larger than the one observed, just due to random sampling variation.

Part d.

Give a conclusion in context at $α = 0.05$ .

Solution:

(spoiler)

Since

$0.0022 < 0.05$

we reject $H_{0}$ . There is strong evidence that engineers without mentors receive their first promotion later on average than engineers with mentors.

Part e.

Given your conclusion, what type of error (Type I or II) might have been made and what would the consequences of this error be?

Solution:

(spoiler)

Because we rejected $H_{0}$ , the possible mistake is a Type I error.

A Type I error here would mean concluding that engineers without mentors take longer to get promoted (on average) even though the population means are actually the same.

A likely consequence is that the company would spend money to hire or maintain mentors to speed up promotions, even though mentoring doesn’t actually change the average time to promotion. That money could have been spent elsewhere.

Conditions for two-sample t-test

Two independent random samples
Each population roughly normal, or both sample sizes ≥ 30 (Central Limit Theorem)
Sample size < 10% of population (if sampling without replacement)

Hypotheses for comparing two means

Null hypothesis: $H_{0} : μ_{1} = μ_{2}$ or $H_{0} : μ_{1} - μ_{2} = 0$
Alternative hypothesis:
- Right-tailed: $H_{a} : μ_{1} > μ_{2}$
- Left-tailed: $H_{a} : μ_{1} < μ_{2}$
- Two-sided: $H_{a} : μ_{1} \neq = μ_{2}$

Test statistic and standard error

Test statistic: $t = \frac{( x ˉ _{1} - x ˉ _{2} ) - 0}{SE}$
Standard error: $SE = \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}}$
Degrees of freedom: use Welch’s approximation

Interpreting the p-value

P-value: probability of observed (or more extreme) difference if $H_{0}$ is true
Small P-value (e.g., $< α$ ): reject $H_{0}$ , evidence for alternative

Types of errors

Type I error: reject $H_{0}$ when $H_{0}$ is true
- Consequence: act as if means differ when they do not
Type II error: fail to reject $H_{0}$ when $H_{a}$ is true

Summary

Two-sample t-test evaluates if sample mean difference reflects a true population mean difference
Requires independence, random sampling, and normality (or large samples)
Key formulas:
- Test statistic: $t = \frac{( x ˉ _{1} - x ˉ _{2} ) - 0}{SE}$
- Standard error: $SE = \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}}$

Next | 7.2 The t distribution

Significance test for the difference of two means

To run a significance test for the difference between two means, check the following:

You have two independent random samples (ideally simple random samples).
Each population is roughly normal, or both sample sizes are at least $30$ so the Central Limit Theorem applies.
If you’re sampling without replacement, each sample is less than $10%$ of its population.

If all three conditions are met, it’s reasonable to conduct a significance test for the difference between two means.

Hypotheses for comparing two means

Definitions

Null hypothesis ( $H_{0}$ ): Says there is no difference between the population means.

$H_{0} or equivalently, H_{0} : μ_{1} = μ_{2} : μ_{1} - μ_{2} = 0$

Alternative hypothesis ( $H_{a}$ ): Depends on the question being asked.

$H_{a} H_{a} H_{a} : μ_{1} - μ_{2} < 0 (left-tailed: μ_{1} < μ_{2}) : μ_{1} - μ_{2} > 0 (right-tailed: μ_{1} > μ_{2}) : μ_{1} - μ_{2} \neq = 0 (two-sided)$

Practice problem

Example:

A tech company’s human resources director claims that engineers with assigned mentors and engineers without mentors spend the same amount of time working at the company before receiving their first promotion, so the company should no longer have mentors for the engineers. One employee thinks that engineers without mentors spend more time at the company before getting their first promotion.

A statistician takes two independent random samples of engineers and records the time (in months) before the first promotion:

Group 1 (no mentor): $25$ , $28$ , $31$ , $26$ , $24$

Group 2 (mentor): $18$ , $22$ , $24$ , $20$ , $19$ , $21$

Let $μ_{1}$ be the population mean for the no-mentor group, and let $μ_{2}$ be the population mean for the mentor group.

Part a.

Is a significance test possible? What assumptions are we making?

To run the test, all three conditions for inference must be checked:

Independent random samples
Normality (or large samples)
10% condition (if sampling without replacement)

Solution:

(spoiler)

Independent random samples: The problem states the samples are independent and random, so this condition is met (no extra assumption needed).
Normality / large samples: The sample sizes are not at least $30$ (they are $5$ and $6$ ), so we must assume both populations are approximately normally distributed. If that assumption is false, the test results may not be reliable.
10% condition (when sampling without replacement): The problem doesn’t give the total number of engineers in each group, so we must assume each sample is less than $10%$ of its population. That means assuming there are at least $50$ engineers without mentors and at least $60$ engineers with mentors. If either population is smaller than that, the test conclusions may not be reliable.

Part b.

Restate the parameters and the hypotheses.

The employee’s claim is that the no-mentor group waits longer, so we use a right-tailed test.

Solution:

(spoiler)

$μ_{1}$ : the mean time to first promotion (in months) for engineers without a mentor
$μ_{2}$ : the mean time to first promotion (in months) for engineers with a mentor

$H_{0} H_{a} : μ_{1} = μ_{2} : μ_{1} > μ_{2}$

Part c.

Complete the test and interpret the P-value.

First, compute the sample means:

$\overset{x}{ˉ}_{1} = \frac{134}{5} = 26.8 months$

$\overset{x}{ˉ}_{2} = \frac{124}{6} \approx 20.67 months$

The difference of sample means:

$\overset{x}{ˉ}_{1} - \overset{x}{ˉ}_{2} \approx 6.13$

Next, find the sample standard deviations using a TI-84:

Enter the data into lists.
Press STAT, highlight CALC, choose 1: 1-Var Stats.
Type L1 (2nd function 1) and press ENTER.

This gives:

$s_{1} s_{2} \approx 2.77 \approx 2.16$

Compute the standard error of $\overset{x}{ˉ}_{1} - \overset{x}{ˉ}_{2}$ :

$SE = \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}} = \frac{7.70}{5} + \frac{4.67}{6} = 1.54 + 0.778 = 2.318 \approx 1.522$

Calculate the test statistic:

$t = \frac{( x ˉ _{1} - x ˉ _{2} ) - 0}{SE} = \frac{26.8 - 20.667}{1.522} = \frac{6.133}{1.522} \approx 4.03$

Compute the degrees of freedom using Welch’s approximation:

To find the p-value on a TI-84, use 2nd → VARS → 6: tcdf(lower, upper, degrees of freedom).

Because this is a right-tailed test:

$tcdf (4.03, 1E99, 7.53) \approx 0.0022$

Solution:

(spoiler)

The p-value is approximately $0.0022$ .

Part d.

Give a conclusion in context at $α = 0.05$ .

Solution:

(spoiler)

Since

$0.0022 < 0.05$

we reject $H_{0}$ . There is strong evidence that engineers without mentors receive their first promotion later on average than engineers with mentors.

Part e.

Given your conclusion, what type of error (Type I or II) might have been made and what would the consequences of this error be?

Solution:

(spoiler)

Because we rejected $H_{0}$ , the possible mistake is a Type I error.

A Type I error here would mean concluding that engineers without mentors take longer to get promoted (on average) even though the population means are actually the same.

Achievable AP Statistics

7. Quantitative data

Our AP Statistics course is currently in development and is a work-in-progress.

Significance test for the difference of two means

6 min read

Font

Discuss

Feedback

Significance test for the difference of two means

To run a significance test for the difference between two means, check the following:

You have two independent random samples (ideally simple random samples).
Each population is roughly normal, or both sample sizes are at least $30$ so the Central Limit Theorem applies.
If you’re sampling without replacement, each sample is less than $10%$ of its population.

If all three conditions are met, it’s reasonable to conduct a significance test for the difference between two means.

Hypotheses for comparing two means

Definitions

Null hypothesis ( $H_{0}$ ): Says there is no difference between the population means.

$H_{0} or equivalently, H_{0} : μ_{1} = μ_{2} : μ_{1} - μ_{2} = 0$

Alternative hypothesis ( $H_{a}$ ): Depends on the question being asked.

$H_{a} H_{a} H_{a} : μ_{1} - μ_{2} < 0 (left-tailed: μ_{1} < μ_{2}) : μ_{1} - μ_{2} > 0 (right-tailed: μ_{1} > μ_{2}) : μ_{1} - μ_{2} \neq = 0 (two-sided)$

Practice problem

Example:

A tech company’s human resources director claims that engineers with assigned mentors and engineers without mentors spend the same amount of time working at the company before receiving their first promotion, so the company should no longer have mentors for the engineers. One employee thinks that engineers without mentors spend more time at the company before getting their first promotion.

A statistician takes two independent random samples of engineers and records the time (in months) before the first promotion:

Group 1 (no mentor): $25$ , $28$ , $31$ , $26$ , $24$

Group 2 (mentor): $18$ , $22$ , $24$ , $20$ , $19$ , $21$

Let $μ_{1}$ be the population mean for the no-mentor group, and let $μ_{2}$ be the population mean for the mentor group.

Part a.

Is a significance test possible? What assumptions are we making?

To run the test, all three conditions for inference must be checked:

Independent random samples
Normality (or large samples)
10% condition (if sampling without replacement)

Solution:

(spoiler)

Independent random samples: The problem states the samples are independent and random, so this condition is met (no extra assumption needed).
Normality / large samples: The sample sizes are not at least $30$ (they are $5$ and $6$ ), so we must assume both populations are approximately normally distributed. If that assumption is false, the test results may not be reliable.
10% condition (when sampling without replacement): The problem doesn’t give the total number of engineers in each group, so we must assume each sample is less than $10%$ of its population. That means assuming there are at least $50$ engineers without mentors and at least $60$ engineers with mentors. If either population is smaller than that, the test conclusions may not be reliable.

Part b.

Restate the parameters and the hypotheses.

The employee’s claim is that the no-mentor group waits longer, so we use a right-tailed test.

Solution:

(spoiler)

$μ_{1}$ : the mean time to first promotion (in months) for engineers without a mentor
$μ_{2}$ : the mean time to first promotion (in months) for engineers with a mentor

$H_{0} H_{a} : μ_{1} = μ_{2} : μ_{1} > μ_{2}$

Part c.

Complete the test and interpret the P-value.

First, compute the sample means:

$\overset{x}{ˉ}_{1} = \frac{134}{5} = 26.8 months$

$\overset{x}{ˉ}_{2} = \frac{124}{6} \approx 20.67 months$

The difference of sample means:

$\overset{x}{ˉ}_{1} - \overset{x}{ˉ}_{2} \approx 6.13$

Next, find the sample standard deviations using a TI-84:

Enter the data into lists.
Press STAT, highlight CALC, choose 1: 1-Var Stats.
Type L1 (2nd function 1) and press ENTER.

This gives:

$s_{1} s_{2} \approx 2.77 \approx 2.16$

Compute the standard error of $\overset{x}{ˉ}_{1} - \overset{x}{ˉ}_{2}$ :

$SE = \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}} = \frac{7.70}{5} + \frac{4.67}{6} = 1.54 + 0.778 = 2.318 \approx 1.522$

Calculate the test statistic:

$t = \frac{( x ˉ _{1} - x ˉ _{2} ) - 0}{SE} = \frac{26.8 - 20.667}{1.522} = \frac{6.133}{1.522} \approx 4.03$

Compute the degrees of freedom using Welch’s approximation:

To find the p-value on a TI-84, use 2nd → VARS → 6: tcdf(lower, upper, degrees of freedom).

Because this is a right-tailed test:

$tcdf (4.03, 1E99, 7.53) \approx 0.0022$

Solution:

(spoiler)

The p-value is approximately $0.0022$ .

Part d.

Give a conclusion in context at $α = 0.05$ .

Solution:

(spoiler)

Since

$0.0022 < 0.05$

we reject $H_{0}$ . There is strong evidence that engineers without mentors receive their first promotion later on average than engineers with mentors.

Part e.

Given your conclusion, what type of error (Type I or II) might have been made and what would the consequences of this error be?

Solution:

(spoiler)

Because we rejected $H_{0}$ , the possible mistake is a Type I error.

A Type I error here would mean concluding that engineers without mentors take longer to get promoted (on average) even though the population means are actually the same.

Conditions for two-sample t-test

Two independent random samples
Each population roughly normal, or both sample sizes ≥ 30 (Central Limit Theorem)
Sample size < 10% of population (if sampling without replacement)

Hypotheses for comparing two means

Null hypothesis: $H_{0} : μ_{1} = μ_{2}$ or $H_{0} : μ_{1} - μ_{2} = 0$
Alternative hypothesis:
- Right-tailed: $H_{a} : μ_{1} > μ_{2}$
- Left-tailed: $H_{a} : μ_{1} < μ_{2}$
- Two-sided: $H_{a} : μ_{1} \neq = μ_{2}$

Test statistic and standard error

Test statistic: $t = \frac{( x ˉ _{1} - x ˉ _{2} ) - 0}{SE}$
Standard error: $SE = \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}}$
Degrees of freedom: use Welch’s approximation

Interpreting the p-value

P-value: probability of observed (or more extreme) difference if $H_{0}$ is true
Small P-value (e.g., $< α$ ): reject $H_{0}$ , evidence for alternative

Types of errors

Type I error: reject $H_{0}$ when $H_{0}$ is true
- Consequence: act as if means differ when they do not
Type II error: fail to reject $H_{0}$ when $H_{a}$ is true

Summary

Two-sample t-test evaluates if sample mean difference reflects a true population mean difference
Requires independence, random sampling, and normality (or large samples)
Key formulas:
- Test statistic: $t = \frac{( x ˉ _{1} - x ˉ _{2} ) - 0}{SE}$
- Standard error: $SE = \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}}$

Next | 7.2 The t distribution

Significance test for the difference of two means

To run a significance test for the difference between two means, check the following:

You have two independent random samples (ideally simple random samples).
Each population is roughly normal, or both sample sizes are at least $30$ so the Central Limit Theorem applies.
If you’re sampling without replacement, each sample is less than $10%$ of its population.

If all three conditions are met, it’s reasonable to conduct a significance test for the difference between two means.

Hypotheses for comparing two means

Definitions

Null hypothesis ( $H_{0}$ ): Says there is no difference between the population means.

$H_{0} or equivalently, H_{0} : μ_{1} = μ_{2} : μ_{1} - μ_{2} = 0$

Alternative hypothesis ( $H_{a}$ ): Depends on the question being asked.

$H_{a} H_{a} H_{a} : μ_{1} - μ_{2} < 0 (left-tailed: μ_{1} < μ_{2}) : μ_{1} - μ_{2} > 0 (right-tailed: μ_{1} > μ_{2}) : μ_{1} - μ_{2} \neq = 0 (two-sided)$

Practice problem

Example:

A tech company’s human resources director claims that engineers with assigned mentors and engineers without mentors spend the same amount of time working at the company before receiving their first promotion, so the company should no longer have mentors for the engineers. One employee thinks that engineers without mentors spend more time at the company before getting their first promotion.

A statistician takes two independent random samples of engineers and records the time (in months) before the first promotion:

Group 1 (no mentor): $25$ , $28$ , $31$ , $26$ , $24$

Group 2 (mentor): $18$ , $22$ , $24$ , $20$ , $19$ , $21$

Let $μ_{1}$ be the population mean for the no-mentor group, and let $μ_{2}$ be the population mean for the mentor group.

Part a.

Is a significance test possible? What assumptions are we making?

To run the test, all three conditions for inference must be checked:

Independent random samples
Normality (or large samples)
10% condition (if sampling without replacement)

Solution:

(spoiler)

Independent random samples: The problem states the samples are independent and random, so this condition is met (no extra assumption needed).
Normality / large samples: The sample sizes are not at least $30$ (they are $5$ and $6$ ), so we must assume both populations are approximately normally distributed. If that assumption is false, the test results may not be reliable.
10% condition (when sampling without replacement): The problem doesn’t give the total number of engineers in each group, so we must assume each sample is less than $10%$ of its population. That means assuming there are at least $50$ engineers without mentors and at least $60$ engineers with mentors. If either population is smaller than that, the test conclusions may not be reliable.

Part b.

Restate the parameters and the hypotheses.

The employee’s claim is that the no-mentor group waits longer, so we use a right-tailed test.

Solution:

(spoiler)

$μ_{1}$ : the mean time to first promotion (in months) for engineers without a mentor
$μ_{2}$ : the mean time to first promotion (in months) for engineers with a mentor

$H_{0} H_{a} : μ_{1} = μ_{2} : μ_{1} > μ_{2}$

Part c.

Complete the test and interpret the P-value.

First, compute the sample means:

$\overset{x}{ˉ}_{1} = \frac{134}{5} = 26.8 months$

$\overset{x}{ˉ}_{2} = \frac{124}{6} \approx 20.67 months$

The difference of sample means:

$\overset{x}{ˉ}_{1} - \overset{x}{ˉ}_{2} \approx 6.13$

Next, find the sample standard deviations using a TI-84:

Enter the data into lists.
Press STAT, highlight CALC, choose 1: 1-Var Stats.
Type L1 (2nd function 1) and press ENTER.

This gives:

$s_{1} s_{2} \approx 2.77 \approx 2.16$

Compute the standard error of $\overset{x}{ˉ}_{1} - \overset{x}{ˉ}_{2}$ :

$SE = \frac{s _{1}^{2}}{n _{1}} + \frac{s _{2}^{2}}{n _{2}} = \frac{7.70}{5} + \frac{4.67}{6} = 1.54 + 0.778 = 2.318 \approx 1.522$

Calculate the test statistic:

$t = \frac{( x ˉ _{1} - x ˉ _{2} ) - 0}{SE} = \frac{26.8 - 20.667}{1.522} = \frac{6.133}{1.522} \approx 4.03$

Compute the degrees of freedom using Welch’s approximation:

To find the p-value on a TI-84, use 2nd → VARS → 6: tcdf(lower, upper, degrees of freedom).

Because this is a right-tailed test:

$tcdf (4.03, 1E99, 7.53) \approx 0.0022$

Solution:

(spoiler)

The p-value is approximately $0.0022$ .

Part d.

Give a conclusion in context at $α = 0.05$ .

Solution:

(spoiler)

Since

$0.0022 < 0.05$

we reject $H_{0}$ . There is strong evidence that engineers without mentors receive their first promotion later on average than engineers with mentors.

Part e.

Given your conclusion, what type of error (Type I or II) might have been made and what would the consequences of this error be?

Solution:

(spoiler)

Because we rejected $H_{0}$ , the possible mistake is a Type I error.

A Type I error here would mean concluding that engineers without mentors take longer to get promoted (on average) even though the population means are actually the same.