Significance tests for difference of two proportions

Significance test for two proportions

The goal of a significance test for two proportions is to decide whether an observed difference between two sample proportions reflects a real difference between the two population proportions, or whether the sample difference could reasonably be explained by natural sampling variation.

For this test, the null hypothesis says there is no difference between the population proportions:

$$\begin{array}{rl}{H}_0& :p_1=p_2\end{array}$$

Conditions for using the two-proportion z test

Large counts (normality) condition

Definitions

Combined (pooled) proportion (${\hat{p}}_c$)

The proportion calculated by combining the successes and sample sizes from both samples.

$$\begin{array}{rl}{\hat{p}}_c& =\frac{x_1+x_2}{n_1+n_2}\end{array}$$

To use the normal approximation, all four of the following quantities must be at least $10$:

$$\begin{array}{rl}{n}_1{\hat{p}}_c& \ge 10\\ n_1(1-{\hat{p}}_c)& \ge 10\\ n_2{\hat{p}}_c& \ge 10\\ n_2(1-{\hat{p}}_c)& \ge 10\end{array}$$

Randomness and independence

• Each sample should be a random sample (ideally a simple random sample).
• The two samples must be independent of each other.
• If sampling is done without replacement, each population size $N_1$ and $N_2$ should be at least $10$ times the corresponding sample size (the “10% condition”).

Choosing the alternative hypothesis

The alternative hypothesis depends on what the question is asking.

Definitions

Two-sided test

Checks for a difference in either direction. Used when the context only asks whether the two proportions differ.

$$\begin{array}{rl}{H}_a& :p_1\ne p_2\end{array}$$

One-sided test

Used only when the context clearly calls for a specific direction.

$$\begin{array}{rl}{H}_a& :p_1>p_2\\ \text{or}{H}_a& :p_1<p_2\end{array}$$

Key facts and equations:

1. The null hypothesis for a significance test for the difference of two proportions is that the population proportions are equal: $H_0:p_1=p_2$.

2. The alternate hypothesis is one of:

$$\begin{array}{rl}{H}_a& :p_1\ne p_2\\ H_a& :p_1<p_2\\ H_a& :p_1>p_2\end{array}$$

3. $n_1{\hat{p}}_c$, $n_1(1-{\hat{p}}_c)$, $n_2{\hat{p}}_c$, and $n_2(1-{\hat{p}}_c)$ must each be at least $10$ in order to apply the normality condition.

4. The combined/pooled proportion equation is:

$$\begin{array}{rl}{\hat{p}}_c& =\frac{x_1+x_2}{n_1+n_2}\end{array}$$

5. The standard error of the distribution of differences in sample proportions is:

$$\begin{array}{rl}\text{SE}& =\sqrt{{\hat{p}}_c(1-{\hat{p}}_c)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}\end{array}$$

Practice problem

Example:

After a major wildfire, an organization surveyed two different local districts to see how many homeowners were able to get the emergency help that they needed. In a random sample of $720$ homeowners from district A, $432$ said that they were able to get the help that they needed. In another random sample of $610$ homeowners from district B, $336$ said they were able to get the help they needed. The $95\%$ confidence interval for $p_A-p_B$ is $(-0.004,0.102)$.

Part a.

Does the data that is given provide convincing evidence that a greater proportion of district A homeowners were able to get help than district B homeowners?

Define the parameters:

• Let $p_A$ represent the true proportion of homeowners in district A who were able to get the help they needed.
• Let $p_B$ represent the true proportion of homeowners in district B who were able to get the help they needed.

State the hypotheses:

$$\begin{array}{rl}{H}_0& :p_A=p_B\\ H_a& :p_A>p_B\end{array}$$

The alternative hypothesis is one-sided because the question is specifically asking whether the proportion in district A is greater than the proportion in district B. The key question is whether the observed sample difference is large enough to be unlikely under $H_0$, given the natural variation we expect from random sampling.

Check the large counts (normality) condition:

First compute the pooled proportion:

$$\begin{array}{rl}{\hat{p}}_c& =\frac{432+336}{720+610}\\ & =\frac{768}{1330}\\ & \approx 0.577\end{array}$$

Now check each value:

$$\begin{array}{rl}{n}_A{\hat{p}}_c& =720(0.577)\\ & =415.44\end{array}$$

$$\begin{array}{rl}{n}_A(1-{\hat{p}}_c)& =720(0.423)\\ & =304.56\end{array}$$

$$\begin{array}{rl}{n}_B{\hat{p}}_c& =610(0.577)\\ & =351.97\end{array}$$

$$\begin{array}{rl}{n}_B(1-{\hat{p}}_c)& =610(0.423)\\ & =258.03\end{array}$$

Since all four values are at least $10$, the large counts condition is met.

Check the independence requirement:

The independence requirement is that each sample size should be less than $10\%$ of its population size. Although the exact number of homeowners in each district is not given, it is reasonable to assume each district has far more than $7,200$ and $6,100$ homeowners, so the $10\%$ condition is likely satisfied.

Complete the test with a TI-84 calculator:

• Step 1: Press STAT, scroll over to TESTS, scroll down to 6: 2-PropZTest, and press ENTER.
• Step 2: Input the following data:
  • $x_1$ = number of successes in sample 1 (using district A data: $432$)
  • $n_1$ = size of sample 1 ($720$)
  • $x_2$ = number of successes in sample 2 (using district B data: $336$)
  • $n_2$ = size of sample 2 ($610$)
  • Choose alternative hypothesis: $>$

The calculator gives:

$$\begin{array}{rl}z& \approx 1.809\\ p& =0.0352\\ {\hat{p}}_1& =0.6\\ {\hat{p}}_2& =0.551\end{array}$$

Compare the p-value to $\alpha =0.05$:

$$\begin{array}{rl}0.0352& <0.05\end{array}$$

Solution:

Part b.

Is the result consistent with the $95\%$ confidence interval that is given above?

Solution:

Part c.

If the null hypothesis had been that the two proportions were equal, what type of error (Type I or Type II) could have potentially been made based on the conclusion?

Solution: