Goodness of fit test

Chi-square goodness of fit test

The chi-square goodness of fit test checks whether a sample of categorical data is consistent with a claimed (theoretical) distribution. In other words, it helps you judge whether the differences between the observed counts and the expected counts are small enough to be explained by random sampling variation.

A perfect match between sample data and a theoretical distribution is unlikely. The real question is whether the mismatch you see is reasonable if the claimed distribution is true.

Definitions

Chi-square goodness of fit test

A hypothesis test that compares observed counts in each category of a categorical variable to the counts you would expect if a claimed distribution were true.

Observed frequencies ($O_i$)

The counts (or amounts) you actually see in your sample.

Expected frequencies ($E_i$)

The counts you would expect if the null hypothesis were true.

Hypotheses:

The null hypothesis $H_0$ for a chi-square goodness of fit test is that the given theoretical distribution does accurately represent the data of the overall population.

The alternate hypothesis $H_a$ for a chi-square goodness of fit test is that the given theoretical distribution does not accurately represent the data of the overall population.

Conditions for inference

1. Random sample: Data should come from a random sample or randomized experiment.
2. Independence: Each observation should be independent (if sampling without replacement, sample size $<10\%$ of population).
3. Sample size: All expected frequencies must be at least $5$ ($E_i\ge 5$ for all categories).

How the test works

The test asks: If the claimed distribution is correct, how likely is it to see discrepancies at least as large as the ones in our sample?

• If that chance (the p-value) is large, the discrepancies are plausible under $H_0$. You fail to reject $H_0$.
• If that chance is small, the discrepancies are unlikely under $H_0$. You reject $H_0$ and conclude the claimed distribution is not a good fit for the population.

Definitions

Chi-square test statistic (${\chi }^2$)

A measure of how far the observed counts are from the expected counts. The smaller the ${\chi }^2$ value, the better the fit.

$$\begin{array}{rl}{\chi }^2& =\sum \frac{(O_i-E_i{)}^2}{E_i}\end{array}$$

p-value

The probability of obtaining a ${\chi }^2$ value as extreme as (or more extreme than) the one obtained, assuming the null hypothesis is true.

How large is “large” for ${\chi }^2$?

A ${\chi }^2$ value is considered “large” when it falls far out in the right tail of the appropriate chi-square distribution (which depends on the degrees of freedom). That’s exactly what the p-value measures. To see why, it helps to know the basic shape of the chi-square distribution.

Definitions

Chi-square (${\chi }^2$) distribution

A probability distribution that is right-skewed and takes only non-negative values. There is a different chi-square distribution for each number of degrees of freedom — the larger the degrees of freedom, the less skewed the distribution becomes, and the closer it is to a normal distribution.

Degrees of freedom ($df$)

A value that determines which specific chi-square distribution applies.

$$\begin{array}{rl}df& =(\text{number of categories})-1\end{array}$$

Practice problem

Example:

Candy World company claims that its premium chocolate mix is made up of the following percentages by weight:

• $30\%$ caramel delights
• $25\%$ almond clusters
• $20\%$ coconut dreams
• $15\%$ mint swirls
• $10\%$ pecan turtles

A customer purchases a bag, sorts the contents, and finds that in her bag she received:

• $170$ grams of caramel delights
• $110$ grams of almond clusters
• $90$ grams of coconut dreams
• $80$ grams of mint swirls
• $50$ grams of pecan turtles

Part a.

State the hypotheses for a chi-square goodness of fit test if the goal is to determine if the customer’s bag differs significantly from the company’s claim.

Solution:

Part b.

Calculate the expected frequencies.

First, find the total weight of the candies in the bag:

$$\begin{array}{rl}\text{Total weight}& =170+110+90+80+50\\ & =500\text{ grams}\end{array}$$

To find each expected frequency, multiply each claimed proportion by the total weight:

$$\begin{array}{rl}{E}_i& =p_i\times \text{Total weight}\end{array}$$

Solution:

Part c.

Calculate the chi-square test statistic (${\chi }^2$).

Use the formula:

$$\begin{array}{rl}{\chi }^2& =\sum \frac{(O-E{)}^2}{E}\end{array}$$

Substitute the observed and expected values:

$$\begin{array}{rl}{\chi }^2& =\frac{(170-150{)}^2}{150}+\frac{(110-125{)}^2}{125}+\frac{(90-100{)}^2}{100}+\frac{(80-75{)}^2}{75}+\frac{(50-50{)}^2}{50}\\ & =\frac{400}{150}+\frac{225}{125}+\frac{100}{100}+\frac{25}{75}+0\\ & =2.667+1.800+1.000+0.333+0\end{array}$$

Solution:

Part d.

Determine the degrees of freedom and find the p-value.

The number of categories is $k=5$.

Compute the degrees of freedom:

$$\begin{array}{rl}df& =k-1\\ & =5-1\end{array}$$

The p-value is the right-tail probability:

$$\begin{array}{rl}p& =P({\chi }_{df}^2\ge 5.8)\end{array}$$

Solution:

Part e.

Using a significance level $\alpha =0.05$, what is your conclusion about the company’s claim in the context of the problem?

Compare the p-value to $\alpha$:

$$\begin{array}{rl}0.215& >0.05\end{array}$$

Solution:

Part f.

What type of error might have been made and what would the potential consequences of that error be in context?

Solution: