Hypothesis test for the slope of a least squares regression

Significance Test for the Slope of a Regression Line

The hypothesis test for the slope in a least squares regression answers a focused question: does the sample provide convincing evidence of a real linear relationship between $x$ and $y$ in the population, or could the observed trend be due to random chance?

You use the sample slope $b$ to make an inference about the true population slope $β$ .

Definitions

Population Slope ( $β$ ): The true slope of the linear relationship between $x$ and $y$ in the entire population.

Sample Slope ( $b$ ): The slope of the least squares regression line calculated from sample data, used as an estimate of $β$ .

Test Statistic ( $t$ ): A standardized value that measures how many standard errors the sample slope $b$ is away from the hypothesized slope $β_{0}$ .

Standard Error of the Slope ( $S E_{b}$ ): A measure of how much the sample slope is expected to vary across repeated samples. Smaller $S E_{b}$ means a more precise estimate of $β$ .

Setting Up the Hypotheses

Before running a test, you need to translate the research question into a pair of hypotheses about the population slope $β$ . The null hypothesis always represents the “no relationship” case ( $β = 0$ ), and the alternative hypothesis describes the kind of relationship you’re looking for — any linear relationship (two-tailed), a positive one, or a negative one. Your choice of alternative depends on the context of the problem and what direction (if any) you have reason to suspect before looking at the data.

Hypotheses

The null hypothesis states that $β = 0$ , meaning there is no linear relationship between $x$ and $y$ in the population:

$H_{0} : β = 0$

The alternative hypothesis is one of the following:

Two-tailed test: $H_{a} : β \neq = 0$ — there is a linear relationship (positive or negative).
One-tailed positive test: $H_{a} : β > 0$ — there is a positive linear relationship.
One-tailed negative test: $H_{a} : β < 0$ — there is a negative linear relationship.

Conditions for the Test

To use this hypothesis test, the following conditions must be met:

Independence: the sample must be random. If sampling is done without replacement, then the sample size $n$ must be less than or equal to $10%$ of the population size.
Normality: for each $x$ -value, the associated $y$ -values are normally distributed. Recall that if the sample size $n \geq 30$ , then the central limit theorem applies.
Linearity: the true relationship between $x$ and $y$ is linear.
Equal Standard Deviation: the standard deviation of $y$ is the same for all $x$ -values.

The Test Statistic

To measure how far the sample slope is from the hypothesized population slope (in standard error units), compute:

$t = \frac{b - β _{0}}{S E _{b}}$

Where:

$t$ : The test statistic, which measures how many standard errors the sample slope is away from the hypothesized population slope.
$b$ : The sample slope from the regression line ( $\overset{y}{^} = a + b x$ ).
$β_{0}$ : The hypothesized value of the slope from the null hypothesis (typically $0$ ).
$S E_{b}$ : The standard error of the slope. This measures how much the sample slope $b$ varies from sample to sample. A smaller $S E_{b}$ means a more precise estimate.

Degrees of freedom: $df = n - 2$ .

(spoiler)

Making a Decision

There are two possible ways to make a decision:

Method 1: p-value approach

Use the test statistic $t$ and the degrees of freedom ( $df = n - 2$ ) to find the p-value.

If $p -value \leq α$ , reject $H_{0}$ — there is significant evidence of a linear relationship.
If $p -value > α$ , fail to reject $H_{0}$ — there is not significant evidence of a linear relationship.

Method 2: Critical value approach

Find the critical t-value from the t-distribution using the degrees of freedom ( $df = n - 2$ ) and the significance level $α$ .

For $H_{a} : β \neq = 0$ : reject $H_{0}$ if $∣ t ∣ \geq t^{*}$ .
For $H_{a} : β > 0$ : reject $H_{0}$ if $t \geq t^{*}$ .
For $H_{a} : β < 0$ : reject $H_{0}$ if $t \leq - t^{*}$ .