Slope and sampling distribution

The Sampling Distribution of the Sample Slope

In linear regression, we model the relationship between two quantitative variables. The sample slope $b$ (calculated from your sample data) is used to estimate the population slope $\beta$ (the true slope in the population). If you repeatedly took samples and computed a slope each time, the pattern of those slopes is described by the sampling distribution of $b$.

Definitions

Sample Slope ($b$): The slope of the least-squares regression line calculated from a sample. It is a statistic used to estimate the true population slope.

Population Slope ($\beta$): The true slope of the linear relationship between the response and explanatory variables in the entire population. It is a parameter we typically do not know.

Sampling Distribution of $b$: The distribution of all possible sample slopes from samples of a given size $n$ taken from the same population.

Standard Error of the Slope ($s_b$): An estimate of how much the sample slope $b$ typically varies from the true population slope $\beta$ across repeated samples.

Theoretical Conditions

The following theoretical conditions must be met for inference on the slope to be valid:

1. Linearity: the true relationship between the response and explanatory variables must be linear.
2. Constant Variability: the standard deviation of the residuals $\sigma$ must be constant for all values of $x$.
3. Normality: for each value of $x$, the $y$-values are approximately normally distributed.

You usually won’t be given confirmation of these conditions directly. Instead, you check whether they’re reasonable in practice by examining the residual plots you’re given.

Key Properties of the Sampling Distribution of $b$

If the true relationship between the variables is linear, and the theoretical conditions listed above are met, then the sampling distribution of the sample slope $b$ has the following properties:

1. Shape: the distribution is approximately normal.
2. Center: the mean of the sample slopes ${\mu }_b$ is equal to the population slope $\beta$. This means $b$ is an unbiased estimator of $\beta$.
3. Spread: the standard deviation of the sample slopes (also called the standard error of the slope) is:

$${\sigma }_b=\frac{\sigma }{{\sigma }_x\sqrt{n-1}}$$

Where:

• $\sigma$ is the standard deviation of the residuals in the population,
• ${\sigma }_x$ is the standard deviation of the $x$-values,
• $n$ is the sample size.

Estimating with Sample Statistics

Typically, you won’t know the population parameters $\sigma$ and ${\sigma }_x$, so you estimate them using sample statistics:

• $s$: the standard deviation of the sample residuals.
• $s_x$: the standard deviation of the sample $x$-values.

The estimated standard error of the slope:

$$s_b=\frac{s}{s_x\sqrt{n-1}}$$

When you use $s_b$ to estimate ${\sigma }_b$, the standardized statistic:

$$t=\frac{b-\beta }{s_b}$$

follows a $t$-distribution with degrees of freedom:

$$df=n-2$$