Statistics

This chapter covers the following topics:

• Dispersion, mean, median, and mode
• Probability density function (PDF)
• Cumulative distribution function (CDF)
• Binomial distribution
• Normal distribution (Gaussian distribution)
• t-distribution
• Chi-square (${\chi }^2$) distribution
• Gamma distribution
• Hypothesis testing
• Linear regression and goodness of fit
• Distribution functions and tables

Dispersion, mean, median, and mode

If $X_1,X_2,\dots ,X_n$ represent the values of a random sample of $n$ items or observations, the arithmetic mean of these items or observations, denoted by $\overline{X}$, is defined as

$$\overline{X}=\frac{1}{n}(X_1+X_2+\cdots +X_n)=\frac{1}{n}\sum _{i=1}^n{X}_i$$

$$\overline{X}\to \mu \text{ for sufficiently large values of }n$$

The weighted arithmetic mean is

$$\overline{X}=\frac{\sum w_i{X}_i}{\sum w_i}$$

where

$$X_i=\text{value of the }i\text{th observation}$$

and

$$w_i=\text{weight applied to }{X}_i$$

The variance of the population is the arithmetic mean of the squared deviations from the population mean. If $\mu$ is the arithmetic mean of a discrete population of size $N$, the population variance is defined by

$${\sigma }^2=\frac{1}{N}\left[(X_1-\mu {)}^2+(X_2-\mu {)}^2+\cdots +(X_N-\mu {)}^2\right]$$

$${\sigma }^2=\frac{1}{N}\sum _{i=1}^N(X_i-\mu {)}^2$$

Standard deviation formulas (assuming statistical independence) are

$${\sigma }_{\text{population}}=\sqrt{\frac{1}{N}\sum (X_i-\mu {)}^2}$$

$${\sigma }_{\text{sum}}=\sqrt{{\sigma }_1^2+{\sigma }_2^2+\cdots +{\sigma }_n^2}$$

$${\sigma }_{\text{series}}=\sigma \sqrt{n}$$

$${\sigma }_{\text{mean}}=\frac{\sigma }{\sqrt{n}}$$

The sample variance is

$$s^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$$

Watch out: Divide by $N$ only when you have data for the entire population, giving the population variance ${\sigma }^2$. When you’re working with a sample - the far more common case on the FE - divide by $n-1$, not $n$, to get the sample variance $s^2$ shown above.

The sample standard deviation is

$$s=\sqrt{\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2}$$

The sample coefficient of variation is

$$CV=\frac{s}{\overline{X}}$$

The sample geometric mean is

$$\sqrt[n]{X_1{X}_2{X}_3\cdots X_n}$$

The sample root-mean-square value is

$$\sqrt{\frac{1}{n}\sum X_i^2}$$

When the discrete data are rearranged in increasing order and $n$ is odd, the median is the value of the

$${\left(\frac{n+1}{2}\right)}^{\text{th}}$$

item.

When $n$ is even, the median is the average of the

$${\left(\frac{n}{2}\right)}^{\text{th}}$$

and

$${\left(\frac{n}{2}+1\right)}^{\text{th}}$$

items.

The mode of a set of data is the value that occurs with the greatest frequency. The sample range $R$ is the largest sample value minus the smallest sample value.

Probability density function (PDF)

A probability density function (PDF) describes how probability is distributed across values of a continuous random variable. The PDF itself is not a probability; probabilities come from areas under the curve.

• The PDF is denoted by $f(x)$.

• The total area under the curve is 1:

$${\int }_{-\infty }^{\infty }f(x)dx=1$$

• The probability that $X$ lies in the interval $[a,b]$ is the area under $f(x)$ from $a$ to $b$:

$$P(a\le X\le b)={\int }_a^{b}f(x)dx$$

Example: PDF of the standard normal distribution

$$f(x)=\frac{1}{\sqrt{2\pi }}{e}^{-x^2/2}$$

Cumulative distribution function (CDF)

The cumulative distribution function (CDF) gives the probability that a random variable $X$ is less than or equal to a value $x$. You can think of it as “accumulating” probability from the far left up to $x$.

$$F(x)=P(X\le x)={\int }_{-\infty }^{x}f(t)dt$$

Properties:

• ${\lim }_{x\to -\infty }F(x)=0$
• ${\lim }_{x\to \infty }F(x)=1$

Example: finding a CDF from a PDF

If $f(x)=2x$ for $0<x<1$, find $F(x)$.

$$F(x)={\int }_0^{x}2tdt=x^2$$

Answer: $F(x)=x^2$

Binomial distribution

The binomial distribution models the number of successes in $n$ independent Bernoulli trials, where each trial has:

• two outcomes (success/failure)
• the same probability of success, $p$

PMF (probability mass function):

$$P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k},k=0,1,...,n$$

• Mean: $\mu =np$
• Variance: ${\sigma }^2=np(1-p)$

Example: binomial probability

If $n=5$ and $p=0.6$, find the probability of exactly 3 successes.

$$P(X=3)=\left(\genfrac{}{}{0ex}{}{5}{3}\right)(0.6{)}^3(0.4{)}^2=10\cdot 0.216\cdot 0.16=0.3456$$

Answer: $0.3456$

Normal distribution (Gaussian distribution)

The normal distribution is a symmetric, bell-shaped distribution used widely in statistics.

PDF:

$$f(x)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}$$

• $\mu$: mean
• $\sigma$: standard deviation

Standard normal distribution: $\mu =0$, $\sigma =1$

Example: standard normal z-lookup

A machined part has diameters that are normally distributed with a mean of $\mu =50$ mm and a standard deviation of $\sigma =2$ mm. What is the probability that a randomly selected part has a diameter less than $47$ mm?

Convert to a standard normal $z$-score:

$$z=\frac{x-\mu }{\sigma }=\frac{47-50}{2}=-1.5$$

Look up $z=-1.5$ in the standard normal table in the FE Reference Handbook: $P(Z<-1.5)=0.0668$.

Answer: about $6.68\%$

t-distribution

The t-distribution is commonly used when the population standard deviation is unknown and the sample size is small. It’s shaped like the standard normal distribution but with heavier tails, and its shape depends on the degrees of freedom, $\nu$ - as $\nu$ grows, the t-distribution approaches the standard normal distribution. The FE doesn’t ask you to evaluate the t-distribution’s PDF; instead, the FE Reference Handbook provides t-tables so you can look up critical values by degrees of freedom, which is what you’ll use in a t-test for comparing sample means when $\sigma$ is unknown.

Chi-square (${\chi }^2$) distribution

The chi-square distribution is often used in hypothesis testing involving variance and categorical data, such as testing whether an observed distribution fits an expected one. If $Z_1,Z_2,\dots ,Z_k$ are independent standard normal variables, then:

$${\chi }^2=Z_1^2+Z_2^2+\cdots +Z_k^2$$

The chi-square distribution’s shape depends on its degrees of freedom, $k$. As with the t-distribution, the FE Reference Handbook provides chi-square tables indexed by degrees of freedom, so you look up critical values rather than evaluate the PDF directly.

Gamma distribution

The gamma distribution is a continuous distribution with two parameters: shape $\alpha$ and rate $\beta$ (or scale $\theta =1/\beta$). It’s used to model the waiting time until the $\alpha$-th event occurs in a Poisson process - for example, the time until the third customer arrives at a service counter. As with the other continuous distributions above, the FE expects you to recognize this application rather than evaluate the gamma PDF directly.

Hypothesis testing

Hypothesis testing is a statistical method for using sample data to make a decision about a population parameter.

Steps in hypothesis testing

1. State the hypotheses:

   • Null hypothesis: $H_0$ (no effect or no difference)
   • Alternative hypothesis: $H_1$ (an effect or difference exists)

2. Choose significance level:

   • Denoted by $\alpha$ (commonly 0.05)

3. Compute test statistic:

   • For a population mean with known standard deviation:

$$Z=\frac{\bar{X}-{\mu }_0}{\sigma /\sqrt{n}}$$

4. Determine p-value:

   • P-value: Probability of observing a value as extreme as the test statistic under $H_0$.

5. Make a decision:

   • If $\text{p-value}<\alpha$ → Reject $H_0$
   • If $\text{p-value}\ge \alpha$ → Fail to reject $H_0$

Example

Scenario

A bottling company claims the mean soda volume is 500 ml. A sample of 36 bottles has:

• Sample mean: $\bar{X}=497$
• Population standard deviation: $\sigma =5$
• Significance level: $\alpha =0.05$

Step 1: hypotheses

• $H_0:\mu =500$
• $H_1:\mu \ne 500$ (two-tailed test)

Step 2: test statistic

Given:

• $\bar{X}=497$, ${\mu }_0=500$, $\sigma =5$, $n=36$

$$Z=\frac{497-500}{5/\sqrt{36}}=\frac{-3}{0.8333}=-3.60$$

Exam tip: Carry full precision through every intermediate step and round only the final answer. Here, $5/\sqrt{36}=0.8\overline{3}$; rounding it to $0.833$ too early can shift your result enough to match a neighboring distractor on the exam.

Step 3: p-value

Using Z-tables:

• $\text{P-value}=2\times P(Z<-3.60)$
• $\text{P-value}\approx 2\times 0.00016=0.00032$

Step 4: decision

Since:

• $0.00032<0.05$ → Reject $H_0$

Conclusion: There is strong evidence that the mean volume is not 500 ml.

Confidence interval (CI)

To construct a 95% CI for the mean:

$$\bar{X}\pm Z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}$$

With:

• $\bar{X}=497$, $Z_{0.025}=1.96$, $\sigma =5$, $n=36$

$$497\pm 1.96\cdot \frac{5}{\sqrt{36}}=497\pm 1.633$$

$$\Rightarrow \text{CI}=[495.37,498.63]$$

The claimed mean of 500 ml falls outside this interval, consistent with the hypothesis test conclusion above.

Type I and Type II errors

Error type	Description	Symbol	Consequence
Type I	Rejecting $H_0$ when it’s true	$\alpha$	False positive
Type II	Failing to reject $H_0$ when it’s false	$\beta$	False negative

Power of the test

• $\text{Power}=1-\beta$
• It is the probability of correctly rejecting a false $H_0$.

Linear regression and goodness of fit

Linear regression models the relationship between a dependent variable $y$ and an independent variable $x$ using a line plus an error term:

$$y=a+bx+\epsilon$$

Where:

• $y$: dependent variable
• $x$: independent variable
• $a$: intercept of the regression line
• $b$: slope of the regression line
• $\epsilon$: error term

Least squares method

The least squares method chooses the line that minimizes the sum of squared differences between observed values and predicted values.

Formulas for best fit line

Given data points $(x_1,y_1),(x_2,y_2),\dots ,(x_n,y_n)$, the slope $b$ and intercept $a$ are calculated as:

$$b=\frac{n\sum x_i{y}_i-\sum x_i\sum y_i}{n\sum x_i^2-(\sum x_i{)}^2}$$

$$a=\bar{y}-b\bar{x}$$

Where:

• $\bar{x}=\frac{1}{n}\sum x_i$
• $\bar{y}=\frac{1}{n}\sum y_i$

Example

Given data

$x$	$y$
1	2
2	3
3	5
4	4
5	6

Step 1: compute sums

$$\sum x=15,\sum y=20,\sum x^2=55,\sum xy=69,n=5$$

Step 2: compute slope and intercept

$$b=\frac{5(69)-(15)(20)}{5(55)-(15{)}^2}=\frac{345-300}{275-225}=\frac{45}{50}=0.9$$

$$a=\bar{y}-b\bar{x}=\frac{20}{5}-0.9\cdot \frac{15}{5}=4-0.9\cdot 3=1.3$$

Final regression line

$$y=1.3+0.9x$$

Goodness of fit

Sample correlation coefficient $R$

The correlation coefficient $R$ measures the strength and direction of the linear relationship:

$$R=\frac{n\sum x_i{y}_i-\sum x_i\sum y_i}{\sqrt{[n\sum x_i^2-(\sum x_i{)}^2][n\sum y_i^2-(\sum y_i{)}^2]}}$$

Given $\sum y^2=90$:

$$R=\frac{5(69)-(15)(20)}{\sqrt{[5(55)-15^2][5(90)-20^2]}}=\frac{45}{\sqrt{2500}}=\frac{45}{50}=0.9$$

Coefficient of determination $R^2$

The coefficient of determination shows the proportion of variance in $y$ explained by the model:

$$R^2=(R{)}^2=(0.9{)}^2=0.81$$

Interpretation: 81% of the variation in $y$ is explained by the linear regression model.

Distribution functions and tables

The FE Reference Handbook is the only reference you’re allowed during the exam, and it includes the normal, t, and chi-square distribution tables along with the linear regression equations covered above. Spend time practicing how to locate and read these tables and formulas in the FE Handbook - the exam tests whether you can find and apply them correctly, not whether you’ve memorized them.