Statistics

This chapter covers the following topics:

• Dispersion, mean, median, and mode
• Probability density function (PDF)
• Cumulative distribution function (CDF)
• Binomial distribution
• Normal distribution (Gaussian distribution)
• t-distribution
• Chi-square (${\chi }^2$) distribution
• Gamma distribution
• Hypothesis testing
• Linear regression and goodness of fit
• Distribution functions and tables

Dispersion, mean, median, and mode

If $X_1,X_2,\dots ,X_n$ represent the values of a random sample of $n$ items or observations, the arithmetic mean of these items or observations, denoted by $\overline{X}$, is defined as

$$\overline{X}=\frac{1}{n}(X_1+X_2+\cdots +X_n)=\frac{1}{n}\sum _{i=1}^n{X}_i$$

$$\overline{X}\to \mu \text{ for sufficiently large values of }n$$

The weighted arithmetic mean is

$$\overline{X}=\frac{\sum w_i{X}_i}{\sum w_i}$$

where

$$X_i=\text{value of the }i\text{th observation}$$

and

$$w_i=\text{weight applied to }{X}_i$$

The variance of the population is the arithmetic mean of the squared deviations from the population mean. If $\mu$ is the arithmetic mean of a discrete population of size $N$, the population variance is defined by

$${\sigma }^2=\frac{1}{N}\left[(X_1-\mu {)}^2+(X_2-\mu {)}^2+\cdots +(X_N-\mu {)}^2\right]$$

$${\sigma }^2=\frac{1}{N}\sum _{i=1}^N(X_i-\mu {)}^2$$

Standard deviation formulas (assuming statistical independence) are

$${\sigma }_{\text{population}}=\sqrt{\frac{1}{N}\sum (X_i-\mu {)}^2}$$

$${\sigma }_{\text{sum}}=\sqrt{{\sigma }_1^2+{\sigma }_2^2+\cdots +{\sigma }_n^2}$$

$${\sigma }_{\text{series}}=\sigma \sqrt{n}$$

$${\sigma }_{\text{mean}}=\frac{\sigma }{\sqrt{n}}$$

The sample variance is

$$s^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$$

The sample standard deviation is

$$s=\sqrt{\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2}$$

The sample coefficient of variation is

$$CV=\frac{s}{\overline{X}}$$

The sample geometric mean is

$$\sqrt[n]{X_1{X}_2{X}_3\cdots X_n}$$

The sample root-mean-square value is

$$\sqrt{\frac{1}{n}\sum X_i^2}$$

When the discrete data are rearranged in increasing order and $n$ is odd, the median is the value of the

$${\left(\frac{n+1}{2}\right)}^{\text{th}}$$

item.

When $n$ is even, the median is the average of the

$${\left(\frac{n}{2}\right)}^{\text{th}}$$

and

$${\left(\frac{n}{2}+1\right)}^{\text{th}}$$

items.

The mode of a set of data is the value that occurs with the greatest frequency. The sample range $R$ is the largest sample value minus the smallest sample value.

Probability density function (PDF)

A probability density function (PDF) describes how probability is distributed across values of a continuous random variable. The PDF itself is not a probability; probabilities come from areas under the curve.

• The PDF is denoted by $f(x)$.

• The total area under the curve is 1:

$${\int }_{-\infty }^{\infty }f(x)dx=1$$

• The probability that $X$ lies in the interval $[a,b]$ is the area under $f(x)$ from $a$ to $b$:

$$P(a\le X\le b)={\int }_a^{b}f(x)dx$$

Example: PDF of the standard normal distribution:

$$f(x)=\frac{1}{\sqrt{2\pi }}{e}^{-x^2/2}$$

Cumulative distribution function (CDF)

The cumulative distribution function (CDF) gives the probability that a random variable $X$ is less than or equal to a value $x$. You can think of it as “accumulating” probability from the far left up to $x$.

$$F(x)=P(X\le x)={\int }_{-\infty }^{x}f(t)dt$$

Properties:

• ${\lim }_{x\to -\infty }F(x)=0$
• ${\lim }_{x\to \infty }F(x)=1$

Example: If $f(x)=2x$ for $0<x<1$, then:

$$F(x)={\int }_0^{x}2tdt=x^2$$

Binomial distribution

The binomial distribution models the number of successes in $n$ independent Bernoulli trials, where each trial has:

• two outcomes (success/failure)
• the same probability of success, $p$

PMF (probability mass function):

$$P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k},k=0,1,...,n$$

• Mean: $\mu =np$
• Variance: ${\sigma }^2=np(1-p)$

Example: If $n=5$ and $p=0.6$, then the probability of 3 successes is:

$$P(X=3)=\left(\genfrac{}{}{0ex}{}{5}{3}\right)(0.6{)}^3(0.4{)}^2=10\cdot 0.216\cdot 0.16=0.3456$$

Normal distribution (Gaussian distribution)

The normal distribution is a symmetric, bell-shaped distribution used widely in statistics.

PDF:

$$f(x)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}$$

• $\mu$: mean
• $\sigma$: standard deviation

Standard normal distribution: $\mu =0$, $\sigma =1$

Example: Human height, IQ scores, etc.

t-distribution

The t-distribution is commonly used when the population standard deviation is unknown and the sample size is small.

PDF:

$$f(t)=\frac{\Gamma \left(\frac{\nu +1}{2}\right)}{\sqrt{\nu \pi }\Gamma \left(\frac{\nu }{2}\right)}{\left(1+\frac{t^2}{\nu }\right)}^{-\frac{\nu +1}{2}}$$

• $\nu$: degrees of freedom
• Mean: 0
• Variance: $\frac{\nu }{\nu -2}$ for $\nu >2$

Example: Used in the t-test for comparing sample means.

Chi-square (${\chi }^2$) distribution

The chi-square distribution is often used in hypothesis testing involving variance and categorical data.

If $Z_1,Z_2,...,Z_k$ are independent standard normal variables, then:

$${\chi }^2=Z_1^2+Z_2^2+\cdots +Z_k^2$$

PDF:

$$f(x;k)=\frac{1}{2^{k/2}\Gamma (k/2)}{x}^{k/2-1}{e}^{-x/2},x>0$$

• Mean: $\mu =k$
• Variance: ${\sigma }^2=2k$

Gamma distribution

The gamma distribution is a continuous distribution with two parameters: shape $\alpha$ and rate $\beta$ (or scale $\theta =1/\beta$).

PDF:

$$f(x;\alpha ,\beta )=\frac{{\beta }^{\alpha }}{\Gamma (\alpha )}{x}^{\alpha -1}{e}^{-\beta x},x>0$$

• Mean: $\mu =\frac{\alpha }{\beta }$
• Variance: ${\sigma }^2=\frac{\alpha }{{\beta }^2}$

Example: Time until the $\alpha$-th event in a Poisson process.

Hypothesis testing

Hypothesis testing is a statistical method for using sample data to make a decision about a population parameter.

Steps in hypothesis testing

1. State the hypotheses:

   • Null Hypothesis: $H_0$ (no effect or no difference)
   • Alternative Hypothesis: $H_1$ (an effect or difference exists)

2. Choose significance level:

   • Denoted by $\alpha$ (commonly 0.05)

3. Compute test statistic:

   • For a population mean with known standard deviation:

$$Z=\frac{\bar{X}-{\mu }_0}{\sigma /\sqrt{n}}$$

4. Determine P-value:

   • P-value: Probability of observing a value as extreme as the test statistic under $H_0$.

5. Make a decision:

   • If $\text{P-value}<\alpha$ → Reject $H_0$
   • If $\text{P-value}\ge \alpha$ → Fail to reject $H_0$

Example

Scenario:

A bottling company claims the mean soda volume is 500 ml. A sample of 36 bottles has:

• Sample mean: $\bar{X}=497$
• Population standard deviation: $\sigma =5$
• Significance level: $\alpha =0.05$

Step 1: Hypotheses

• $H_0:\mu =500$
• $H_1:\mu \ne 500$ (Two-tailed test)

Step 2: Test statistic

Given:

• $\bar{X}=497$, ${\mu }_0=500$, $\sigma =5$, $n=36$

$$Z=\frac{497-500}{5/\sqrt{36}}=\frac{-3}{0.833}\approx -3.60$$

Step 3: P-value

Using Z-tables:

• $\text{P-value}=2\times P(Z<-3.60)$
• $\text{P-value}\approx 2\times 0.00016=0.00032$

Step 4: Decision

Since:

• $0.00032<0.05$ → Reject $H_0$

Conclusion: There is strong evidence that the mean volume is not 500 ml.

Confidence interval (CI)

To construct a 95% CI for the mean:

$$\bar{X}\pm Z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}$$

With:

• $\bar{X}=497$, $Z_{0.025}=1.96$, $\sigma =5$, $n=36$

$$497\pm 1.96\cdot \frac{5}{\sqrt{36}}=497\pm 1.633$$

$$\Rightarrow \text{CI}=[495.37,498.63]$$

Since 500 is not in the interval, we again reject $H_0$.

Type I and Type II errors

Error type	Description	Symbol	Consequence
Type I	Rejecting $H_0$ when it’s true	$\alpha$	False positive
Type II	Failing to reject $H_0$ when it’s false	$\beta$	False negative

Power of the test:

• $\text{Power}=1-\beta$
• It is the probability of correctly rejecting a false $H_0$.

Linear regression and goodness of fit

Linear regression models the relationship between a dependent variable $y$ and an independent variable $x$ using a line plus an error term:

$$y=a+bx+\epsilon$$

Where:

• $y$: dependent variable
• $x$: independent variable
• $a$: intercept of the regression line
• $b$: slope of the regression line
• $\epsilon$: error term

Least squares method

The least squares method chooses the line that minimizes the sum of squared differences between observed values and predicted values.

Formulas for best fit line:

Given data points $(x_1,y_1),(x_2,y_2),\dots ,(x_n,y_n)$, the slope $b$ and intercept $a$ are calculated as:

$$b=\frac{n\sum x_i{y}_i-\sum x_i\sum y_i}{n\sum x_i^2-(\sum x_i{)}^2}$$

$$a=\bar{y}-b\bar{x}$$

Where:

• $\bar{x}=\frac{1}{n}\sum x_i$
• $\bar{y}=\frac{1}{n}\sum y_i$

Example

Given data:

$x$	$y$
1	2
2	3
3	5
4	4
5	6

Step 1: Compute sums

$$\sum x=15,\sum y=20,\sum x^2=55,\sum xy=69,n=5$$

Step 2: Compute slope and intercept

$$b=\frac{5(69)-(15)(20)}{5(55)-(15{)}^2}=\frac{345-300}{275-225}=\frac{45}{50}=0.9$$

$$a=\bar{y}-b\bar{x}=\frac{20}{5}-0.9\cdot \frac{15}{5}=4-0.9\cdot 3=1.3$$

Final regression line:

$$y=1.3+0.9x$$

Goodness of fit

Sample correlation coefficient $R$

The correlation coefficient $R$ measures the strength and direction of the linear relationship:

$$R=\frac{n\sum x_i{y}_i-\sum x_i\sum y_i}{\sqrt{[n\sum x_i^2-(\sum x_i{)}^2][n\sum y_i^2-(\sum y_i{)}^2]}}$$

Given $\sum y^2=90$:

$$R=\frac{5(69)-(15)(20)}{\sqrt{[5(55)-15^2][5(90)-20^2]}}=\frac{45}{\sqrt{2500}}=\frac{45}{50}=0.9$$

Coefficient of determination $R^2$

The coefficient of determination shows the proportion of variance in $y$ explained by the model:

$$R^2=(R{)}^2=(0.9{)}^2=0.81$$

Interpretation: 81% of the variation in $y$ is explained by the linear regression model.

Distribution functions and tables

Please refer to the FE Handbook for detailed distribution functions and tables.