Expected value & dispersion

Expected value of a random variable

The expected value $E[X]$ of a random variable $X$ is:

• For discrete $X$:

$$E[X]=\sum _{\text{all }x}x\cdot p(x)$$

• For continuous $X$:

$$E[X]=\mu ={\int }_{-\infty }^{\infty }x\cdot f(x)dx$$

In other words, $E(X)$ is the arithmetic mean (average) of the values of $X$ weighted by their respective probabilities, providing a sense of the “typical” value of $X$.

Properties of the expected value function for random variables $X$ and $Y$ are arbitrary fixed constants $a$ and $b$ include:

• $E[aX+b]=aE[X]+b$
• $E[X+Y]=E[X]+E[Y]$
• $E[XY]=E[X]\cdot E[Y]$ (if $X$ and $Y$ are independent)

Note that in statistics, mathematicians distinguish between the population mean ($\mu$, the mean of the entire target population) and the sample mean ($\bar{x}$, the mean of just a sample of the target population).

Median

Another measure of the “typical” value of random variables is the median, which is the middle value when data are ordered:

• For discrete random variable $X$:
  • For odd $n$: median = value at position $\frac{n+1}{2}$
  • For even $n$: median = average of values at positions $\frac{n}{2}$ and $\frac{n}{2}+1$
• For continuous random variable $X$ with, the median $m$ is the value such that

$${\int }_{-\infty }^{m}f(x)dx={\int }_m^{+\infty }f(x)dx$$

The main advantages of considering the median rather than the expected value (mean) of a distribution is that the median is resistant to outliers and is better representative of typical values for skewed distributions. For example, demographers and economists frequently report the median household income in a society, rather than the mean household income, which may be skewed by a small number of very wealthy families.

Mode

The mode is the most frequently occurring value. A distribution may be unimodal (a single mode), bimodal (two modal values), or multimodal (several modal values).

Dispersion, range, variance & standard deviation

Measures of dispersion describe the spread or variability of data. The three most common are:

• Range
• Variance
• Standard deviation

The range of a data set or a probability distribution is the distance between the lowest value and the highest value:

$$\text{Range}=\text{Maximum}-\text{Minimum}$$

The range is a simple measure of dispersion, but it is highly sensitive to outliers.

In contrast, the variance and standard deviation of a random variable $X$ provide more nuanced measures of how spread out the various values of $X are:

The variance of a random variable is:

$$\text{var}(X)=E[(X-E(X){)}^2]=E[X^2]-(E[X]{)}^2$$

In other words, the variance is the expected value of the square difference between the random variable $X$ and the expected value of $X$.

Properties of variance function for random variables $X$ and $Y$ are arbitrary fixed constants $a$ and $b$ include:

• $\text{var}(aX+b)=a^2\text{Var}(X)$
• $\text{Var}(X+Y)=\text{Var}(X)+\text{Var}(Y)$ (if $X$ and $Y$ are independent)

Note that in statistics, mathematicians distinguish between the population variance (${\sigma }^2$), the variance of the entire target population,

$${\sigma }^2=\frac{1}{N}\sum _{i=1}^N(x_i-\mu {)}^2,$$

and the sample variance ($s^2$), the variance of just a sample of the target population,

$$s^2=\frac{1}{n-1}\sum _{i=1}^n(x_i-\bar{x}{)}^2.$$

Note: We divide by $n-1$ (not $n$) to get an unbiased estimator.

Standard deviation

The standard deviation of a random variable is:

$$\sigma =\sqrt{\text{var}(X)}$$

The standard deviation thus represents the “typical” (root mean square) distance of random variable $X$ from its mean. The standard deviation has the same units as the original data, making it more interpretable than variance.