Probability distributions

Random variables & probability distributions

A random variable provides a numerical description of the outcome of a random experiment, and may be thought of as a function that assigns a numerical value to each outcome in a sample space. For example, if $X$ is the random variable associated with the sum of the results of rolling two fair six-sided dice, then the probability that $X=4$ is $\frac{1}{12}$, because of the 36 possible dice rolls, 3 result in a sum of $4$: $1+3$, $2+2$, and $3+1$.

There are two types of random variables:

• Discrete Random Variable: Takes on countable values (finite or countably infinite). For example: the result of dice rolls, the number of children in a household in a town.
• Continuous Random Variable: Takes on uncountably infinite values (any value in an interval). For example: the time between cars arriving at an intersection, the amount of rain expected tomorrow.

Probability distributions provide a mathematical framework for modeling uncertainty by assigning probabilities to possible values of a random variable. Probability distributions for discrete random variables are described in terms of probability mass functions (PMFs), and those for continuous random variables are described in terms of probability density functions (PDFs).

Probability mass functions (PMFs)

For a discrete random variable $X$, the probability mass function (PMF) is:

$$p(x)=P(X=x)$$

PMFs must satisfy the following properties:

• $p(x)\ge 0$ for all $x$
• ${\sum }_{\text{all }x}p(x)=1$

Probability density function (PDFs)

For a continuous random variable $X$, the probability density function (PDF) $f(x)$ satisfies:

$$P(a\le X\le b)={\int }_a^{b}f(x)dx$$

In other words, the area under the curve $f(x)$ between $x=a$ and $x=b$ is the probability that $X$ is between $a$ and $b$.

Note that the probability of obtaining any exact value of $X$ is zero: $P(X=a)=0$ for any specific value $a$.

PMFs must satisfy the following properties:

• $f(x)\ge 0$ for all $x$
• ${\int }_{-\infty }^{\infty }f(x)dx=1$

Cumulative distribution functions (CDF)

The cumulative distribution function (CDF) is the function $F(x)$ that describes the probability of random variable $X$ being at or less than a particular value $x$:

$$F(x)=P(X\le x)$$

For discrete random variables:

$$F(x)=\sum _{t\le x}p(t)$$

For continuous random variables:

$$F(x)={\int }_{-\infty }^{x}f(t)dt$$

Properties of CDFs include:

• $0\le F(x)\le 1$
• $F(x)$ is non-decreasing
• ${\lim }_{x\to -\infty }F(x)=0$
• ${\lim }_{x\to \infty }F(x)=1$

The original PMF or PDF may be recovered from the CDF:

• For discrete $X$ taking on integer values: $p(x)=F(x)-F(x-1)$
• For continuous $X$: $f(x)=\frac{dF(x)}{dx}$

Note that the PMF of discrete random variables $X$ that do not take on integer values may also be recovered from the CDF by subtracting off the CDF of the next discrete value of $X$ just below the target $x$.

Some probability problems are more easily solved using CDFs instead of traditional counting techniques. For example, let’s determine the probability $p(x)$ that highest result in rolling $n$ fair six-sided dice is $x$.

Let $X$ be the random variable the highest result in rolling $n$ fair six-sided dice. Then $p(x)$ is the desired PMF. The CDF $F(x)={\sum }_{k=1}^{k=x}p(x)$ thus represents the probability that the highest roll is $x$ or lower. By the multiplication rule, the probability that the highest roll is $x$ or lower is the product of the probability that each of the $n$ rolls results in $x$ is lower:

$$\begin{array}{rlr}F(x)& =& P(\text{highest roll is x or lower})\\ & =& P(\text{all n rolls are x or lower})\\ & =& P(\text{a particular roll are is x or lower}{)}^n\\ & =& {\left(\frac{x}{6}\right)}^n\end{array}$$

Then, we use the formula $p(x)=F(x)-F(x-1)$ to recover the desired PMF:

$$\begin{array}{rlr}p(x)& =& F(x)-F(x-1)\\ & =& {\left(\frac{x}{6}\right)}^n-{\left(\frac{x-1}{6}\right)}^n\end{array}$$