Common probability distributions

Many probability distributions capture various real-world patterns, from equally likely outcomes to natural variation around an average or the frequency of rare events. Understanding these distributions helps analysts choose appropriate models, interpret data correctly, and make informed predictions across fields such as science, economics, engineering, and social research.

The most common probability distributions are described below, where functions of the form $p_X(k)$ represents the probability mass function (PMF) of a discrete random variable $X$, and functions of the form $f_X(x)$ represents the probability density function (PDF) of a continuous random variable $X$. Some are described in terms of a collection of independent and identically distributed (IID) random variables, where each random variable has the same probability distribution as the others and all are mutually independent.

• Uniform (discrete)
• Uniform (continuous)
• Bernoulli
• Binomial
• Poisson
• Geometric
• Exponential
• Normal (Gaussian, $Z$)
• Pascal
• Erlang
• Hypergeometric

Uniform (discrete)

The uniform (discrete) distribution models a scenario where the integers from $a$ to $b$ are all equally likely to occur.

The PMF is:

$$\begin{array}{rcl}{p}_X(k)& =& \frac{1}{b-a+1},\\ & & \text{for }k=a,a+1,\dots ,b\end{array}$$

The expected value is:

$$\frac{a+b}{2}$$

The variance is:

$$\frac{(b-a)(b-a+2)}{12}$$

Uniform (continuous)

The uniform (continuous) distribution models a scenario where the real numbers from $a$ to $b$ are all equally likely to occur.

The PDF is:

$$\begin{array}{rlr}{f}_X(x)& =& \frac{1}{b-a},\\ & & \mathrm{for}a\le x\le b\end{array}$$

The expected value is:

$$\frac{a+b}{2}$$

The variance is:

$$\frac{(b-a{)}^2}{12}$$

Bernoulli

The Bernoulli distribution models the number of successes $k$ (either $0$ or $1$) coming up in a single flip of a biased coin, where $p$ is the probability of the coin flipping a “success” (e.g., heads).

The PMF is:

$$px(k)=\{\begin{array}{ll}p,& \text{if }k=1\\ & \\ 1-p,& \text{if }k=0\end{array}$$

The expected value is:

$$p$$

The variance is:

$$p(1-p)$$

Binomial

The binomial distribution models the number of successes $k$ in $n$ flips of a biased coin, where $p$ is the probability of the coin flipping a “success” (e.g., heads). The binomial distribution is the sum of $n$ IID Bernoulli RVs, and may be considered the “converse” of the Pascal distribution (see below) in the sense that the binomial distribution fixes the number of flips and models the number of successes, whereas the Pascal distribution fixes the number of successes and models the number of flips.

The PMF is:

$$\begin{array}{rlr}{p}_X(k)& =& \left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k},\\ & & \mathrm{for}k=0,1,...,n\end{array}$$

The expected value is:

$$np$$

The variance is:

$$np(1-p)$$

Poisson

The Poisson distribution models the number of events $k$ in a unit time interval of a Poisson process. The Poisson distribution approximates the binomial when $n$ is large and $p$ is small, so $\lambda =np$. The Poisson distribution may be considered the “converse” of the Erlang distribution (see below) in the sense that the Poisson distribution fixes the length of the time interval and models the number of events, whereas the Erlang distribution fixes the number of events and models the length of the time interval.

The PMF is:

$$\begin{array}{rlr}{p}_X(k)& =& e^{-\lambda }\frac{{\lambda }^k}{k!},\\ & & \mathrm{for}k=0,1,...\end{array}$$

The expected value is:

$$\lambda$$

The variance is:

$$\lambda$$

Geometric

The geometric distribution models the number of tries $k$ until the first success in a sequence of independent flips of a biased coin, where $p$ is the probability of the coin flipping a “success” (e.g., heads). The geometric distribution is equivalent to a Pascal distribution (see below) for just a single success.

The PMF is:

$$\begin{array}{rlr}{p}_X(k)& =& (1-p{)}^{k-1}p,\\ & & \mathrm{for}k=1,2,...\end{array}$$

The expected value is:

$$\frac{1}{p}$$

The variance is:

$$\frac{1-p}{p^2}$$

Exponential

The exponential distribution models the wait time $x$ until the first event in a Poisson process. The exponential distribution is equivalent to an Erlang distribution for just a single success, and it approximates the geometric distribution when $p$ is very small, so $p=1-e^{-\lambda }$, $\lambda =-\ln (1-p)$.

The PDF is:

$$\begin{array}{rlr}{f}_X(x)& =& \lambda e^{-\lambda x},\\ & & \mathrm{for}x\ge 0\end{array}$$

The expected value is:

$$\frac{1}{\lambda }$$

The variance is:

$$\frac{1}{{\lambda }^2}$$

Normal (Gaussian, $Z$)

The normal (Gaussian, $Z$) distribution, the so-called “bell curve,” models many real-life scenarios and is one of the most important probability distributions. The mean of nearly every arbitrary IID random variable approaches the normal distribution due to the central limit theorem.

The PDF is:

$$\begin{array}{rlr}{f}_X(x)& =& \frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2{\sigma }^2},\\ & & \mathrm{for}\mathrm{all}\mathrm{real}x\end{array}$$

The expected value is:

$$\mu$$

The variance is:

$${\sigma }^2$$

Pascal

The Pascal (or “negative binomial”) distribution models the number of tries $t$ until the $k$th success in a sequence of independent flips of a biased coin, where $p$ is the probability of the coin flipping a “success” (e.g., heads). $Y_k$ is the sum of $k$ IID geometric RVs. The Pascal distribution may be considered the “converse” of the binomial distribution in the sense that the binomial distribution fixes the number of flips and models the number of successes, whereas the Pascal distribution fixes the number of successes and models the number of flips.

The PMF is:

$$\begin{array}{rlr}{p}_{Y_k}(t)& =& \left(\genfrac{}{}{0ex}{}{t-1}{k-1}\right)p^k(1-p{)}^{t-k},\\ & & \mathrm{for}t=k,k+1,...\end{array}$$

The expected value is:

$$\frac{k}{p}$$

The variance is:

$$\frac{k(1-p)}{p^2}$$

Erlang

The Erlang distribution models the wait time $y$ until the $k$th success in a Poisson process. $Y_k$ is the sum of $k$ IID exponential random variables. The Erlang distribution approximates the Pascal distribution when $p$ is very small, and may be considered the “converse” of the Poisson distribution in the sense that the Poisson distribution fixes the length of the time interval and models the number of events, whereas the Erlang distribution fixes the number of events and models the length of the time interval.

The PDF is:

$$\begin{array}{rlr}{f}_{Y_k}(y)& =& \frac{{\lambda }^k{y}^{k-1}{e}^{-\lambda y}}{(k-1)!},\\ & & \mathrm{for}y\ge 0\end{array}$$

The expected value is:

$$\frac{k}{\lambda }$$

The variance is:

$$\frac{k}{{\lambda }^2}$$

Hypergeometric

The hypergeometric distribution models the scenario of an urn that contains $N$ marbles, of which $K$ are green and $N-K$ are red. A total of $n$ marbles are drawn without replacement. $X$ is the random variable whose outcome is $k$, the number of green marbles drawn. The hypergeometric distribution reduces to the binomial distribution when $n\ll K$ and $n\ll N$, where $p=K/N$.

The PMF is:

$$\begin{array}{rlr}{p}_X(k)& =& \frac{\left(\genfrac{}{}{0ex}{}{K}{k}\right)\left(\genfrac{}{}{0ex}{}{N-K}{n-k}\right)}{\left(\genfrac{}{}{0ex}{}{N}{n}\right)}\end{array}$$

The expected value is:

$$n\frac{K}{N}$$

The variance is:

$$n\frac{K}{N}\frac{N-K}{N}\frac{N-n}{N-1}$$