Introduction to probability

Definitions

Probability

Measures how likely an event is to occur. It describes uncertainty using a number between $0$ and $1$:

• A probability of $0$ means the event will not occur.
• A probability of $1$ means the event will certainly occur.

We usually write the probability of an event $A$ as $P(A)$. In many basic situations (when all outcomes are equally likely), you can calculate it with:

$$P(A)=\frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}}$$

Key concepts

Before working through problems, it’s important to understand the vocabulary used to describe probability experiments. These terms appear throughout this topic, so take a moment to review them carefully.

Definitions

Sample space

The set of all possible outcomes from a certain experiment.

Event

One outcome or a subset of outcomes from a probability experiment.

Compound event

An event that consists of two or more simple events.

Mutually exclusive events

Two or more events that cannot happen at the same time.

Independent events

Two or more events where the outcome of one does not impact the outcome of the other(s).

Dependent events

Two or more events where the outcome of one does impact the outcome of the other(s).

Here is what each of these concepts looks like in context.

Sample space

The sample space gives you a complete picture of everything that could possibly happen in an experiment. Identifying it first makes calculating probabilities much easier.

Example:

The sample space for rolling a six-sided die is $\{1,2,3,4,5,6\}$. The sample space for flipping a coin is $\{\text{heads},\text{tails}\}$.

Event

An event is what you’re actually interested in measuring — it’s the specific outcome or group of outcomes you want to find the probability of.

Example:

Rolling higher than a $4$ on a six-sided die is an event. Rolling an even number on a six-sided die is another event.

Compound event

Compound events arise whenever an experiment involves more than one action or step. The probability of a compound event depends on whether those steps are independent or dependent.

Example:

Rolling a $3$ on a die and then flipping heads on a coin is a compound event. Drawing a king from a deck of cards, then rolling an odd number on a die, then drawing a blue marble from an urn is another compound event.

Mutually exclusive events

Recognizing mutually exclusive events matters because it changes how you calculate the probability of either one occurring — there’s no overlap to account for.

Example:

If homeroom starts at 8:00am and ends at 9:00am for every class at a school, then a student can’t be enrolled in two homerooms at once — that would create a time conflict. So the event “a student is in homeroom A” and the event “a student is in homeroom B” are mutually exclusive. Another example: drawing a card that is both red and a spade is impossible, since all spades are black.

Independent and dependent events

Whether events are independent or dependent affects which formula you use to calculate compound probabilities. The key question to ask is: does the first outcome change anything about the second?

Example:

Drawing two marbles from an urn with replacement creates independent events, because the first draw doesn’t change what’s in the urn for the second draw. Drawing two marbles without replacement creates dependent events — if you draw a red marble first and don’t replace it, there is now one fewer red marble available, making it less likely you’ll draw red again.

Axioms of probability

All of probability theory rests on three fundamental axioms. These aren’t things you need to prove — they’re starting assumptions from which everything else follows.

Non-negativity: For any event $A$, the probability of $A$ is non-negative.

$$\begin{array}{rl}P(A)& \ge 0\end{array}$$

This makes sense because even an impossible event has probability $0$, not less than $0$.

Normalization: The probability of the sample space $S$ is equal to $1$.

$$\begin{array}{rl}P(S)& =1\end{array}$$

This is because the sample space includes every possible outcome. Something in the sample space must happen, so the probability that “one of the possible outcomes occurs” is certain, which is $1$.

Additivity: For any two mutually exclusive events $A$ and $B$, the probability of their union is the sum of their individual probabilities.

$$\begin{array}{rl}P(A\cup B)& =P(A)+P(B)\text{if }A\cap B=\varnothing \end{array}$$

These axioms lead to several important properties worth keeping in mind as you work through problems:

Sidenote

Important notes!

$$\begin{array}{rl}P(A\cup B)& =P(A)+P(B)-P(A\cap B)\end{array}$$

This means the probability that at least one of $A$ or $B$ occurs equals the sum of their individual probabilities minus any overlap.

• Two events $A$ and $B$ are independent if and only if:

$$\begin{array}{rl}P(A\cap B)& =P(A)\times P(B)\end{array}$$

• Two events $A$ and $B$ are mutually exclusive if and only if:

$$\begin{array}{rl}P(A\cap B)& =0\end{array}$$

• Two events cannot be mutually exclusive if:

$$\begin{array}{rl}P(A)+P(B)& >1\end{array}$$

If they were mutually exclusive, $P(A\cup B)=P(A)+P(B)$, but $P(A\cup B)$ cannot exceed $1$. So if the sum is greater than $1$, there must be overlap.

• If:

$$\begin{array}{rl}P(A)+P(B)& \le 1\end{array}$$

it is not confirmed whether or not events $A$ and $B$ are mutually exclusive.

Calculating probability

Let’s apply these ideas to real examples, starting with basic probability and then moving to compound events involving independent and dependent outcomes.

Practice problem

Example:

There are $15$ marbles in an urn: $5$ are red, $3$ are blue, and the rest are purple. If Ethan randomly draws a marble, find the probability that he draws each of the following.

Part a.

A red marble

$$\begin{array}{rl}P(\text{Red})& =\frac{\text{Number of red marbles}}{\text{Total number of marbles}}\\ & =\frac{5}{15}\\ & =\frac{1}{3}\end{array}$$

Part b.

A purple marble

Since there are $15$ marbles total and $5$ are red and $3$ are blue, the remaining $7$ must be purple.

$$\begin{array}{rl}P(\text{Purple})& =\frac{\text{Number of purple marbles}}{\text{Total number of marbles}}\\ & =\frac{7}{15}\end{array}$$

Part c.

Any marble

$$\begin{array}{rl}P(\text{Any marble})& =\frac{\text{Total number of marbles}}{\text{Total number of marbles}}\\ & =\frac{15}{15}\\ & =1\end{array}$$

Part d.

A black marble

There are no black marbles in the urn, so this outcome is impossible.

$$\begin{array}{rl}P(\text{Black})& =\frac{\text{Number of black marbles}}{\text{Total number of marbles}}\\ & =\frac{0}{15}\\ & =0\end{array}$$

Notice how the two extreme cases play out: drawing any marble at all gives a probability of $1$ (certain to happen), while drawing a black marble gives $0$ (impossible). For compound events, the formula changes depending on whether the events are independent or dependent.

Probability of compound events:

For independent events:

$$P(A\cap B)=P(A)\times P(B)$$

For dependent events:

$$P(A\cap B)=P(A)\times P(B\mid A)$$

When cards are drawn with replacement, each draw is independent — the deck resets between draws. Let’s practice.

Practice problem

Example:

There are $52$ cards in a standard deck. The suits are Hearts (♥), Diamonds (♦), Clubs (♣), and Spades (♠). The face values are number cards $2$ through $10$, face cards Jack (J), Queen (Q), King (K), and Ace (A). Lucas randomly draws one card, puts it back, then draws again. Find the probability of

Part a.

Drawing aces both times

Since there are $4$ aces in a deck of $52$ cards, the probability of drawing an ace on any single draw is $\frac{4}{52}=\frac{1}{13}$. Because the card is replaced, this probability is the same for both draws.

$$\begin{array}{rl}P(\text{Ace and Ace})& =P(\text{Ace})\times P(\text{Ace})\\ & =\frac{1}{13}\times \frac{1}{13}\\ & =\frac{1}{169}\end{array}$$

Part b.

Drawing a club, then a spade

Each suit contains $13$ of the $52$ cards, so the probability of drawing any specific suit is $\frac{13}{52}=\frac{1}{4}$.

$$\begin{array}{rl}P(\text{Club and Spade})& =P(\text{Club})\times P(\text{Spade})\\ & =\frac{1}{4}\times \frac{1}{4}\\ & =\frac{1}{16}\end{array}$$

Part c.

Drawing a king, then a queen

There are $4$ kings and $4$ queens in the deck, each with probability $\frac{4}{52}=\frac{1}{13}$.

$$\begin{array}{rl}P(\text{King and Queen})& =P(\text{King})\times P(\text{Queen})\\ & =\frac{1}{13}\times \frac{1}{13}\\ & =\frac{1}{169}\end{array}$$

When cards are drawn without replacement, the composition of the deck changes after each draw, making the events dependent. This affects the probability of each subsequent outcome.

Practice problem

Example:

Using the same standard deck of $52$ cards, Lucas now draws one card and throws it away before drawing again. Find the probability of: drawing aces both times, drawing a club then a spade, and drawing a king then a queen,

Part a.

Drawing aces both times

After drawing one ace, only $3$ aces remain in a deck of $51$ cards, so the second probability is lower than the first.

$$\begin{array}{rl}P(\text{Ace and Ace})& =P(\text{Ace on first})\times P(\text{Ace on second}\mid \text{Ace on first})\\ & =\frac{1}{13}\times \frac{3}{51}\\ & =\frac{3}{663}\\ & =\frac{1}{221}\end{array}$$

Part b.

Drawing a club, then a spade

Removing a club doesn’t affect the number of spades, so all $13$ spades remain in the $51$-card deck for the second draw.

$$\begin{array}{rl}P(\text{Club and Spade})& =P(\text{Club on first})\times P(\text{Spade on second}\mid \text{Club on first})\\ & =\frac{1}{4}\times \frac{13}{51}\\ & =\frac{13}{204}\end{array}$$

Part c.

Drawing a king, then a queen

Removing a king doesn’t affect the number of queens, so all $4$ queens remain among the $51$ remaining cards.

$$\begin{array}{rl}P(\text{King and Queen})& =P(\text{King on first})\times P(\text{Queen on second}\mid \text{King on first})\\ & =\frac{1}{13}\times \frac{4}{51}\\ & =\frac{4}{663}\end{array}$$

Another useful rule involves the negation of an event. Since all probabilities in a sample space must sum to $1$, you can find the probability that something does not occur simply by subtracting from $1$:

Since all the probabilities for all the events in the sample space must sum to $1$ (as stated in axiom 2), you can find the probability of a negation by subtracting from $1$:

$$P(A)+P(A^{\prime })=1$$

Where $A$ is an event and $A^{\prime }$ is the negation of the event.

Practice problem

Example:

The probability of drawing a red marble or a blue marble from one particular urn is $34\%$. What is the probability of drawing a marble that is neither blue nor red?

Solution:

Bayes’ theorem

All of the probability rules so far assume you know the full situation upfront. Bayes’ theorem handles something different: what if you start with an initial probability, then receive new evidence, and need to update your estimate?

Bayes’ theorem is used to update a probability when you get new information. You start with an initial probability, observe evidence, and then compute a revised probability:

$$P(A\mid B)=\frac{P(B\mid A)\cdot P(A)}{P(B)}$$

The result is often surprising — even a highly accurate test can have a low probability of being correct, depending on how rare the condition being tested for actually is:

Practice problem

Example:

A disease affects $1\%$ of the population. A test for the disease gives a positive result to $99\%$ of people who actually have the disease, and gives a negative result to $95\%$ of people who do not have the disease. If a person tests positive, what is the probability that they actually have the disease?

Solution: