Law of large numbers

Experimental and theoretical probability

Definitions

Experimental probability

The number of times that the desired event occurs over the total number of trials.

$P(Event)=\frac{\text{Number of times event occurs}}{\text{Total number of trials}}$

Theoretical probability

The number of favorable outcomes over the total number of possible outcomes.

$P(Event)=\frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}}$

Experimental probability

Experimental probability is based on what actually happens when you run trials. It can vary depending on how an experiment goes, especially with a small number of trials. Here’s a simple example:

Example:

If Jane rolls a fair six-sided die four times and a two is rolled exactly once, then the experimental probability of rolling a two is $\frac{1}{4}$.

Theoretical probability

Theoretical probability, by contrast, doesn’t depend on running any trials at all — it’s calculated from the known structure of the situation. As long as all outcomes are equally likely, you can determine the theoretical probability before an experiment even begins:

Example:

The theoretical probability of rolling a two on a six-sided die is $\frac{1}{6}$ because there is one favorable outcome (rolling a two) and six possible outcomes in total (rolling a one, rolling a two, rolling a three, rolling a four, rolling a five, and rolling a six).

Law of large numbers

The law of large numbers says that as the sample size (the number of trials) increases, the experimental probability tends to get closer to the theoretical probability. For instance, if Jane rolls a fair six-sided die only $10$ times, the experimental probabilities for each outcome will usually be farther from the theoretical probabilities than if she rolled the same die $100$ times.

A key point is that the law of large numbers describes what happens in the long run. It does not predict what will happen in the short term. A common mistake students make is assuming that past results influence what’s coming next — but that’s not how independent trials work:

Example:

If Jane flips a fair coin and gets three heads in a row, that does not mean tails is “due” next. The probability of flipping tails on the fourth flip is the same as it was before. This is because the trials are independent.

Gambler’s Fallacy

The gambler’s fallacy is the mistaken belief that short-term results must compensate for previous outcomes in order to match the theoretical probability.

For example, suppose a fair coin lands on heads three times in a row. Some people may believe that tails is now more likely because tails is “due.” However, each coin flip is independent, meaning previous flips do not affect future flips. The probability of getting heads or tails on the next flip remains 50%.

This misconception occurs because people expect random events to look balanced in the short term. In reality, random variation can produce streaks and uneven results over small numbers of trials.

The mistaken belief that a certain outcome is due because of previous results is called the gambler’s fallacy. You’ll want to avoid this kind of reasoning. Even so, if Jane continues flipping the coin many more times, the experimental probabilities of heads and tails will tend to move closer to the theoretical probabilities. This happens because, over many trials, the law of large numbers causes experimental results to better approximate the expected theoretical probabilities.

Important note:

The law of large numbers applies only when the probability stays the same from trial to trial. If the probability changes, the law does not apply. It’s also important not to make definite conclusions from a small number of trials, in general, we can only talk about what is more likely to occur.

The main point is this: although larger numbers of trials tend to make experimental probability closer to theoretical probability on average, exceptions can happen along the way. The law of large numbers is a guideline about the overall trend. It does not say that every additional trial will always make the experimental probability closer than it was on the previous trial.

The two examples below illustrate these limitations. The first shows what happens when the probability changes between draws — a situation where the law of large numbers no longer applies. The second shows how experimental and theoretical probabilities can briefly align before drifting apart again with additional trials.

Example 1:

Suppose Jana has a box of marbles with some red marbles and some green marbles. If the first marble she randomly draws is green and she does not replace it, then the probability that the next marble is green is lower than it was before. There is now one fewer green marble available, so the probability of drawing a red marble next is higher than it was on the first draw. Jana would only be incorrect in making that assumption if the marbles were put back into the box before each subsequent draw.

Example 2:

If Sarah rolls a fair six-sided die $18$ times, it’s possible she could get exactly $3$ ones. In that case, the experimental probability of rolling a one would match the theoretical probability. On the $19$th roll, the probabilities will no longer match exactly, because no matter what she rolls next, the experimental proportion will change.

Practice problem

Now let’s apply these ideas to a more complex situation — one where understanding the relationship between experimental and theoretical probability can actually change the right strategy to take:

Example:

Neesh is playing a game that involves rolling a fair six-sided die a number of times.

Part a.

In the first round, if he can roll a six at least $50\%$ of the time, he wins a prize. He has the choice of either rolling the die twice or twenty times. Which gives him the best chances of winning the prize?

Solution:

Part b.

In the second round, if he can roll a number less than three between $30\%$ and $35\%$ of the time, he wins a prize. He has the choice of either rolling the die sixty times or six hundred times. Which gives him the best chances of winning the prize?

Solution: