PMF, PDF, and CDF: three views of one distribution

A random quantity can be described by a list of probabilities, by a density curve, or by a running total. How do the three fit together?

The running total is the cumulative distribution function, F(x) = P(X ≤ x): all the probability at or below x. A list of probabilities adds up into it, and a density curve gives it as an area. Stage 1 asks you to draw that running total from the curve.

Before you start · 3 quick checks on areas under graphs, how fast an area grows, and adding probabilities, about 4 min

Definitions

Random variable. A random variable \(X\) is a number determined by the outcome of a random experiment: the number of heads in three tosses, a waiting time, the loss on a loan. For a number \(x\), \(\{X \le x\}\) is the event that the outcome gives a value of at most \(x\). The lesson on conditional probability and the Monty Hall problem worked with events; a random variable turns outcomes into numbers.

PMF. \(X\) is discrete when its possible values can be listed, \(x_1, x_2, \dots\). Its probability mass function is

\[p(x) = P(X = x).\]

Each \(p(x)\) is at least 0, and the values \(p(x_1), p(x_2), \dots\) add up to 1.

PDF. \(X\) is continuous when it has a probability density function: a function \(f \ge 0\) with total area 1 under its graph, such that for all \(a \le b\)

\[P(a \le X \le b) = \int_a^b f(x)\,dx.\]

On this page, continuous means exactly this: \(X\) has a density. (A few random variables have a CDF with no jumps and still no density; none appears here.)

CDF. Every random variable has a cumulative distribution function

\[F(x) = P(X \le x).\]

For a discrete \(X\), \(F(x)\) is the sum of \(p(x_i)\) over the values \(x_i \le x\). For a continuous \(X\), \(F(x)\) is the area under the density to the left of \(x\):

\[F(x) = \int_{-\infty}^{x} f(t)\,dt.\]

Fine print: the rules of probability used here, and variables that are neither kind

Every event has a probability between 0 and 1, the event containing every possible outcome has probability 1, and the probabilities of events that share no outcome add up. That last rule, additivity, also covers countably many events, no two of which share an outcome.

Some random variables are neither discrete nor continuous. The loss on a loan might be 0 with probability 0.9 and otherwise spread over a range of amounts. There is no list of values that carries all the probability and no density, but the CDF still exists, and the result proved in stage 3 holds for such variables too.

1 See

Sketch the CDF from the density

Choose a distribution. The upper figure shows its density, or for the discrete one its PMF as bars. In the lower figure, drag each of the nine handles up or down to where you think \(F(x) = P(X \le x)\) is at that \(x\). You can also focus a handle and use the arrow keys (steps of 0.01), Page Up and Page Down (0.1), Home (0) and End (1). Then press Check my sketch. A handle counts as close when it is within 0.05 of the actual value.

2 Try

Points, heights and areas

Goals: read probabilities as rises of F

The explorer below has the same distributions, with the actual CDF drawn. Drag \(a\) and \(b\) on the axis of the upper figure; the readout gives the shaded area and \(F(b) - F(a)\), which is \(P(a \lt X \le b)\).

3 Prove

Every CDF is nondecreasing

Theorem

Assumptions. \(X\) is any random variable, discrete, continuous or neither, with CDF \(F(x) = P(X \le x)\). The proof uses two rules of probability: a probability is never negative, and if two events have no outcome in common, the probability of their union is the sum of their probabilities (additivity).

Conclusion. For all real numbers \(a \le b\),

\[P(a \lt X \le b) = F(b) - F(a).\]

In particular \(F(a) \le F(b)\) whenever \(a \le b\): F is nondecreasing.

What else every CDF does

\(F(x)\) tends to 0 as \(x\) decreases without bound, and to 1 as \(x\) increases without bound.
F is right-continuous: \(F(x + h)\) tends to \(F(x)\) as \(h\) shrinks to 0 through positive values.
From the left, \(F(x - h)\) tends to \(P(X \lt x)\), written \(F(x^-)\). So \(P(X = x) = F(x) - F(x^-)\) is the size of the jump of F at \(x\). For a continuous \(X\), F has no jumps, every single value has probability 0, and \(P(a \le X \le b) = P(a \lt X \lt b) = F(b) - F(a)\).
Where the density \(f\) is continuous, \(F'(x) = f(x)\), by the fundamental theorem of calculus (the second check in Before you start).
A median is a number \(m\) with \(P(X \le m) \ge \tfrac{1}{2}\) and \(P(X \ge m) \ge \tfrac{1}{2}\). When F is continuous, every \(m\) with \(F(m) = \tfrac{1}{2}\) is a median. For the exponential distribution with rate 0.5, \(1 - e^{-0.5m} = \tfrac{1}{2}\) gives \(m = 2\ln 2 \approx 1.386\).

Where the first three facts come from

They follow from one more rule of probability that the proof above does not need: for events that grow, \(E_1 \subseteq E_2 \subseteq \dots\), the probability of their union is the limit of \(P(E_n)\), and for events that shrink, the probability of their intersection is the limit. For example, the events \(\{X \le x + \tfrac{1}{n}\}\) shrink to \(\{X \le x\}\) as \(n\) grows, which gives right-continuity. The reading list below has the details.

4 Apply

Problems

Each problem has hints and a full solution.