Markov Chain Stationary Explorer

Transition matrix simulator with k-step evolution and stationary-state intuition.

Worked example — Markov Chain Stationary Explorer

1	0.65985
2	0.218714
3	0.121436
Distance to stationarity	0.08893

These are the values the calculator shows when the page opens. Change any input above to recompute them.

Loading interactive simulation...

Lesson

The theory — Markov Chain Stationary Explorer

A distribution is stationary when one more step changes nothing: πP = π. It is a fixed point of the transition matrix — the chain still moves between states, but the proportions stop shifting.

What each symbol means

P: the transition matrix. Entry in row i, column j is the probability of moving from state i to state j, so every row must sum to 1 — which the grid above checks for you.
p(0): the initial distribution: where the chain starts. The default (1, 0, 0) means it begins in state 1 with certainty.
p(k): the distribution after k steps, obtained by multiplying by P once per step.
π: the stationary distribution — the one that satisfies πP = π.
k: how many steps to take, set above. The trajectory table shows every distribution from p(0) to p(k).

Where the formula comes from

Write what stationarity demands: after one step the distribution is unchanged, so πP = π.
Rearranged, that is π(P − I) = 0 — a system of linear equations, one per state. On its own it has infinitely many solutions, because any multiple of a solution is also one.
Add the condition that makes it a probability distribution, Σπ = 1, and the answer is pinned down. For the default matrix it is exactly 8/13, 3/13, 2/13 — the 0.615385, 0.230769, 0.153846 reported above.

How to read what you see

Three panels. p(k) is where the chain actually is after your k steps; π is where it is heading, with the iteration count and whether it converged; and distance to stationarity measures the gap between them — 0.08893 at the default k = 8, which is why the note says the chain has not fully mixed yet. The trajectory table below shows every step, starting from (1, 0, 0).

Assumes: A finite set of states and probabilities that never change with time. Each row summing to 1 is not a formatting rule but the statement that the chain must go somewhere on every step.
Breaks when: The π above is labelled an estimate for a reason: it is reached by stepping the chain repeatedly — 83 iterations for the default matrix — and stopped when successive distributions stop moving. That is a numerical judgement, not a proof. The exact answer here is the set of rationals 8/13, 3/13, 2/13, and the printed decimals are those rounded to six places.

PageRank is this same calculation, run on the whole web

The stationary distribution is not only a textbook exercise — Google was founded on one. Treat every web page as a state and every link as a transition, and the stationary distribution of that enormous chain says how often a reader clicking links forever would land on each page. That is PageRank. The famous damping factor of 0.85, which sends the surfer to a random page 15% of the time, is not a heuristic tweak either: it exists to guarantee that every state can reach every other, which is exactly the condition that makes the stationary distribution unique and reachable in the first place. Without it, dead-end pages and closed loops would trap the walk and break the answer.

What the long run settles into

Each row of the transition matrix is just a set of odds: given you are in one state, how likely you jump to each other one on the next step. Multiply your current probability vector by P once per step and the numbers drift toward a fixed mix, the stationary distribution π. That mix tells you the fraction of time the system spends in each state over the long haul — try the weather sample and watch p(k) settle.

A unique target it never reaches

Set the two-state matrix to [[0,1],[1,0]] — a coin that always flips. It has a perfectly valid, unique stationary distribution π = (0.5, 0.5), yet start at (1, 0) and p(k) bounces 1,0 → 0,1 → 1,0 forever, never converging. Such periodic chains are exactly why the explorer can report converged: no; guaranteed convergence needs an aperiodic chain, not just a unique π.

Practice

Check yourself

Predict the answer first, then use the controls above to find out. Reveal only after you have committed to a guess — that is what makes it practice.

The default starts the chain in state 1 with certainty, p(0) = (1, 0, 0). Change it to (0, 0, 1) so it starts in state 3 instead. Predict which readouts move and which do not.

Show answer

p(k) and the distance move; π and the iteration count do not. From state 1, eight steps reach 0.659852, 0.218714, 0.121436, a distance of 0.08893. From state 3 they reach 0.522304, 0.255274, 0.222421, a distance of 0.18616 — further out, because state 3 is the rarest destination. Both times π reads 0.615385, 0.230769, 0.153846 after the same 83 iterations. Where the chain starts decides how far along it is, never where it is heading: πP = π mentions the matrix and nothing else.
Back to (1, 0, 0), and now raise the step count. How many steps before the distance to stationarity reads 0 — and has the chain then arrived?

Show answer

k = 80 prints 0, and no, it has not. At k = 40 the distance is already 0.000024; the six decimals the panel prints simply run out before the gap does. This chain approaches π geometrically and never reaches it in finitely many steps, so the 0 is the display rounding — the same reason π itself is labelled an estimate. Compare k = 8, where the distance is 0.08893 and the note still says the chain has not mixed.

Problem solved in full

Eight steps of a three-state chain started in state 1 5 steps
A three-state chain started entirely in state 1, run for eight steps. Work out where it is heading, and how far it still has to go.
1. One step is a vector-matrix product: the chance of being in state j next is the sum over i of being in i now and moving i → j.
  
  $p_{k+1}=p_kP,\qquad p_0=(1,0,0)$
2. Eight steps of that gives the distribution the panel prints. State 1 still holds two thirds of the probability, which is a lot given the chain started there with certainty.
  
  $p_8=(0.65985,\ 0.218714,\ 0.121436)$
3. The destination is the stationary distribution — the one that a step leaves unchanged. Solving πP = π with the probabilities summing to 1 gives exact fractions, eighths and thirteenths, not decimals.
  
  $\pi P=\pi,\ \textstyle\sum\pi_i=1\ \Longrightarrow\ \pi=\left(\tfrac{8}{13},\tfrac{3}{13},\tfrac{2}{13}\right)=(0.615385,\ 0.230769,\ 0.153846)$
4. Comparing the two gives the distance the panel reports, and it is not small: after eight steps the chain is still about 0.089 away in total.
  
  $\|p_8-\pi\|_1=0.04447+0.01206+0.03241=0.08893$
5. That gap shrinks geometrically, governed by the second-largest eigenvalue of P. Geometric decay means it halves in a fixed number of steps and never reaches zero in finitely many.
  
  $\text{the gap shrinks like}\ |\lambda_2|^{k}\ \text{— geometric, never zero}$
Answer

The tool prints 0.65985, 0.218714, 0.121436 after eight steps, still 0.08893 from stationarity. The stationary distribution is exactly (8/13, 3/13, 2/13), and the fact worth keeping is that it does not depend on where you started: those same three fractions are the limit from every starting distribution, which is what makes them a property of the transition matrix rather than of this particular run. That independence is what makes stationary distributions useful: PageRank, MCMC sampling and queue occupancy all rely on the long-run answer being a property of the transition rules alone. Only the time taken to get there depends on the start.

$\pi=\left(\tfrac{8}{13},\tfrac{3}{13},\tfrac{2}{13}\right)\ \text{is where it goes, and}\ p_0\ \text{does not matter}$

References (1)

The stationary distribution put to work on the whole web, damping factor and all: S. Brin & L. Page, “The anatomy of a large-scale hypertextual Web search engine.” Computer Networks and ISDN Systems 30, 107–117, 1998.

By the 7 Raven Room team · Reviewed against published sources. Corrections welcome.