Inference for Two Proportions

Type the successes and the sample size of each group, then choose the alternative, the confidence level and α. The upper picture is the sampling distribution of p̂₁ − p̂₂ if p₁ = p₂, with the p-value shaded; under it, on the same scale, is the confidence interval for p₁ − p₂.

Standard deviation of p̂₁ − p̂₂	$\sigma_{\hat p_1-\hat p_2}=\sqrt{\frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2}}$
Confidence interval	$(\hat p_1-\hat p_2)\pm z^*\sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1}+\frac{\hat p_2(1-\hat p_2)}{n_2}}$
Test statistic, pooled	$z=\frac{\hat p_1-\hat p_2}{\sqrt{\hat p_c(1-\hat p_c)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}},\quad\hat p_c=\frac{x_1+x_2}{n_1+n_2}$

Worked example — Inference for Two Proportions

Sample proportion p̂₁	0.18
Sample proportion p̂₂	0.1
Difference p̂₁ − p̂₂	0.08
Pooled proportion p̂c	0.116
Test statistic z	2.23
p-value	0.0255
Fisher’s exact p-value	0.0464
Critical value z* for 95%	1.96

These are the values the calculator shows when the page opens. Change any input above to recompute them.

Loading interactive simulation...

Lesson

The theory — Inference for Two Proportions

Inference for two proportions compares p₁ and p₂, the success rates of two populations or two treatments, through the difference p̂₁ − p̂₂: a confidence interval estimates p₁ − p₂, and a significance test judges H₀: p₁ = p₂.

Where the formula comes from

The sampling distribution of p̂₁ − p̂₂. Over repeated independent samples, p̂₁ − p̂₂ has mean p₁ − p₂. Because the samples are independent, their variances add, so the standard deviations combine under a square root and never by plain addition. The shape is close to normal when n₁p₁, n₁(1 − p₁), n₂p₂ and n₂(1 − p₂) are all at least 10.
$\mu_{\hat p_1-\hat p_2}=p_1-p_2,\qquad\sigma^2_{\hat p_1-\hat p_2}=\frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2}$
Check the conditions. Random: two independent random samples, or two groups formed by random assignment in an experiment. 10%: when sampling without replacement, each sample is at most a tenth of its population; with random assignment there is no population to sample from and the condition does not apply. Large counts: for an interval, at least 10 successes and 10 failures in each group; for a test, the four expected counts from the pooled proportion.
$\begin{aligned}&n_1\le0.1\,N_1,\quad n_2\le0.1\,N_2\\&x_i\ge10,\quad n_i-x_i\ge10\\&n_i\hat p_c\ge10,\quad n_i(1-\hat p_c)\ge10\end{aligned}$
The interval. (p̂₁ − p̂₂) ± z* times the standard error, with each group’s own p̂ inside it, because the interval assumes nothing about p₁ and p₂. If it contains 0, no difference is a plausible value; if it lies wholly above or below 0, it is convincing evidence of a difference in that direction.
$(\hat p_1-\hat p_2)\pm z^*\,SE,\qquad z^*=1.645,\ 1.96,\ 2.576$
The test. State H₀: p₁ = p₂ and the alternative. If H₀ is true the two samples estimate one proportion, so the test pools them into p̂c and computes the standard deviation of p̂₁ − p̂₂ from p̂c alone. z is the observed difference measured in that standard deviation, and the p-value is the chance, if H₀ is true, of a difference at least that far in the direction of Hₐ.
$H_0:\ p_1=p_2\ \Rightarrow\ \hat p_c=\frac{x_1+x_2}{n_1+n_2},\qquad\sigma_0=\sqrt{\hat p_c(1-\hat p_c)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}$
Conclude, and say what the design allows. Compare the p-value with α and conclude in context, as for one proportion. Then the scope: random assignment lets you conclude that the treatment caused the difference, and random sampling lets you generalise to the populations sampled. Two random samples in an observational study can show that two populations differ; they cannot show why.

Assumes: The normal model approximates two binomial counts. When large counts fail, the page prints Fisher’s exact p-value: it takes the total number of successes as fixed and asks how often a random split of the n₁ + n₂ subjects would put at least x₁ of them in group 1. Its two-sided version doubles the smaller tail.

Problems solved in full

The sampling distribution of p̂₁ − p̂₂ 3 steps
In a large town, 30% of adults and 20% of teenagers say they read a newspaper. Independent random samples of 100 adults and 150 teenagers are taken. Describe the sampling distribution of p̂₁ − p̂₂, and find how likely it is that the adults’ sample proportion comes out smaller than the teenagers’.

Open this problem in the tool
1. Its centre and spread follow from the two proportions and the two sample sizes:
  
  $\begin{aligned}\mu_{\hat p_1-\hat p_2}&=0.3-0.2=0.1\\\sigma_{\hat p_1-\hat p_2}&=\sqrt{\frac{0.3\,(0.7)}{100}+\frac{0.2\,(0.8)}{150}}\\&\approx0.0563\end{aligned}$
2. Its shape is approximately normal, because all four expected counts reach 10:
  
  $\begin{aligned}&100\,(0.3)=30,\quad100\,(0.7)=70\\&150\,(0.2)=30,\quad150\,(0.8)=120\end{aligned}$
3. The adults’ proportion is the smaller one when p̂₁ − p̂₂ falls below 0:
  
  $\begin{aligned}P(\hat p_1-\hat p_2<0)&=P\!\left(Z<\frac{0-0.1}{0.0563}\right)\\&=P(Z<-1.78)\approx0.038\end{aligned}$
Answer

Approximately normal, with mean 0.1 and standard deviation 0.0563. The chance that the adults’ sample proportion is the smaller is about 0.038.

$\begin{aligned}&\hat p_1-\hat p_2\ \dot\sim\ N(0.1,\ 0.0563)\\&P(\hat p_1<\hat p_2)\approx0.038\end{aligned}$
Constructing a confidence interval 3 steps
In a random sample of 300 adults in one city, 168 back a ban on cars in the city centre; in a random sample of 350 adults in a second city, 140 do. Construct and interpret a 95% confidence interval for the difference between the two cities’ proportions.

Open this problem in the tool
Show solution
1. Conditions: two independent random samples; a city has far more than the 3,000 and 3,500 adults that would make each sample a tenth of its population; 168, 132, 140 and 210 are all at least 10. The two proportions:
  
  $\hat p_1=\frac{168}{300}=0.56,\qquad\hat p_2=\frac{140}{350}=0.4$
2. The standard error, with each city’s own proportion:
  
  $\begin{aligned}SE&=\sqrt{\frac{0.56\,(0.44)}{300}+\frac{0.4\,(0.6)}{350}}\\&\approx0.0388\end{aligned}$
3. Multiply by z* = 1.96 and add and subtract:
  
  $0.16\pm1.96\,(0.0388)=0.16\pm0.076$
Answer

(0.084, 0.236). We are 95% confident that the proportion of adults who back the ban is between 8.4 and 23.6 percentage points higher in the first city than in the second.

$(0.084,\ 0.236)$
Using an interval to judge a claim 2 steps
In random samples of customers, 45 of 150 at store A and 52 of 200 at store B returned an item. The manager claims the two stores’ return rates differ. Use a 95% confidence interval to judge the claim.

Open this problem in the tool
Show solution
1. The difference and its standard error:
  
  $\begin{aligned}\hat p_1-\hat p_2&=0.3-0.26=0.04\\SE&=\sqrt{\frac{0.3\,(0.7)}{150}+\frac{0.26\,(0.74)}{200}}\\&\approx0.0486\end{aligned}$
2. The interval, and where 0 lies:
  
  $\begin{aligned}0.04\pm1.96\,(0.0486)&=0.04\pm0.095\\&=(-0.055,\ 0.135)\end{aligned}$
Answer

The interval (−0.055, 0.135) contains 0, so no difference is a plausible value: the data do not give convincing evidence that the return rates differ.

$0\in(-0.055,\ 0.135)$
A two-sided test in an experiment 3 steps
200 volunteers were split at random into two groups of 100. One group was sent a text reminder; 62 of them got a flu shot, against 45 of the other group. Test at the 5% level whether the reminder makes a difference, and say what the conclusion allows and which error it risks.

Open this problem in the tool
Show solution
1. Hypotheses: H₀: p₁ = p₂, Hₐ: p₁ ≠ p₂. Conditions: random assignment; the expected counts 53.5 and 46.5 in each group are at least 10. If H₀ is true the groups share one proportion:
  
  $\begin{aligned}&\hat p_c=\frac{62+45}{100+100}=0.535\\&100\,(0.535)=53.5\\&100\,(0.465)=46.5\end{aligned}$
2. The test statistic, with the standard deviation from the pooled proportion:
  
  $\begin{aligned}z&=\frac{0.62-0.45}{\sqrt{0.535\,(0.465)\left(\frac{1}{100}+\frac{1}{100}\right)}}\\&=\frac{0.17}{0.0705}\approx2.41\end{aligned}$
3. The two-sided p-value:
  
  $P=2\,P(Z\ge2.41)\approx0.016$
Answer

p = 0.016 < 0.05, so reject H₀: there is convincing evidence that the reminder changes the proportion who get a flu shot. The groups were formed by random assignment, so the reminder caused the difference; they were volunteers, so the conclusion reaches people like them. If the reminder in fact makes no difference, this conclusion is a Type I error.

$P\approx0.016<0.05$
A one-sided test on observational data 3 steps
A school district has 2,600 students who play a musical instrument and 7,400 who do not. Random samples of 200 players and 180 non-players are taken. 26 of the first group and 18 of the second are on the honour roll. Is there convincing evidence, at α = 0.05, that a higher proportion of instrument players are on the honour roll?

Open this problem in the tool
Show solution
1. Hypotheses: H₀: p₁ = p₂, Hₐ: p₁ > p₂. Conditions: two independent random samples; 200 is less than a tenth of 2,600 and 180 less than a tenth of 7,400; expected counts 23.2, 176.8, 20.8 and 159.2. The pooled proportion:
  
  $\hat p_c=\frac{26+18}{200+180}\approx0.1158$
2. The test statistic:
  
  $\begin{aligned}z&=\frac{0.13-0.1}{\sqrt{0.1158\,(0.8842)\left(\frac{1}{200}+\frac{1}{180}\right)}}\\&=\frac{0.03}{0.0329}\approx0.91\end{aligned}$
3. The one-sided p-value:
  
  $P=P(Z\ge0.91)\approx0.181$
Answer

p = 0.181 ≥ 0.05, so fail to reject H₀: there is not convincing evidence that instrument players are more often on the honour roll. Even a small p-value would not have shown that playing an instrument raises grades: these are two samples observed, not groups assigned.

$P\approx0.181\ge0.05$
When large counts fail: Fisher’s exact test 2 steps
In a pilot study, 20 patients were assigned at random, 10 to a new drug and 10 to a placebo. 5 on the drug improved and 1 on the placebo did. Is the z-test appropriate? Find the exact p-value for Hₐ: p₁ > p₂.

Open this problem in the tool
Show solution
1. Large counts for the test fail:
  
  $\hat p_c=\frac{5+1}{20}=0.3,\qquad10\,(0.3)=3<10$
2. If the drug does nothing, the 6 patients who improved would have improved anyway, and the random assignment alone decided how many of them landed in the drug group. The chance that it put 5 or 6 of them there:
  
  $\begin{aligned}P(X_1\ge5)&=\frac{\binom{10}{5}\binom{10}{1}+\binom{10}{6}\binom{10}{0}}{\binom{20}{6}}\\&=\frac{2520+210}{38760}\approx0.070\end{aligned}$
Answer

No: each group’s expected counts are 3 and 7. The exact p-value is 0.070, against 0.025 from the normal curve, so the pilot is not convincing evidence at the 5% level.

$P_{\mathrm{exact}}\approx0.070,\quad P_{\mathrm{normal}}\approx0.025$

Supplementary problems, answers only

For the two cities’ 168 of 300 and 140 of 350, find a 90% confidence interval for the difference in proportions.

Show answer

(0.096, 0.224)

$\begin{aligned}0.16\pm1.645\,(0.0388)&=0.16\pm0.064\\&=(0.096,\ 0.224)\end{aligned}$
Redo the flu-shot test, 62 of 100 against 45 of 100, at α = 0.01. What is the decision, and which error could it be?

Show answer

p = 0.016 ≥ 0.01: fail to reject H₀. If the reminder really does make a difference, this is a Type II error.

$P\approx0.016\ge0.01$
Why does the test pool the two samples while the interval does not?

Show answer

The test is computed as if H₀ were true, and under H₀ both samples estimate the same proportion, so combining them gives the best estimate of it. The interval estimates p₁ − p₂ without assuming the two are equal, so each group keeps its own p̂.
Using z ≈ (a − b)/√(a + b), estimate z for the vaccine trial’s 8 cases against 162.

Show answer

z ≈ −11.8

$z\approx\frac{8-162}{\sqrt{170}}\approx-11.8$
A 95% confidence interval for p₁ − p₂ is (−0.12, −0.03). What does it say about the two proportions?

Show answer

0 is not in it, so there is convincing evidence that p₁ is smaller than p₂, by somewhere between 3 and 12 percentage points.

$0\notin(-0.12,\ -0.03)$
Random samples of 400 smokers and 400 non-smokers show a much higher proportion of coughs among smokers, with p < 0.0001. Does the study show that smoking causes coughing?

Show answer

Not on its own: it is an observational study, so it shows that the two populations differ, and a lurking variable could account for the difference. Only random assignment would let it show cause, and assigning people to smoke would be unethical.
For 40 of 100 against 27 of 100, the chi-square statistic of the 2 × 2 table is 3.793. Find the z-statistic of the two-proportion test without its formula.

Show answer

z = √3.793 ≈ 1.95, positive because p̂₁ = 0.4 is the larger proportion

$z=\sqrt{3.793}\approx1.95$

With equal groups, only the interval can be the bolder one

Press 18 of 100 against 40 of 400. The test rejects p₁ = p₂ with p = 0.0255, yet the 95% interval for p₁ − p₂, −0.0008 to 0.1608, contains 0. Now press 40 of 100 against 27 of 100: the interval, 0.0004 to 0.2596, leaves 0 out, and the test keeps H₀ with p = 0.0515. Fisher’s exact test sides with the z-test both times (0.0464 and 0.0718), so the normal curve is not what fails here. The two procedures measure the same difference with different standard errors. The test assumes H₀, so it pools both samples into one proportion, 0.116 in the first case; the interval assumes nothing and lets each group keep its own. In the first case the small group’s 0.18 carries most of the unpooled variance, and pooling replaces it with a proportion pulled down by the 400. When the two groups are the same size, the pooled variance exceeds the unpooled one by exactly (p̂₁ − p̂₂)²/(2n), so the test’s ruler is never the shorter: the interval can leave 0 out while the test keeps H₀, and the opposite needs groups of different sizes.

z² is the chi-square statistic of the same table

Write the counts of 18 of 100 against 40 of 400 as a 2 × 2 table of successes and failures: 18 and 82, 40 and 360. Its chi-square statistic, the sum of (O − E)²/E without a continuity correction, is 4.993, and so is z² (z is 2.2345 before rounding). That holds for every table, and only because the test pools: a chi-square test builds its expected counts from the overall success rate, which is p̂c. The z of the interval’s standard error, 0.08/0.0412 = 1.94, squares to 3.76 and matches nothing. The p-values agree as well, 0.0255 both ways, since a chi-square variable with one degree of freedom is the square of a standard normal one. Squaring throws the sign away, which is why the chi-square test of a 2 × 2 table can only answer the two-sided question. Type the four counts into the chi-square page as a table with 2 rows and 2 columns and it prints the same 4.993.

The vaccine trial’s evidence was 170 cases, not 43,448 people

Press Vaccine trial, 2020. In the trial of the Pfizer–BioNTech vaccine, 21,720 people received the vaccine and 21,728 a placebo, and the investigators counted 8 cases of Covid-19 in the first group and 162 in the second (Polack and colleagues, 2020). The difference in proportions is under one percentage point, and z = −11.83. With two groups of equal size, as these are near enough, the test statistic can be rewritten exactly as z = (a − b)/√((a + b)(1 − p̂c)), where a and b are the two counts of cases, and for a rare outcome the last factor is close to 1: z ≈ (8 − 162)/√170 = −11.8. The number of people has dropped out. Ten times as many volunteers with the same 8 and 162 cases would give almost the same z; the same volunteers with 85 cases in each group would give z = 0. (The paper counted its cases among the participants with no sign of an earlier infection, fewer than these totals, and for the same reason z hardly moves.) A trial of a rare outcome needs tens of thousands of people because that is how many it takes to see enough cases.

Learning path

Statistics: from data to proportions

References (2)

Insight block 3 and the vaccine preset: 21,720 people received the BNT162b2 vaccine and 21,728 a placebo, and 8 cases of Covid-19 were counted in the first group against 162 in the second, at least 7 days after the second dose: Polack, F. P., Thomas, S. J., Kitchin, N. et al. (2020). Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. New England Journal of Medicine, 383(27), 2603–2615.
Fisher's exact test, which the page prints beside the z-test: it conditions on both margins of the 2 × 2 table, so the count in one cell is hypergeometric, and a large-sample approximation is unnecessary when the exact distribution can be computed: Agresti, A. (1992). A survey of exact inference for contingency tables. Statistical Science, 7(1), 131–153.

By the 7 Raven Room team · Reviewed against published sources. Corrections welcome.

Inference for Two Proportions

Where the formula comes from

Supplementary problems, answers only

With equal groups, only the interval can be the bolder one 🖖

z² is the chi-square statistic of the same table 🖖

The vaccine trial’s evidence was 170 cases, not 43,448 people 🖖

With equal groups, only the interval can be the bolder one

z² is the chi-square statistic of the same table

The vaccine trial’s evidence was 170 cases, not 43,448 people