Chi-Square Tests: Testing Categories

So far in this module, we have been testing numbers. T-Tests and ANOVA are designed for continuous, quantitative data—things you can measure, like test scores, weight loss, or revenue.

But what happens when your data isn’t a number at all?

What if you are looking at categories?

• Do Gen Z customers prefer the Dark Mode or Light Mode app theme compared to Millennials?
• Is there a relationship between a user’s Region (North America, Europe, Asia) and the Subscription Tier they choose (Basic, Pro, Enterprise)?

You cannot calculate the “average” of a color, a region, or a subscription tier. Because there are no means, T-Tests and ANOVA are completely useless here.

To test categorical data, we use the Chi-Square Test (pronounced Kai-Square, like the Greek letter $\chi$).

The Core Concept: Observed vs. Expected

The magic of the Chi-Square test is incredibly simple. It compares two things: What actually happened versus What should have happened if everything was completely random.

Imagine you run an A/B test on a marketing email. You send out two subject lines.

• Subject A: “Sale ends tonight!”
• Subject B: “Your exclusive discount inside.”

You get 1,000 total opens.

If the subject line made absolutely no difference (our Null Hypothesis), you would expect the opens to be split perfectly 50/50. You Expected 500 opens for A and 500 opens for B.

But when you look at your actual data, you see the Observed reality:

• Subject A got 600 opens.
• Subject B got 400 opens.

The Chi-Square test simply measures the gap between the Expected (500/500) and the Observed (600/400).

The Chi-Square Statistic ($\chi^2$)

To find out if that gap is statistically significant, we calculate the Chi-Square statistic using this formula:

$$\chi^2 = \sum \frac{(\text{Observed} – \text{Expected})^2}{\text{Expected}}$$

Let’s translate that into plain English:

1. Find the difference between what happened and what you expected ($O – E$).
2. Square that difference (so negative numbers don’t cancel out positive numbers).
3. Divide by the expected number (to scale it appropriately).
4. Add them all up for every category ($\sum$).

• If your Observed numbers are exactly the same as your Expected numbers, your $\chi^2$ score is 0. You Fail to Reject the Null Hypothesis. Nothing interesting happened.
• If the gap between Observed and Expected is massive, your $\chi^2$ score explodes. A high score gives you a tiny P-Value, allowing you to Reject the Null Hypothesis. The categories are mathematically related!

Chi-Square Statistic Visualiser

Use this interactive Chi-Square simulator to test the relationship between two categories. Watch what happens when you adjust the “Observed” sliders to pull the actual data further and further away from the flat, “Expected” baseline. The bigger the gap, the bigger the $\chi^2$ score.

How to Run a Chi-Square Test in Software

When you have a massive table of categorical data (like 5 regions cross-referenced with 4 product types), calculating the expected values by hand takes forever. Let the software do it.

(Assume you have a 2×2 contingency table of counts, like [ [600, 400], [450, 550] ]).

Microsoft Excel

Excel handles this with the CHISQ.TEST function, but it requires you to manually calculate the Expected table first.

1. Build your table of Observed counts (e.g., A1:B2).
2. Build a second table next to it calculating your Expected counts based on the row and column totals (e.g., D1:E2).
3. Formula: =CHISQ.TEST(A1:B2, D1:E2)
4. The output is your exact P-Value.

Python

Python’s scipy.stats library calculates the expected values, the chi-square statistic, and the p-value all at once using the chi2_contingency function.

from scipy.stats import chi2_contingency

# A 2x2 table of observed counts
# e.g., [[Theme Light: GenZ, Millennial], [Theme Dark: GenZ, Millennial]]
observed_data = [[120, 80], 
                 [90, 110]]

# Run the Chi-Square Test of Independence
chi2_stat, p_value, dof, expected = chi2_contingency(observed_data)

print(f"Chi-Square Statistic: {chi2_stat:.2f}")
print(f"P-Value: {p_value:.5f}")

if p_value < 0.05:
    print("Significant! There is a relationship between Age and Theme Preference.")
else:
    print("Not significant. Theme preference is independent of age.")

R

In R, you can pass a matrix directly into the chisq.test() function, making it incredibly fast.

# Create a matrix of the observed counts
observed_data <- matrix(c(120, 90, 80, 110), nrow = 2, byrow = TRUE)

# Run the Chi-Square test
result <- chisq.test(observed_data)

# Print the results
print(result)

You Did It.

You have reached the end of the curriculum.

You started by learning what data is (Path 1), how to describe it (Path 2), how to predict it (Path 3), and finally, how to prove it (Path 4). You now have the exact statistical foundation used by modern data scientists to cut through the noise, reject false claims, and uncover the mathematical truth of the real world.