Simpsons Paradox

The Big picture is not always right

If data science and statistics has made one thing easy, it is to gain inferences from the big
picture. Hypothesize various statements and infer results with a certain confidence level. But at
times, because of the missing frequency and the use of probability or percentage can hide the
crucial information. One such paradox is Simpsons Paradox.
What is a paradox?
Any self-contradictory or seemingly absurd statement when investigated can is found to be true.
What is Simpsons Paradox?
Simpsons Paradox is mainly observed when we are studying various groups in aggregated form.
What the aggregated data shows may completely reverse when the data is divided into groups.
Or especially when the data is perceived in numerical form instead of using probability or
percentage.
Simpsons Paradox decoded.
Let us consider an example of two students and their assignment completion trend A & B
First day - A received 1 assignment and he failed to complete it
- B received 4 assignments and he completed 1 of them.
Second day - A received 4 assignments and he completed 3 of them.
- B received 1 assignments and he completed it.
The table would look something like this.
Day 1 Day 2 total
A 0/2 6/8 6/10
B 2/8 2/2 4/10

Lets convert the above table into percentages.

Day 1 Day 2 Total
A 0% 75% 60%
B 25% 100% 40%

If we only observe the second table, we notice that B has completed more than A on both days
yet, the percentage of B is less when calculated on an overall level with the same sample size.
This easily misleads us into making different wrong assumptions.
The above example is a classic phenomenon of Simpsons Paradox.
Why is it important?
The practicality of Simpsons paradox surfaces when the data possess the dilemma of which part
of data must be used. Whether it should be the aggregate data or the partitioned one.
Even if we do take the partitioned data into, the unequal sample size can give misleading
inferences like in the example above.
Without aggregating the data, we could have easily concluded that B always performed better
than A just looking at the percentages and forget to appreciate the improving in A from Day 1 to
Day to and hence the overall completion.
The practical significance! Kidney stone Treatment
During one of the medical advancement, doctors were comparing the success rate of old kidney
stone treatment and the new treatment. The new treatment which was non-surgical were found to
be more successful than the old one. But a deeper investigation revealed that the new treatment
was mostly operated on patients with small kidney stones.
Hence, the doctors had to reevaluate the results taking the size of the stone as a crucial variable
and look at the partitioned data to conclude and avoid Simpsons paradox.