In statistics, the Simpson Paradox happens when a trend clearly shows up in clusters/brackets of data. But it disappears or, at worse it reverses when the data is grouped and combined. In short, the Simpson paradox shows that when the data moves from clusters to combined data, it hides several distributions, which end up creating a biased overall effect.
The Simpson paradox origin story
As Tom Grigg explained exceptionally well, the Simpson paradox took its name from Edward Hugh Simpson thanks to a technical paper in 1951.
Yet it was made famous when another statistician, Peter Bickel, was called – in 1971 – to analyze the admission data at UC Berkley’s suspected gender bias.
As the story goes, the university feared a lawsuit, so they had the data analyzed by Bickel.
When the data were combined, it really gave the impression that more males had been selected over women.
In fact, of the total male applicants, 44% were selected, and of the total female applicants, 35% were selected.
Yet when the data were analyzed by the department, it showed something completely different.
The admissions were biased toward women in four of the six departments analyzed.
But, as women applied to departments where fewer applicants were selected when the data combined, it gave an impression of bias toward male applicants.
Understanding the Simpson paradox
A good example is Nassim Taleb’s video on the topic.
While this is related to vaccine data, it can be easily translated into business, as we’ll see.
As Taleb explained about the vaccine data.
When the data are grouped under the same umbrella, after having been analyzed in clusters and homogeneous groups, it suddenly gives an opposite effect.
It’s like the data not only doesn’t give the same result when analyzed in brackets, but it gives the reverse effect.
This is what happens when the Simpson paradox messes up the statistics data.
Why?
Intuitively, when data, before compared under brackets, get combined, it disperses, thus making that worthless for the initial scope.
In the case of the vaccine, because many people over 60s were vaccinated, and a few people under 20s were vaccinated, when the data gets combined, it’s skewed toward the mortality of people over 60s, thus creating a bias and.
Beware of the Lurking variable
To keep things short, hidden variables in the combined spurs the overall analysis, making it worthless.
This is known as a “lurking variable” or a variable that affects the data at the point of creating a “spurious association” (in short, the cause-effect relationship ceases).
The Simpson paradox in business
The Simpson paradox can hide in many of the business and marketing analyses, as when the data is combined, it’s easy to mistake a correlation for causation.
Take the case of, as explained by adexchanger.com, for instance, when deciding on a programmatic campaign, when looking at the data for gender only, it shows how the male budget has seemingly more conversions, thus skewing the data toward males.
Yet from an age analysis, you figure that females between 18-24 have higher conversion rates.
If you don’t understand this bias, it’s easy to overspend on an overrepresented audience, not because it’s more aligned with your audience but because you’re misreading the data.
And as you can imagine, this can have substantial consequences on your bottom line (money wasted on ineffective campaigns and lost revenues as you’re not targeting the right audience).
And as most businesses now have a lot of data available, it’s easy to fall into the trapping of misusing it.
For that, it’s critical to establish projectbusiness processes, whereas it gets clear to the internal team when to use quantitative vs. qualitative data or both.
The characteristics of quantitative research contribute to methods that use statistics to make generalizations about something. These generalizations are constructed from data used to find patterns and averages and causal test relationships.
Qualitative research is performed by businesses that acknowledge the human condition and want to learn more about it. Some of the key characteristics of qualitative research that enable practitioners to perform qualitative inquiry comprise small data, the absence of definitive truth, the importance of context, the researcher’s skills and are of interests.
Indeed, quantitative data is extremely helpful to improve business processes.
However, it’s critical to know when human judgment needs to kick in, when some qualitative data is available that completely flips things upside down.
Indeed, if Amazon was going to look into these endeavors with a quantitative mindset, it would have never undertaken them as they did not make sense from a quantitative standpoint.
Yet, the intuitive understanding of how those things that might seem negative from a first-order effect standpoint (losing profits in the short-term) might make complete sense from a second-order effect standpoint (becoming way more successful in the long run).
Second-order thinking is a means of assessing the implications of our decisions by considering future consequences. Second-order thinking is a mental model that considers all future possibilities. It encourages individuals to think outside of the box so that they can prepare for every and any eventuality. It also discourages the tendency for individuals to default to the most obvious choice.
Understanding the implications of second-order effects is something that qualitative understanding and human judgment together can do.
Whereas quantitative data can be extremely useful to improve, in the short-term, business processes to make them way more efficient, which also, in the long-term, if properly used can create a competitive moat for the business.
For instance, going back to Amazon’s example, the company processes like inventorymanagement and order fulfillment are part of its core strategic advantage, and they are driven by quantitative data!
Key takeaways
The Simpson paradox is an effect that in statistics and probability can create biased analyses. In fact, when present the data combined from an analysis gives a reverse effect compared to the data analyzed in buckets.
The Simpson paradox can create biased analyses also in business and marketing creating overspending toward the wrong audience.
The Simpson paradox also makes it much harder to make decisions in business when doing statistical analysis.
As highlighted by German psychologist Gerd Gigerenzer in the paper “Heuristic Decision Making,” the term heuristic is of Greek origin, meaning “serving to find out or discover.” More precisely, a heuristic is a fast and accurate way to make decisions in the real world, which is driven by uncertainty.
Bounded rationality is a concept attributed to Herbert Simon, an economist and political scientist interested in decision-making and how we make decisions in the real world. In fact, he believed that rather than optimizing (which was the mainstream view in the past decades) humans follow what he called satisficing.
Second-order thinking is a means of assessing the implications of our decisions by considering future consequences. Second-order thinking is a mental model that considers all future possibilities. It encourages individuals to think outside of the box so that they can prepare for every and eventuality. It also discourages the tendency for individuals to default to the most obvious choice.
Lateral thinking is a business strategy that involves approaching a problem from a different direction. The strategy attempts to remove traditionally formulaic and routine approaches to problem-solving by advocating creative thinking, therefore finding unconventional ways to solve a known problem. This sort of non-linear approach to problem-solving, can at times, create a big impact.
Moonshot thinking is an approach to innovation, and it can be applied to business or any other discipline where you target at least 10X goals. That shifts the mindset, and it empowers a team of people to look for unconventional solutions, thus starting from first principles, by leveraging on fast-paced experimentation.
The concept of cognitive biases was introduced and popularized by the work of Amos Tversky and Daniel Kahneman in 1972. Biases are seen as systematic errors and flaws that make humans deviate from the standards of rationality, thus making us inept at making good decisions under uncertainty.
The Dunning-Kruger effect describes a cognitive bias where people with low ability in a task overestimate their ability to perform that task well. Consumers or businesses that do not possess the requisite knowledge make bad decisions. What’s more, knowledge gaps prevent the person or business from seeing their mistakes.
Occam’s Razor states that one should not increase (beyond reason) the number of entities required to explain anything. All things being equal, the simplest solution is often the best one. The principle is attributed to 14th-century English theologian William of Ockham.
The Mandela effect is a phenomenon where a large group of people remembers an event differently from how it occurred. The Mandela effect was first described in relation to Fiona Broome, who believed that former South African President Nelson Mandela died in prison during the 1980s. While Mandela was released from prison in 1990 and died 23 years later, Broome remembered news coverage of his death in prison and even a speech from his widow. Of course, neither event occurred in reality. But Broome was later to discover that she was not the only one with the same recollection of events.
The bandwagon effect tells us that the more a belief or idea has been adopted by more people within a group, the more the individual adoption of that idea might increase within the same group. This is the psychological effect that leads to herd mentality. What is marketing can be associated with social proof.
Gennaro is the creator of FourWeekMBA, which reached about four million business people, comprising C-level executives, investors, analysts, product managers, and aspiring digital entrepreneurs in 2022 alone | He is also Director of Sales for a high-tech scaleup in the AI Industry | In 2012, Gennaro earned an International MBA with emphasis on Corporate Finance and Business Strategy.
