Cherry picking

Cherry Picking

Updated October 2026 · reviewed for today
In short

Cherry picking, also called the fallacy of incomplete evidence, is pointing to the data that supports a position while leaving out the data that contradicts it. Each example shown may be true; the conclusion misleads because of what is missing. In science the same problem appears as selective publication, where favourable results are more likely to reach print than unfavourable ones.

Cherry Picking: hand-drawn explainer by The Business Engineer
Drawing by The Business Engineer.

How it works

01
Start from the answer
The conclusion comes first; the evidence is gathered to fit it.
02
Filter quietly
Unhelpful results, periods, regions or quotes are dropped without saying so.
03
Present the sample as the whole
The audience sees only the selection and assumes it is representative.

Where you see it today

AI leaderboards

Model rankings can be cherry-picked at scale. The Leaderboard Illusion (Singh et al., 2025) found that some providers privately tested several model variants on Chatbot Arena and published only the best score; the authors identified 27 private variants tested by Meta before the Llama 4 release. They argue this selective disclosure biases the rankings.

Drug trials

A review of antidepressant trials registered with the US Food and Drug Administration (Turner et al., 2008, NEJM) found that, in the published literature, 94% of the trials appeared positive, against 51% in the FDA’s own analysis. Negative trials had mostly gone unpublished or been written up as positive.

Asking an AI to back you up

Ask an assistant for evidence that you are right and it will usually find some. AI assistants tend to match a user’s stated view over the truthful answer, a behaviour called sycophancy (Sharma et al., 2023), so a one-sided question returns one-sided evidence.

Demos and dashboards

A launch demo shows the best run out of many; a quarterly review shows the metrics that went up. Neither is a lie, but each is a sample chosen by the person who benefits from it.

Cherry picking and its relatives

Cherry picking
What gets left outInconvenient data points
ExampleQuoting the best quarter, not the full year
Publication bias
What gets left outStudies with null or negative results
ExampleJournals print the trials that worked
Survivorship bias
What gets left outThe cases that failed and disappeared
ExampleStudying only the startups that made it
Texas sharpshooter
What gets left outThe pattern is drawn after the data
ExampleFinding a cluster, then calling it the target
Confirmation bias
What gets left outEvidence you never went looking for
ExampleSearching only for reviews that agree with you

How to spot and avoid it

  • Ask for the denominator: how many runs, trials, customers or months were there in total?
  • Request the full set before the highlights, including the failures and the flat results.
  • Decide which metrics and time window you will judge by before you see the data, the logic behind pre-registration in science (Nosek et al., 2018).
  • For AI models, prefer independent evaluations and look for spread across runs, not one headline score.
  • When you use an AI assistant, ask it explicitly for the strongest evidence against your view.
The Business Engineer lens

In the AI economy each side has a pile of true examples: record benchmarks and failed pilots, soaring capex and idle capacity. The Business Engineer reads spending, usage and revenue together, because any one of them alone can be made to tell the story you want.

Read: Has The AI Bubble Popped? →Explore the platform →

Related: Confirmation bias · Survivorship bias · Texas sharpshooter fallacy · Hasty generalization · Straw man fallacy

Frequently asked questions

What is cherry picking?

It is presenting only the evidence that supports a conclusion while ignoring the evidence that contradicts it, so a true sample gives a false picture.

Is cherry picking the same as lying?

Not quite. Each data point shown may be accurate; the deception comes from what is left out and from presenting the selection as representative.

How does cherry picking affect AI benchmarks?

When developers can test many versions privately and publish only the best result, leaderboard scores overstate typical performance; a 2025 study of Chatbot Arena described this as selective disclosure.

How do scientists guard against it?

By registering the research question and analysis plan before seeing the results, and by publishing null and negative results as well as positive ones.

Sources

The full guide

The complete explainer, cleaned up and reformatted for mobile.

Cherry picking is a cognitive bias that occurs when individuals or organizations selectively choose and present information, data, or evidence to support a preconceived idea, argument, or agenda, while ignoring or suppressing contradictory information. It is akin to the act of picking ripe cherries from a tree while leaving the unripe or spoiled ones behind.

Key Characteristics of Cherry Picking

Cherry picking is characterized by several key features

  • Selective Presentation: It involves the deliberate choice to present only a subset of available data or evidence, often those that support a specific position.
  • Confirmation Bias: Cherry picking is closely related to confirmation bias, as individuals tend to seek and emphasize information that confirms their existing beliefs or preferences.
  • Distorted Perspective: By presenting a biased selection of data, cherry picking distorts the true nature of a subject, issue, or argument.
  • Misleading Interpretation: It can lead to misleading interpretations of data, as the omitted information may provide crucial context or alternative perspectives.

Types of Cherry Picking

Cherry picking can manifest in various forms, depending on the context and the nature of the information being manipulated. Some common types include:

1. Data Cherry Picking:

  • Selectively presenting data points, statistics, or research findings that support a particular claim while ignoring data that contradict it.

2. Quote Mining:

  • Extracting quotes from individuals or sources out of context to give a false impression of their views or opinions.

3. Anecdotal Evidence:

  • Relying on isolated anecdotes or personal experiences to support a broader argument while disregarding systematic data or research.

4. Out-of-Context Information:

  • Presenting information, statements, or events without providing the necessary context, leading to a distorted interpretation.

5. Historical Cherry Picking:

  • Selectively highlighting historical events or examples that support a particular narrative while omitting contradictory historical facts.

Examples of Cherry Picking

Cherry picking can be observed in various domains, including science, politics, advertising, and everyday decision-making. Here are a few examples to illustrate its occurrence:

Example 1: Scientific Research

In scientific research, cherry picking can occur when researchers selectively report only the results that show a significant effect of a treatment or intervention, while omitting data that indicate no effect or negative outcomes. This can lead to biased conclusions about the efficacy of a particular intervention.

Example 2: Political Discourse

Politicians and political commentators may cherry pick statistics or anecdotes that support their policy proposals or criticisms of opponents, ignoring data that provide a more nuanced or balanced view of the issue.

Example 3: Advertising and Marketing

Advertisers may selectively present positive customer reviews and testimonials for a product, while ignoring negative feedback or complaints. This can create a misleading perception of the product’s quality.

Example 4: Personal Decision-Making

Individuals may selectively seek advice or information that confirms their preconceived beliefs or decisions, while avoiding sources that offer alternative viewpoints or evidence. This can lead to biased decision-making.

Consequences of Cherry Picking

Cherry picking can have several detrimental consequences:

1. Misleading Information:

  • It can mislead individuals or the public by presenting a skewed or incomplete view of a subject, leading to incorrect beliefs or decisions.

2. Loss of Credibility:

  • Individuals or organizations that engage in cherry picking may lose credibility and trust when their biased presentation of information is exposed.

3. Undermining Objectivity:

  • Cherry picking undermines the principles of objectivity and impartiality, which are essential in research, journalism, and public discourse.

4. Policy and Decision Errors:

  • In policy-making and decision-making contexts, cherry picking can lead to flawed policies or strategies based on incomplete or biased information.

Identifying Cherry Picking

Recognizing cherry picking is crucial for critically evaluating information and making informed decisions. Here are some strategies to identify cherry picking:

1. Look for Missing Context:

  • Examine whether the presented information lacks essential context or additional data that could provide a more complete picture.

2. Check for Selective Omissions:

  • Assess whether certain data or evidence that contradicts the presented argument or narrative is conspicuously absent.

3. Evaluate the Source:

  • Consider the credibility and trustworthiness of the source presenting the information. Biased or unreliable sources are more likely to engage in cherry picking.

4. Seek Alternative Perspectives:

  • Look for alternative viewpoints, sources, or analyses that provide a more balanced and comprehensive understanding of the subject.

Mitigating Cherry Picking in Research

Researchers and analysts must take proactive measures to mitigate cherry picking in research and reporting. Here are some strategies to address this bias:

1. Transparency and Full Disclosure:

  • Researchers should disclose all relevant data, even if it contradicts their hypotheses or expectations. Full transparency helps readers or reviewers evaluate the validity of the findings.

2. Peer Review:

  • Peer review processes can help identify and address cherry picking in research publications by involving independent experts who assess the completeness and accuracy of the reported data.

3. Robust Methodology:

  • Researchers should employ rigorous and unbiased data collection and analysis methods to minimize the risk of cherry picking.

4. Pre-registration:

  • Pre-registering research plans and hypotheses can reduce the temptation to cherry pick data after the fact, as the study’s design and objectives are established in advance.

5. Statistical Analysis:

  • Use appropriate statistical methods to analyze data, and report all relevant statistical measures, including confidence intervals and effect sizes.

Real-World Applications of Mitigating Cherry Picking

Mitigating cherry picking is essential in various real-world contexts:

1. Scientific Journals:

  • Academic journals use peer review processes and guidelines to ensure that research publications adhere to rigorous standards of reporting and transparency.

2. Data Journalism:

  • Data journalists and investigative reporters use careful analysis and fact-checking to identify and expose cherry picking in news stories and reports.

3. Consumer Protection:

  • Regulatory agencies and consumer protection organizations monitor advertising and marketing practices to prevent misleading or deceptive cherry-picked claims.

4. Public Policy:

  • Policymakers and legislators rely on comprehensive and unbiased research to make informed decisions that benefit the public interest.

The Future of Mitigating Cherry Picking

As information dissemination and data analysis continue to evolve, addressing cherry picking may require new approaches:

1. Data Verification Technologies:

  • Emerging technologies and tools for data verification and fact-checking may help identify and mitigate cherry picking in real-time.

2. Algorithmic Auditing:

  • Algorithms that audit and assess the completeness and accuracy of data presentations may become more prevalent in data-driven industries.

3. Education and Media Literacy:

  • Promoting media literacy and critical thinking skills can empower individuals to recognize and resist cherry-picked information.

Conclusion

Cherry picking is a pervasive cognitive bias that distorts the presentation of information and evidence in various domains, from scientific research to politics and advertising. It can lead to misleading conclusions, erode trust, and undermine the integrity of communication and decision-making processes. Recognizing cherry picking and employing strategies to mitigate it are essential steps in fostering transparency, objectivity, and informed decision-making in an information-rich world. As technology and information dissemination methods continue to evolve, the challenge of addressing cherry picking will remain a critical aspect of maintaining the quality and reliability of information and research.

Key Highlights

  • Introduction to Cherry Picking:
    • Cherry picking involves presenting selective information or evidence to support a preconceived idea or agenda while ignoring contradictory information.
  • Characteristics of Cherry Picking:
    • Selective presentation, confirmation bias, distorted perspective, and misleading interpretation are key characteristics of cherry picking.
  • Types of Cherry Picking:
    • Cherry picking can manifest as data cherry picking, quote mining, anecdotal evidence, out-of-context information, and historical cherry picking.
  • Examples of Cherry Picking:
    • Examples include selectively reporting scientific research findings, using statistics to support political arguments, showcasing positive reviews in advertising, and seeking information that confirms personal beliefs.
  • Consequences of Cherry Picking:
    • Cherry picking can lead to misleading information, loss of credibility, undermining objectivity, and errors in policy and decision-making.
  • Identifying Cherry Picking:
    • Strategies for identifying cherry picking include looking for missing context, checking for selective omissions, evaluating the source, and seeking alternative perspectives.
  • Mitigating Cherry Picking in Research:
    • Researchers can mitigate cherry picking through transparency, peer review, robust methodology, pre-registration, and proper statistical analysis.
  • Real-World Applications:
    • Mitigating cherry picking is crucial in scientific journals, data journalism, consumer protection, and public policy.
  • The Future of Mitigating Cherry Picking:
    • Emerging technologies, algorithmic auditing, and education in media literacy are potential future approaches to addressing cherry picking.
  • Conclusion:
    • Recognizing and mitigating cherry picking are essential for maintaining transparency, objectivity, and informed decision-making across various domains in an information-rich world. Continued efforts to address cherry picking will be crucial as technology and information dissemination methods evolve.

Connected · Cherry Picking

Where to go next from Cherry Picking

Picked for this page by topic. Every link checked live.

Has The AI Bubble Popped? The Business Engineer readHas The AI Bubble Popped? Every market correction revives the same question: “Is this the AI bubble finally popping?”.Read on businessengineer.ai →
Confirmation biasRelated conceptConfirmation biasConfirmation Bias refers to the tendency to interpret information in a way that supports existing beliefs.Survivorship biasRelated conceptSurvivorship biasSurvivorship bias is a pervasive fallacy that exists in business, where people focus on the few survived players, in any given market, without…Texas sharpshooter fallacyRelated conceptTexas sharpshooter fallacyThe Texas Sharpshooter Fallacy is a type of cognitive bias that involves selectively focusing on data points or patterns that support a preconceived…Hasty generalizationRelated conceptHasty generalizationThe Hasty Generalization Fallacy is an informal fallacy that occurs when a person makes a general conclusion about a group, category, or population…Straw man fallacyRelated conceptStraw man fallacyA straw man is a fallacy in which someone replaces an opponent's real argument with a weaker or more extreme version, knocks that version down, and…
Scroll to Top
FourWeekMBA