Demand characteristics are the cues in a study that let participants guess what the researcher expects, which can change how they behave. Psychologist Martin Orne described them in a 1962 paper in American Psychologist on the social psychology of the experiment. Researchers reduce the effect with blinding, cover stories and designs that hide the hypothesis.
Blinding, between-subject designs, cover stories, neutral instructions and post-study questions about what people guessed.
Where you see it today
AI models that know they are tested
The same problem now applies to AI evaluation. A study testing frontier language models found they could tell evaluation transcripts from real-world use well above chance, and could often guess what a test was measuring (Needham et al., 2025). If a model behaves differently when it suspects a test, benchmark results may not reflect deployment.
Sycophancy as a demand effect
AI assistants trained on human feedback tend to tell users what they appear to want to hear. Researchers found this sycophancy across several leading assistants and linked it partly to human preference data (Sharma et al., 2023). Leading questions to a chatbot work like leading questions to a study participant.
Experienced online participants
Behavioural research relies heavily on online panels. Work on Amazon Mechanical Turk found many workers had already taken part in similar studies, which can make them better at spotting the hypothesis (Chandler, Mueller and Paolacci, 2014).
Experimental economists treat demand effects as a design issue and recommend techniques such as non-deceptive obfuscation (Zizzo, 2010). Product teams face the same trap in user interviews and beta tests, which is one reason live A/B tests, where users do not know they are in a study, are trusted more.
Who changes behaviourThe researcher, who signals expectations without meaning to
Typical fixDouble-blind procedures
Social desirability bias
Who changes behaviourRespondents, who answer to look good
Typical fixAnonymity; indirect questions
Evaluation awareness in AI
Who changes behaviourA model that detects it is being tested
Typical fixRealistic test environments; monitoring in deployment
How to spot and avoid it
Write instructions that describe the task, not the hypothesis.
Use between-group designs so no one sees all the conditions and works out the comparison.
Ask participants at the end what they thought the study was about, and analyse those who guessed separately.
In customer interviews, ask about past behaviour, not about the feature you hope they want.
When testing AI models or prompts, use realistic tasks and compare results with behaviour in real use.
The Business Engineer lens
Any measurement that the measured party can detect will drift towards what the measurer wants, and that now includes AI systems being benchmarked. The Business Engineer reads AI capability claims with that in mind: what a model does in a test is evidence, not proof, of what it will do in production.
Cues in a study that tell participants what the researcher expects, which can lead them to change their behaviour and bias the results.
Who coined the term demand characteristics?
Psychologist Martin Orne, in a 1962 paper in American Psychologist.
How do researchers reduce demand characteristics?
With blinding, cover stories, between-subject designs, neutral instructions and post-study checks on what participants guessed.
Do demand characteristics apply to AI?
A version does: research shows frontier models can often tell when they are being evaluated, which can make test results less representative of real use.
The complete explainer, cleaned up and reformatted for mobile.
Demand characteristics are a crucial but often overlooked aspect of psychological research. These are subtle cues or signals within an experiment that may lead participants to form expectations about the study’s purpose or how they should behave. These expectations can inadvertently affect participants’ responses and compromise the validity of the research.
Demand characteristics refer to cues or aspects of a research study that unintentionally convey information to participants about the researcher’s expectations or the purpose of the study. When participants become aware of these cues, they may adjust their behavior, responses, or attitudes to align with what they believe the researcher wants or expects.
Key points to understand about demand characteristics
Implicit Cues: Demand characteristics are typically implicit and unintentional. Researchers do not deliberately introduce these cues, but they can emerge from various aspects of the study design, such as instructions, procedures, or even the behavior of the experimenter.
Participant Expectations: When participants perceive demand characteristics, they may feel a social or psychological pressure to conform to what they believe is expected of them. This can lead to altered behavior, self-presentation, or responses during the study.
Threat to Validity: The presence of demand characteristics can threaten the internal validity of a study. Internal validity refers to the accuracy of conclusions drawn about cause-and-effect relationships within the study.
Blind Studies: Researchers often employ blind procedures or double-blind experiments to mitigate the impact of demand characteristics. In a blind study, either the participants or the experimenters (or both) are unaware of critical aspects of the research.
Examples of Demand Characteristics
Demand characteristics can manifest in various ways within a research setting. Here are some common examples:
Experimenter Expectancy Effect: If an experimenter expects a certain outcome, they may inadvertently convey this expectation through non-verbal cues or subtle behaviors. Participants may then adjust their responses to align with the experimenter’s expectation.
Subtle Cues in Instructions: The wording of instructions provided to participants can contain cues that lead them to believe they should respond in a particular manner. For instance, using phrases like “most people agree thatโฆ” can imply a socially desirable response.
Social Desirability Bias: Participants may perceive that certain responses are socially desirable or more acceptable than others. This perception can lead them to provide answers they believe align with social norms or expectations.
Experiment Environment: Aspects of the experiment environment, such as the presence of cameras or one-way mirrors, may suggest to participants that they are being observed or that their behavior is being assessed, potentially altering their behavior.
Impact of Demand Characteristics on Psychological Research
Understanding the impact of demand characteristics is crucial for researchers, as these cues can affect various aspects of psychological studies:
1. Altered Behavior
Participants may modify their behavior or responses to align with what they perceive as the experimenter’s expectations. This can lead to inauthentic or biased data, as participants may not express their true thoughts or behaviors.
2. Reduced Validity
The presence of demand characteristics can compromise the internal validity of a study. When participants respond based on perceived expectations rather than their genuine beliefs or behaviors, it becomes challenging to draw accurate conclusions about the variables being studied.
3. Difficulty in Generalization
Researchers may find it challenging to generalize findings from studies influenced by demand characteristics to real-world settings. The artificial influence of cues within the study environment may not accurately reflect how individuals would naturally respond.
4. Ethical Concerns
The unintentional manipulation of participant responses through demand characteristics raises ethical concerns. Researchers have a responsibility to conduct studies that respect participants’ autonomy and provide them with informed consent.
Minimizing the Influence of Demand Characteristics
Researchers employ several strategies to minimize the influence of demand characteristics and enhance the validity of their studies:
1. Double-Blind Procedures
In double-blind experiments, neither the participants nor the experimenters are aware of critical details of the study, such as the experimental condition or the expected outcomes. This reduces the risk of experimenter bias and participant reactivity.
2. Use of Deception
While controversial, some studies use deception to mask the true purpose of the research from participants. However, ethical guidelines require that participants be fully debriefed after the study and that the deception does not cause harm.
3. Minimizing Explicit Instructions
Researchers can minimize explicit instructions that may suggest desired responses or behaviors. Instead, they can use neutral or open-ended instructions that do not lead participants toward a specific outcome.
4. Control Conditions
Including control conditions in the study design allows researchers to compare participant responses in experimental conditions with those in non-manipulated or placebo conditions. This helps identify whether demand characteristics are influencing results.
5. Post-Experimental Interviews
Conducting post-experimental interviews or debriefing sessions with participants can help researchers gauge their perceptions of the study’s purpose and identify any demand characteristics that may have influenced their behavior.
6. Naturalistic Settings
Whenever possible, researchers can conduct studies in more naturalistic settings to reduce the artificial influence of the research environment. This can help elicit more authentic responses from participants.
Broader Implications in Psychological Research
The consideration of demand characteristics extends beyond individual studies and has broader implications for the field of psychology:
1. Replicability Crisis
The presence of demand characteristics in psychological research has contributed to the replicability crisisโa phenomenon where many published psychological findings cannot be reliably replicated. The effect of these cues can lead to inflated effect sizes that are not generalizable.
2. Scientific Integrity
Addressing demand characteristics is essential for upholding scientific integrity. Researchers must aim to produce valid and reliable findings that accurately represent psychological phenomena.
3. Ethical Research
Mitigating the impact of demand characteristics aligns with ethical research practices. Researchers have an ethical responsibility to ensure that participants’ responses are not manipulated by cues within the study environment.
4. Advancing Knowledge
Minimizing the influence of demand characteristics allows researchers to advance the understanding of human behavior and cognition more accurately. Reliable and valid findings contribute to the growth of psychological knowledge.
Conclusion
Demand characteristics are a subtle yet significant influence in psychological research. Researchers must be vigilant in recognizing and mitigating their impact to ensure the validity and reliability of their studies. By employing strategies such as double-blind procedures, minimizing explicit instructions, and conducting post-experimental interviews, researchers can enhance the quality of their research and contribute to the advancement of psychological science. Understanding and addressing demand characteristics is not only vital for individual studies but also essential for upholding the integrity of the field of psychology as a
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
