External Validity | Types, Threats & Examples
External validity is the extent to which you can generalise the findings of a study to other situations, people, settings, and measures. In other words, can you apply the findings of your study to a broader context?
The aim of scientific research is to produce generalisable knowledge about the real world. Without high external validity, you cannot apply results from the laboratory to other people or the real world.
In qualitative studies, external validity is referred to as transferability.
Types of external validity
There are two main types of external validity: population validity and ecological validity.
Population validity refers to whether you can reasonably generalise the findings from your sample to a larger group of people (the population).
Population validity depends on the choice of population and on the extent to which the study sample mirrors that population. Non-probability sampling methods are often used for convenience. With this type of sampling, the generalisability of results is limited to populations that share similar characteristics with the sample.
Here, your sample is not representative of the whole population of students at your university. The findings can only reasonably be generalised to populations that share characteristics with the participants, e.g. university-educated men studying STEM subjects.
For higher population validity, your sample would need to include people with different characteristics (e.g., women, nonbinary people, and students from different fields, countries, and socioeconomic backgrounds).
Samples like this one, from Western, educated, industrialised, rich, and democratic (WEIRD) countries, are used in an estimated 96% of psychology studies, even though they represent only 12% of the world’s population. As outliers in terms of visual perception, moral reasoning, and categorisation (among many other topics), WEIRD samples limit broad population validity in the social sciences.
Ecological validity refers to whether you can reasonably generalise the findings of a study to other situations and settings in the ‘real world’.
In the example above, it is difficult to generalise the findings to real-life driving conditions. A computer-based task using a mouse does not resemble real-life driving conditions with a steering wheel. Additionally, a static image of an orange cat may not represent common real-life hurdles when driving.
To improve ecological validity in a lab setting, you could use an immersive driving simulator with a steering wheel and foot pedal instead of a computer and mouse. This increases psychological realism by more closely mirroring the experience of driving in the real world.
Alternatively, for higher ecological validity, you could conduct the experiment using a real driving course.
Trade-off between external and internal validity
Internal validity is the extent to which you can be confident that the causal relationship established in your experiment cannot be explained by other factors.
Threats to external validity and how to counter them
Threats to external validity are important to recognise and counter in a research design for a robust study.
|Sampling bias||The sample is not representative of the population.||The sample includes only people with depression. They have characteristics (e.g., negative thought patterns) that may make them very different from other clinical populations, like people with personality disorders or schizophrenia.|
|History||An unrelated event influences the outcomes.||Right before the pretest, a natural disaster takes place in a neighbouring state. As a result, pretest anxiety scores are higher than they might be otherwise.|
|Experimenter effect||The characteristics or behaviours of the experimenter(s) unintentionally influence the outcomes.||The trainer of the mindfulness sessions unintentionally stressed the importance of this study for the research department’s funding. Participants work extra hard to reduce their anxiety levels during the study as a result.|
|Hawthorne effect||The tendency for participants to change their behaviours simply because they know they are being studied.||The participants actively avoid anxiety-inducing situations for the period of the study because they are conscious of their participation in the research.|
|Testing effect||The administration of a pre- or post-test affects the outcomes.||Because participants become familiar with the pre-test format and questions, they are less anxious during the post-test and recall less anxiety then.|
|Aptitude-treatment||Interactions between characteristics of the group and individual variables together influence the dependent variable.||Interactions between certain characteristics of the participants with depression (e.g., negative thought patterns) and the mindfulness exercises (e.g., focus on the present) improve anxiety levels. The findings are not replicated with people with personality disorders or schizophrenia.|
|Situation effect||Factors like the setting, time of day, location, and researchers’ characteristics limit generalisability of the findings.||The study is repeated with one change; the participants practise mindfulness at night rather than in the morning. The outcomes do not show any improvement this time.|
How to counter threats to external validity
There are several ways to counter threats to external validity:
- Replications counter almost all threats by enhancing generalisability to other settings, populations and conditions.
- Field experiments counter testing and situation effects by using natural contexts.
- Probability sampling counters selection bias by making sure everyone in a population has an equal chance of being selected for a study sample.
- Recalibration or reprocessing also counters selection bias using algorithms to correct weighting of factors (e.g., age) within study samples.
Frequently asked questions about external validity
- What is external validity?
The external validity of a study is the extent to which you can generalise your findings to different groups of people, situations, and measures.
- What are the two types of external validity?
The two types of external validity are population validity (whether you can generalise to other groups of people) and ecological validity (whether you can generalise to other situations and settings).
- What are threats to external validity?
There are seven threats to external validity: selection bias, history, experimenter effect, Hawthorne effect, testing effect, aptitude-treatment, and situation effect.
- How does attrition threaten external validity?
With a biased final sample, you may not be able to generalise your findings to the original population that you sampled from, so your external validity is compromised.