Approaches toward External Validity of Randomized Controlled Trials in the Social Sciences

NSF Award Search · 01002324DB NSF RESEARCH & RELATED ACTIVIT · $231,858 · view on nsf.gov ↗

Abstract

Social scientists and decision makers have been interested in whether and how well they can generalize the results of randomized controlled trials (RCTs) to new populations, places, and contexts. This question of external validity is a foundation of evidence-based decision making because study populations and sites are often different from the real-world populations and sites to which an intervention might be scaled up. This research develops statistical approaches to improve the external validity of RCTs in the social sciences. The research makes two methodological contributions. The first is a new framework to design RCTs for external validity. Specifically, this project develops an algorithm to select experimental sites such that researchers can credibly estimate generalizable causal effects. Site selection is essential because experimental results in the social sciences are often heterogeneous across places, and biased selection of experimental sites can lead to low generalizability and replicability. The second contribution is a statistical tool to quantify the robustness to external validity bias. This new measure of external robustness allows researchers to evaluate external validity even when RCTs were conducted without external validity considerations. These two methods complement each other and together provide a unified pipeline to improve the external validity of RCTs in the social sciences. The project is co-funded by the Science of Science: Discovery, Com

Key facts

NSF award ID
2554173
Awardee
Massachusetts Institute of Technology (MA)
SAM.gov UEI
E2NYLCDML6V1
PI
Naoki Egami
Primary program
01002324DB NSF RESEARCH & RELATED ACTIVIT
All programs
GRADUATE INVOLVEMENT
Estimated total
$231,858
Funds obligated
$116,947
Transaction type
Standard Grant
Period
10/01/2025 → 08/31/2026