Image of An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations

Journal Articles

An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations

In social policy evaluations, the multiple testing problem occurs due to the many hypothesis tests that are typically conducted across multiple outcomes and subgroups, which can lead to spurious impact findings. This article discusses a framework for addressing this problem that balances Types I and II errors. The framework involves specifying confirmatory and exploratory analyses in study protocols, delineating confirmatory outcome domains, conducting t tests on composite domain outcomes, and applying multiplicity corrections to composites across domains to obtain summative impact evidence. The article presents statistical background and discusses multiplicity issues for subgroup analyses, designs with multiple treatments, and reporting.

Availability

No copy data

Detail Information
Series Title
Evaluation Review
Call Number
-
Publisher
: sage publisher.,
Collation
-
Language
English
ISBN/ISSN
0193841x
Classification
-
Content Type
-
Media Type
-
Carrier Type
-
Edition
Vol. 33, No. 6, December 2009. pp. 519-538
Subject(s)
An Approach for Addressing the Multiple Testing
Multiple comparisons
Impact evaluations
Design of experiments
Specific Detail Info
-
Statement of Responsibility
Other version/related

No other version available

File Attachment
Comments
You must be logged in to post a comment