Evaluation of Legislation: Skating on Thin Ice

While parliaments and politicians increasingly call for the evaluation of legislation, the methodological implications of statutes as an evaluand have not received much interest. This article explores the challenges and methods of evaluating statutes. It describes the methodology and the costs of eight all-encompassing evaluations of statutes at Swiss federal level. It then presents approaches …

Four Waves of Evaluation Diffusion

This article investigates the dissemination of evaluation as it appears from a Swedish and to a lesser extent an Atlantic vantage point since 1960. Four waves have deposited sediments, which form present-day evaluative activities. The scientific wave entailed that academics should test, through two-group experimentation, appropriate means to reach externally set, admittedly subjective, goals. P…

Criteria and Theory in the Evaluation of Organizations

This article argues that evaluation-specific logic, which involves the identification of relevant criteria and performance standards for the evaluand (considered as a whole or in its single dimensions), can benefit from research on the causal chains leading from programme inputs to the final outcomes; and on the contexts and mechanisms responsible, which are normally investigated when adopting …

Evaluating Development Interventions in Peace-Precarious Situations

International development assistance tackles sociopolitical and socioeconomic problems, typically with formal host government and population buy-in (or acquiescence), in settings with complex interrelated challenges. A new subset of contemporary interventions, however, faces all of the usual development challenges plus security threats affecting foreign and local staff, host country leadership,…

Involving Developing Countries in the Evaluation of Development Cooperation: …

Joint evaluation of development cooperation is becoming more imperative within the framework of the Paris Declaration. It has, however, been noted that the documentation of practical experience with evaluations conducted in a joint fashion is missing. This article seeks to contribute to the debate on the practical implications of conducting joint evaluations. The model adopted involves one dono…

Evaluation of Cluster Policy: A Methodological Overview

Cluster policy is increasingly becoming part of many governments’ economic policy strategies. At the same time, evidence-based policy-making is gaining importance, bringing about a call for policy evaluation. Since the quality of the evaluation results depends highly on the method used, data, assumptions and techniques must be adequate for the specific evaluation question. This holds for cluste…

Peer Evaluation and its Blurred Boundaries: Results from a Meta-evaluation in…

Peer evaluation has proved to be a popular approach with both peers and the evaluated, but there has been considerable variation in the ways in which peer evaluations have been implemented. There are different forms, purposes and ways of organizing peer evaluation. Peer evaluations are also not uniform and links to other evaluations can be made. After providing a general overview of different k…

Unpacking the Politics of Evaluation: A Dramaturgical Analysis

This article draws on a four-year evaluation that assessed the delivery of support services by 15 British hospices and social agencies to family carers of terminally ill people. It aims to examine the politics of evaluation research. Three main arguments are posited: first, that evaluation research is distinguishable from ‘quick and dirty’ evaluations, which are insufficiently resourced and not…

To Be or Not to Be a Profession: Pros, Cons and Challenges for Evaluation

Debates on the professionalization of evaluation regularly fuel controversies. Evaluation literature contains varied points of view in favour of or against means of restricting access to the profession and quality control mechanisms. This article examines the aims pursued (e.g. institutionalization, quality improvement, ethical practice) and challenges faced by the promoters of the professional…

What Doesn’t Work? Three Failures, Many Answers

Current debates on impact evaluation have addressed the question ‘what works and what doesn’t?’ mainly focussing on methodology failures when providing evidence of impact. In order to answer that question, this article contrasts different approaches to evaluation in terms of the way they address different kinds of possible failures. First, there is more to be debated than simply methodological …

Gender Similarities and Differences in the Treatment, Relapse, and Recovery C…

This study explores the influence of gender on changes in recovery status among participants in a longitudinal study. The study sample (N = 1,202; 60% female) is recruited on referral to treatment, and annual interviews are conducted from Years 2 to 6 following intake. At each annual observation, participants are classified into one of four statuses (recovery, treatment, incarcerated, and using…

Longitudinal Research That Can Inform Dynamic Models for the Treatment of Add…

Edition

Vol. 32, No. 1, February 2008. Page 3 - 6

ISBN/ISSN

0193841X

Collation

-

Series Title

Evaluation Review

Call Number

-

Patterns of Crime and Drug Use Trajectories in Relation to Treatment Initiati…

Drug abusers vary considerably in their drug use and criminal behavior over time, and these trajectories are likely to influence drug treatment participation and treatment outcomes. Drawing on longitudinal natural history data from three samples of adult male drug users, we identify four groups with distinctive drug use and crime trajectories during the 5 years prior to their first treatment ep…

Longitudinal HIV Risk Behavior Among the Drug Abuse Treatment Outcome Studies…

Longitudinal trajectories for HIV risk were examined over 5 years following treatment among 1,393 patients who participated in the nationwide Drug Abuse Treatment Outcome Studies. Both injection drug use and sexual risk behavior declined over time, with most of the decline occurring between intake and the first-year follow-up. However, results of the application of growth mixture models for bot…

Nine-Year Psychiatric Trajectories and Substance Use Outcomes: An Application…

This study identifies longitudinal psychiatric trajectories of 934 adult individuals entering chemical dependency treatment in a private, managed care health plan and examines the relationship of these trajectories with substance use (SU) outcomes. The authors apply a group-based modeling approach to identify trajectory groups based on repeated measures of psychiatric severity for 9 years and i…

The Interaction of Co-Occurring Mental Disorders and Recovery Management Chec…

This article examines the effectiveness of quarterly Recovery Management Checkups (RMCs) for people with substance disorders by level of co-occurring mental disorders (34% none, 27% internalizing disorders, and 39% internalizing and externalizing) across two randomized experiments with 92% to 97% follow-up. The 865 participants are 82% African American, 53% female, and age 37 on average. …

Designing a Household Survey to Address Seasonality in Child Care Arrangements

In household telephone surveys, a long field period may be required to maximize the response rate and achieve adequate sample sizes. However, long field periods can be problematic when measures of seasonally affected behavior are sought. Surveys of child care use are one example because child care arrangements vary by season. Options include varying the questions posed about school-year and sum…

The Truncation-by-Death Problem: What To Do in an Experimental Evaluation Whe…

Although experiments are viewed as the gold standard for evaluation, some of their benefits may be lost when, as is common, outcomes are not defined for some sample members. In evaluations of marriage interventions, for example, a key outcome—relationship quality—is undefined when a couple splits up. This article shows how treatment-control differences in mean outcomes can be misleadi…

Mixed Results in a Transitional Planning Program for Alternative School Students

Disciplinary alternative schools have a reputation as gateways to the juvenile and criminal justice systems. The authors conducted an evaluation of an intervention (Strategies for Success) designed to divert seventh-, eighth-, and ninth-grade alternative school students from this gateway. They used propensity score matching and a multivariate random effects model to estimate program impacts and…

Using an Empirical Binomial Hierarchical Bayesian Model as an Alternative to …

This article explores the statistical methodologies used in demonstration and effectiveness studies when the treatments are applied across multiple settings. The importance of evaluating and how to evaluate these types of studies are discussed. As an alternative to standard methodology, the authors of this article offer an empirical binomial hierarchical Bayesian model as a way to effectively e…