A Contribution to Current Debates in Impact Evaluation

A debate on approaches to impact evaluation has raged in development circles in recent years. This paper makes a contribution to this debate through discussion of four issues. First, I point out that there are two definitions of impact evaluation. Neither is right or wrong, but they refer to completely different things. There is no point in methodological debates unless they agree a common star…

Evaluating Efficiency in the Implementation of Structural Funds Operations

This article proposes a methodology to assess the efficiency of the European Union Structural Funds (SF) operations using two alternative approaches: first, by a unit cost analysis of the programme’s output indicators; second, by the comparative study of the tendering, contract-awarding and implementation costs of a sample of projects. This methodology has been applied to the mid-term evaluatio…

Multicultural Awareness in Evaluation: Dilemmas and Challenges

The purpose of this article is to discuss what is meant by multicultural competence in evaluation and how policies and programmes aiming at multicultural awareness and ‘validity’ can be evaluated. The article discusses three main ways of understanding multiculturalism and how multicultural competence in evaluation can be defined. It also develops evaluation criteria that can be used for assessi…

Evaluating Diplomacy: A Mission Impossible?

This article presents a model to evaluate national diplomatic interventions in multilateral decision-making arenas. The performance-driven evaluation model that is commonly used in new public management settings does not do justice to the trade of diplomacy as a distinctive profession. This article argues that when evaluating diplomatic interventions in multilateral settings the process-driven …

Peace and Conflict Impact Assessment (PCIA) in Community Development: A Case …

Peace and conflict impact assessment (PCIA) is a tool that potentially can improve the quality of development work in conflict zones. PCIA’s conceptual strengths and weaknesses are much debated but few studies to date have examined its use in practice. For this article, PCIA was used to structure research on conflict and peace dynamics in post-war Mozambique. The findings address both local pea…

A Developing Evaluation Culture in Romania: Myths, Gaps and Triggers

Most of the efforts undertaken to build national evaluation capacity in Romania are directed towards institutionalizing evaluation. These efforts might lead to the establishment of a national evaluation system. But a national evaluation culture — not easily created by regulations, formal tools and institutional hierarchies alone — is nevertheless emerging and is already having an impact on the …

Dealing with Distrust and Power Dynamics: Asymmetric Relations among Stakehol…

Asymmetric relations among stakeholders create challenges in participatory evaluation processes. Power and conflict may hinder equal and genuine communication about the value of the practices evaluated. Guidelines to deal with these sociopolitical challenges are scarce and usually focus on the need for evaluators to remain distant or to handle resistance. In this article we discuss the use of a…

Evaluation of Legislation: Skating on Thin Ice

While parliaments and politicians increasingly call for the evaluation of legislation, the methodological implications of statutes as an evaluand have not received much interest. This article explores the challenges and methods of evaluating statutes. It describes the methodology and the costs of eight all-encompassing evaluations of statutes at Swiss federal level. It then presents approaches …

Four Waves of Evaluation Diffusion

This article investigates the dissemination of evaluation as it appears from a Swedish and to a lesser extent an Atlantic vantage point since 1960. Four waves have deposited sediments, which form present-day evaluative activities. The scientific wave entailed that academics should test, through two-group experimentation, appropriate means to reach externally set, admittedly subjective, goals. P…

Criteria and Theory in the Evaluation of Organizations

This article argues that evaluation-specific logic, which involves the identification of relevant criteria and performance standards for the evaluand (considered as a whole or in its single dimensions), can benefit from research on the causal chains leading from programme inputs to the final outcomes; and on the contexts and mechanisms responsible, which are normally investigated when adopting …

Evaluating Development Interventions in Peace-Precarious Situations

International development assistance tackles sociopolitical and socioeconomic problems, typically with formal host government and population buy-in (or acquiescence), in settings with complex interrelated challenges. A new subset of contemporary interventions, however, faces all of the usual development challenges plus security threats affecting foreign and local staff, host country leadership,…

Involving Developing Countries in the Evaluation of Development Cooperation: …

Joint evaluation of development cooperation is becoming more imperative within the framework of the Paris Declaration. It has, however, been noted that the documentation of practical experience with evaluations conducted in a joint fashion is missing. This article seeks to contribute to the debate on the practical implications of conducting joint evaluations. The model adopted involves one dono…

Evaluation of Cluster Policy: A Methodological Overview

Cluster policy is increasingly becoming part of many governments’ economic policy strategies. At the same time, evidence-based policy-making is gaining importance, bringing about a call for policy evaluation. Since the quality of the evaluation results depends highly on the method used, data, assumptions and techniques must be adequate for the specific evaluation question. This holds for cluste…

Peer Evaluation and its Blurred Boundaries: Results from a Meta-evaluation in…

Peer evaluation has proved to be a popular approach with both peers and the evaluated, but there has been considerable variation in the ways in which peer evaluations have been implemented. There are different forms, purposes and ways of organizing peer evaluation. Peer evaluations are also not uniform and links to other evaluations can be made. After providing a general overview of different k…

Unpacking the Politics of Evaluation: A Dramaturgical Analysis

This article draws on a four-year evaluation that assessed the delivery of support services by 15 British hospices and social agencies to family carers of terminally ill people. It aims to examine the politics of evaluation research. Three main arguments are posited: first, that evaluation research is distinguishable from ‘quick and dirty’ evaluations, which are insufficiently resourced and not…

To Be or Not to Be a Profession: Pros, Cons and Challenges for Evaluation

Debates on the professionalization of evaluation regularly fuel controversies. Evaluation literature contains varied points of view in favour of or against means of restricting access to the profession and quality control mechanisms. This article examines the aims pursued (e.g. institutionalization, quality improvement, ethical practice) and challenges faced by the promoters of the professional…

What Doesn’t Work? Three Failures, Many Answers

Current debates on impact evaluation have addressed the question ‘what works and what doesn’t?’ mainly focussing on methodology failures when providing evidence of impact. In order to answer that question, this article contrasts different approaches to evaluation in terms of the way they address different kinds of possible failures. First, there is more to be debated than simply methodological …

Gender Similarities and Differences in the Treatment, Relapse, and Recovery C…

This study explores the influence of gender on changes in recovery status among participants in a longitudinal study. The study sample (N = 1,202; 60% female) is recruited on referral to treatment, and annual interviews are conducted from Years 2 to 6 following intake. At each annual observation, participants are classified into one of four statuses (recovery, treatment, incarcerated, and using…

Longitudinal Research That Can Inform Dynamic Models for the Treatment of Add…

Edition

Vol. 32, No. 1, February 2008. Page 3 - 6

ISBN/ISSN

0193841X

Collation

-

Series Title

Evaluation Review

Call Number

-

Patterns of Crime and Drug Use Trajectories in Relation to Treatment Initiati…

Drug abusers vary considerably in their drug use and criminal behavior over time, and these trajectories are likely to influence drug treatment participation and treatment outcomes. Drawing on longitudinal natural history data from three samples of adult male drug users, we identify four groups with distinctive drug use and crime trajectories during the 5 years prior to their first treatment ep…