Over the last decade, external assessments of organizational effectiveness have played an increasingly pivotal role in the regulation of UK local authorities. Unlike most approaches to evaluation, which are concerned with the implementation and outcomes of policies, programmes or interventions, these assessments are concerned primarily with internal organizational processes. And in contrast to …
This article presents and discusses five challenges encountered in conducting a knowledge synthesis on primary healthcare, commissioned by the Canadian Health Services Research Foundation. These challenges are (1) conceptualizing, defining and operationalizing complex interventions; (2) integrating quantitative and qualitative studies and assessing strength of evidence; (3) incorporating expert…
A debate on approaches to impact evaluation has raged in development circles in recent years. This paper makes a contribution to this debate through discussion of four issues. First, I point out that there are two definitions of impact evaluation. Neither is right or wrong, but they refer to completely different things. There is no point in methodological debates unless they agree a common star…
This article proposes a methodology to assess the efficiency of the European Union Structural Funds (SF) operations using two alternative approaches: first, by a unit cost analysis of the programme’s output indicators; second, by the comparative study of the tendering, contract-awarding and implementation costs of a sample of projects. This methodology has been applied to the mid-term evaluatio…
The purpose of this article is to discuss what is meant by multicultural competence in evaluation and how policies and programmes aiming at multicultural awareness and ‘validity’ can be evaluated. The article discusses three main ways of understanding multiculturalism and how multicultural competence in evaluation can be defined. It also develops evaluation criteria that can be used for assessi…
This article presents a model to evaluate national diplomatic interventions in multilateral decision-making arenas. The performance-driven evaluation model that is commonly used in new public management settings does not do justice to the trade of diplomacy as a distinctive profession. This article argues that when evaluating diplomatic interventions in multilateral settings the process-driven …
Peace and conflict impact assessment (PCIA) is a tool that potentially can improve the quality of development work in conflict zones. PCIA’s conceptual strengths and weaknesses are much debated but few studies to date have examined its use in practice. For this article, PCIA was used to structure research on conflict and peace dynamics in post-war Mozambique. The findings address both local pea…
Most of the efforts undertaken to build national evaluation capacity in Romania are directed towards institutionalizing evaluation. These efforts might lead to the establishment of a national evaluation system. But a national evaluation culture — not easily created by regulations, formal tools and institutional hierarchies alone — is nevertheless emerging and is already having an impact on the …
Asymmetric relations among stakeholders create challenges in participatory evaluation processes. Power and conflict may hinder equal and genuine communication about the value of the practices evaluated. Guidelines to deal with these sociopolitical challenges are scarce and usually focus on the need for evaluators to remain distant or to handle resistance. In this article we discuss the use of a…
While parliaments and politicians increasingly call for the evaluation of legislation, the methodological implications of statutes as an evaluand have not received much interest. This article explores the challenges and methods of evaluating statutes. It describes the methodology and the costs of eight all-encompassing evaluations of statutes at Swiss federal level. It then presents approaches …
This article investigates the dissemination of evaluation as it appears from a Swedish and to a lesser extent an Atlantic vantage point since 1960. Four waves have deposited sediments, which form present-day evaluative activities. The scientific wave entailed that academics should test, through two-group experimentation, appropriate means to reach externally set, admittedly subjective, goals. P…
This article argues that evaluation-specific logic, which involves the identification of relevant criteria and performance standards for the evaluand (considered as a whole or in its single dimensions), can benefit from research on the causal chains leading from programme inputs to the final outcomes; and on the contexts and mechanisms responsible, which are normally investigated when adopting …
International development assistance tackles sociopolitical and socioeconomic problems, typically with formal host government and population buy-in (or acquiescence), in settings with complex interrelated challenges. A new subset of contemporary interventions, however, faces all of the usual development challenges plus security threats affecting foreign and local staff, host country leadership,…
Joint evaluation of development cooperation is becoming more imperative within the framework of the Paris Declaration. It has, however, been noted that the documentation of practical experience with evaluations conducted in a joint fashion is missing. This article seeks to contribute to the debate on the practical implications of conducting joint evaluations. The model adopted involves one dono…
Cluster policy is increasingly becoming part of many governments’ economic policy strategies. At the same time, evidence-based policy-making is gaining importance, bringing about a call for policy evaluation. Since the quality of the evaluation results depends highly on the method used, data, assumptions and techniques must be adequate for the specific evaluation question. This holds for cluste…
Peer evaluation has proved to be a popular approach with both peers and the evaluated, but there has been considerable variation in the ways in which peer evaluations have been implemented. There are different forms, purposes and ways of organizing peer evaluation. Peer evaluations are also not uniform and links to other evaluations can be made. After providing a general overview of different k…
This article draws on a four-year evaluation that assessed the delivery of support services by 15 British hospices and social agencies to family carers of terminally ill people. It aims to examine the politics of evaluation research. Three main arguments are posited: first, that evaluation research is distinguishable from ‘quick and dirty’ evaluations, which are insufficiently resourced and not…
Debates on the professionalization of evaluation regularly fuel controversies. Evaluation literature contains varied points of view in favour of or against means of restricting access to the profession and quality control mechanisms. This article examines the aims pursued (e.g. institutionalization, quality improvement, ethical practice) and challenges faced by the promoters of the professional…
Current debates on impact evaluation have addressed the question ‘what works and what doesn’t?’ mainly focussing on methodology failures when providing evidence of impact. In order to answer that question, this article contrasts different approaches to evaluation in terms of the way they address different kinds of possible failures. First, there is more to be debated than simply methodological …
This study explores the influence of gender on changes in recovery status among participants in a longitudinal study. The study sample (N = 1,202; 60% female) is recruited on referral to treatment, and annual interviews are conducted from Years 2 to 6 following intake. At each annual observation, participants are classified into one of four statuses (recovery, treatment, incarcerated, and using…