Over the past 20 years, the development industry has changed rapidly and now presents evaluators with a series of new challenges.This article considers this process of change. It argues that there has been a move away from a `project-based' approach to a broader vision of development centred on the Millennium Development Goals, which has led to a change of the evaluation approach: from an ass…
Increasingly, in the world of evaluation, `systems of evaluation' have been developed.This article outlines four criteria that help characterize such systems. One criterion is the existence of a distinctive epistemological perspective; another is that, in order to be labelled a system, evaluation activities are carried out by evaluators within organizational structures and institutions and n…
This article discusses the importance of social relations in evaluation. It focuses first on social relations among stakeholders in the evaluation setting. It is argued that social relations are crucial as an object of study, since they are integral to and partly constitutive of the quality and effectiveness of the programme being evaluated. Next, it focuses on the social relations between th…
Impact studies of prevention programmes, in particular meta-analyses, usually interpret outcome and impact statistics as tests of an underlying theory of prevention. However, these programmes usually combine various interventions linked to different theoretical perspectives. Consequently, the effects of the programme can easily be misinterpreted.This article introduces an interpretation method …
This article uses a sociopolitical perspective to analyse the results of a case study of six Canadian Health Technology Assessment (HTA) agencies and their stakeholders: health care administrators, provider associations, patient associations and the biomedical industry. A total of 40 HTA agency representatives and 46 stakeholders were interviewed.A self-administered survey was also filled out b…
While evaluators have considered accountability pressures and examined the consequences of performance measurement for organizations, they have paid less attention to the accountability relationship in which their work is used. Drawing on theoretical literature and empirical data, this article shows how these accountability relationships drive the use of evaluative frameworks, like performance …
This article focuses on negotiation and discusses its relevance for evaluators. Given the impetus for participatory evaluation, evaluators would benefit from improving skills that enable them to make collaborative decisions and work alongside stakeholders, in particular in community evaluations. Negotiation skills are explored through post hoc reflection of a Sure Start programme evaluation in …
The realist evaluation (RE) approach is a promising contribution to the literature on theory-driven evaluation.Thus far, RE has mainly been applied to the evaluation of human service programmes, but the aim of this article is to investigate if it can be extended to large-scale public sector reforms, in casu: the reform of the electricity market in Denmark. Five areas of regulation under the ref…
Partnerships are often considered an alternative way to deliver programs provided by governments and organizations (potentially) more cost effectively. However, this assumption needs to be verified. Evaluators and auditors now face a challenge: how to assess the performance of this hybrid organizational form.This article suggests one powerful way of evaluating partnerships: transaction cost eco…
This article argues that complexity theory has the potential to bring important insights in reframing of the role and practice of evaluation but that the utility of complexity theory needs to be developed to support its application in evaluation research.This article focuses on the implications of a reformed relationship between theory and the empirical setting for what we can learn in evaluati…
Advocacy has become an important area of development support. Simultaneously, the interest in learning-oriented monitoring of advocacy programmes has increased. Starting from the premise that learning has sociopolitical dimensions, this article explores how the challenges and contradictions of such monitoring present themselves in Latin American advocacy activities supported by a Danish NGO…
Based on our experience in helping design, implement, and evaluate a non-evidence-based program, we explore a non-conventional fidelity and adaptation relationship and its implications for program evaluation.We find that in non-evidence-based interventions, program fidelity and adaptation may not be competitors but serve to complement each other.This non-conventional relationship leads to ex…
Ever increasing demands are being made on welfare organizations to display efficiency. Evaluation workshops constitute a form of learning for the purposes of building up competence to conduct evaluations within welfare organizations, with the support of research and development units. In workshops of this kind, welfare work professionals meet in order to conduct evaluations together with resear…
This contribution portrays the institutionalization and practice of evaluation in Switzerland.The main actors and patterns of evaluation development patterns differ from those presented so far in relation to other countries. This raises questions regarding the relevance of the theories previously advanced.The entry points of evaluation into the political process are described.As suggested by ot…
Recent developments in innovation theory and policy have led policy makers to assign particular importance to supporting networks of cooperation among heterogeneous economic actors, especially in production systems composed of small and medium enterprises. Such innovative policies call for parallel innovations in policy analysis, monitoring and assessment. Our analysis of a policy experiment ai…
This article is a case study of the emergence of an evaluation culture in the public sector and particularly in education in Ireland. It suggests that the emergence of this culture was strongly influenced by external bodies, particularly the EU and, to a lesser but significant degree, the OECD. It is further argued that the continuation of systematic evaluation is still dependent on external fo…
Evidence-based evaluation of adult learning (training and capacity development) programs is essential for guiding investment decisions regarding such programs. But standard practice for evaluation of training and development programs relies on participants' self-reports regarding the quality, effectiveness, and impact of such training. More rigorous evaluation methods — including reliable measu…
Embedded in the critical realist tradition, this article aims to explore the potentialities of the case study for evaluation purposes when complexity and specificity are moderate.Three objectives are pursued. First, it is stated that the focus of the evaluation effort is not necessarily the programme itself, but can be one premise upon which the programme is built. Second, taking into account D…
This article provides some reflective thoughts on current research on regulatory impact assessment (RIA) in Europe.The narrative draws on the author's fieldwork diaries. It starts from the methodological problems of identifying and analysing regulatory impact assessment in the UK, Denmark, the Netherlands and Sweden.This leads to more fundamental theoretical questions about research design, the…
The objective was to assess the impact of new guidance on the conduct of narrative synthesis in systematic reviews of effectiveness, by means of a blinded comparison of guidance-led narrative synthesis against a meta-analysis of the same study data.The conclusions of the two syntheses were broadly similar. However, differences between the approaches meant that conclusions about the impact of mo…