Gathering time-series data of behaviors and psychological variables is important to understand, guide, and evaluate behavior-change campaigns and other change processes. However, repeated measurement can affect the phenomena investigated, particularly frequent face-to-face interviews, which are often the only option in developing countries. This article presents three intervention control studi…
In panel studies on sensitive topics, respondent-generated identification codes are often used to link records across surveys. However, usually a substantial number of cases are lost due to the codes. These losses may cause biased estimates. Using more components and linking the codes by the Levenshtein string distance function will reduce the losses. In a simulation study and two field experim…
This article describes and discusses a case of deliberative responsive stakeholder evaluation that was employed as a strategy to support the development of an undergraduate medical programme at a Swedish university. It focuses on received information and reflection about the process of evaluation as a dialogue for learning. By highlighting the complexity of educational work and the ex…
This article describes an evaluation conducted in a region of northern Italy, based on the realistic evaluation approach.This case study is related to the application at regional level of national legislative measures against personal drug consumption, the so-called Article 75, through the implementation of a pilot programme. Article 75 DPR 309/90, is a `sanctions-based' intervention main…
This article proposes ways to use programme theory for evaluating aspects of programmes that are complicated or complex. It argues that there are useful distinctions to be drawn between aspects that are complicated and those that are complex, and provides examples of programme theory evaluations that have usefully represented and address both of these. While complexity has been defined in…
This article reports on the structured literature survey methodology, which was used to identify how and at what stage users of medical devices are engaged in the medical device technology cycle, including methods and tools for evaluating device outcomes.This was not a conventional systematic review of the literature. The stated purpose of the survey is broader than synthesizing best evi…
Complex, area-based initiatives are a widely adopted policy response to problems of health inequalities and social exclusion in industrialized countries such as the UK and the US.This article explores the problems of evaluating such initiatives in a context of rapid national and local policy change by drawing on the Children's Fund, a UK government initiative that established local partne…
This article examines the relationships of the National Audit Office (NAO) — the state audit institution (SAI) of the United Kingdom — with a range of third parties that shape the performance audit work the NAO undertakes. In particular, it considers from a practitioner perspective how the NAO has sought to balance its independence with the desire to be responsive to the expectations of o…
The US federalist system has been at the origin of evaluation in many ways: providing a laboratory for experimentation of innovative policies, and requiring evaluation of the growing federal intervention from the War on Poverty onwards. Evaluation approaches have been developed that took into consideration the multiple forms of collaboration activated by the federalist system.This experienc…
This article focuses on what some sceptics see as disillusionment with conventional evaluation practice, in that many governments experience only limited use of evaluation findings.This has contributed to a significant increase in results-based performance measurement. Yet not everyone in the evaluation community welcomes this development.The authors make the point that significant compleme…
This article, grounded in evaluation research on intermunicipal cooperation, contends that processes of change in local government present specific challenges for evaluation. In the course of data collection, researchers encountered a phenomenon that they called `the rhetorical wall', where there is a wide gap between intentions as expressed at the rhetorical level and what actually occurs.T…
Over the past 20 years, the development industry has changed rapidly and now presents evaluators with a series of new challenges.This article considers this process of change. It argues that there has been a move away from a `project-based' approach to a broader vision of development centred on the Millennium Development Goals, which has led to a change of the evaluation approach: from an ass…
Increasingly, in the world of evaluation, `systems of evaluation' have been developed.This article outlines four criteria that help characterize such systems. One criterion is the existence of a distinctive epistemological perspective; another is that, in order to be labelled a system, evaluation activities are carried out by evaluators within organizational structures and institutions and n…
This article discusses the importance of social relations in evaluation. It focuses first on social relations among stakeholders in the evaluation setting. It is argued that social relations are crucial as an object of study, since they are integral to and partly constitutive of the quality and effectiveness of the programme being evaluated. Next, it focuses on the social relations between th…
Impact studies of prevention programmes, in particular meta-analyses, usually interpret outcome and impact statistics as tests of an underlying theory of prevention. However, these programmes usually combine various interventions linked to different theoretical perspectives. Consequently, the effects of the programme can easily be misinterpreted.This article introduces an interpretation method …
This article uses a sociopolitical perspective to analyse the results of a case study of six Canadian Health Technology Assessment (HTA) agencies and their stakeholders: health care administrators, provider associations, patient associations and the biomedical industry. A total of 40 HTA agency representatives and 46 stakeholders were interviewed.A self-administered survey was also filled out b…
While evaluators have considered accountability pressures and examined the consequences of performance measurement for organizations, they have paid less attention to the accountability relationship in which their work is used. Drawing on theoretical literature and empirical data, this article shows how these accountability relationships drive the use of evaluative frameworks, like performance …
This article focuses on negotiation and discusses its relevance for evaluators. Given the impetus for participatory evaluation, evaluators would benefit from improving skills that enable them to make collaborative decisions and work alongside stakeholders, in particular in community evaluations. Negotiation skills are explored through post hoc reflection of a Sure Start programme evaluation in …
The realist evaluation (RE) approach is a promising contribution to the literature on theory-driven evaluation.Thus far, RE has mainly been applied to the evaluation of human service programmes, but the aim of this article is to investigate if it can be extended to large-scale public sector reforms, in casu: the reform of the electricity market in Denmark. Five areas of regulation under the ref…
Partnerships are often considered an alternative way to deliver programs provided by governments and organizations (potentially) more cost effectively. However, this assumption needs to be verified. Evaluators and auditors now face a challenge: how to assess the performance of this hybrid organizational form.This article suggests one powerful way of evaluating partnerships: transaction cost eco…