Based on the example of the evaluation of service organization models, this article shows how a configurational approach overcomes the limits of traditional methods which for the most part have studied the individual components of various models considered independently of one another. These traditional methods have led to results (observed effects) that are difficult to interpret. The configur…
Laws are important vehicles for policy. They are generally complex and involve various interventions. Despite their pervasive presence and numerous evaluation studies, laws have not been a topic of meta-analytical interest among social scientists. As a result, we lack an overall picture of the type(s) of interventions involved in laws, of the chains of events these are expected to set off and o…
At the outset of a policy evaluation, it is often necessary to gather data from the few personnel charged with responsibility for the policy in order to better understand the policy and its objectives. The objective hermeneutics method provides a tool for deriving the maximum benefit from several core text sequences of such interviews. Two examples of interviews with administrative officials ar…
Comparative performance evaluation has taken different forms depending upon the purposes of performance monitoring and the types of measures available. This paper investigates the different performance measurement systems in place in the social care setting, in particular for older people receiving community care services. In England, earlier systems to assist performance management within orga…
International comparison of performance has become an influential lever for change in the provision of public services. For health care, patients’ views and opinions are increasingly being recognized as legitimate means for assessing the provision of services, to stimulate quality improvements, and more recently, in evaluating system performance. This has shifted the focus of analyses towards t…
The Netherlands and England are near neighbours whose health care systems have much in common and whose health policy communities have also usually been well aware of what is going on in the other country. Nevertheless, for the two decades from 1982, England adopted and repeatedly redeveloped performance indicator (PI) systems in the health care field while the Netherlands virtually shunned the…
This article discusses a comparative study of how local actors tackle health inequalities in England, Scotland and Wales. The main method used in this study was a thematic analysis of 200 interview transcripts. Its focus was on how health inequalities are framed for intervention by performance assessment systems and the challenge for these systems that their nature as a ‘wicked issue’ presents.…
Over the last decade, external assessments of organizational effectiveness have played an increasingly pivotal role in the regulation of UK local authorities. Unlike most approaches to evaluation, which are concerned with the implementation and outcomes of policies, programmes or interventions, these assessments are concerned primarily with internal organizational processes. And in contrast to …
This article presents and discusses five challenges encountered in conducting a knowledge synthesis on primary healthcare, commissioned by the Canadian Health Services Research Foundation. These challenges are (1) conceptualizing, defining and operationalizing complex interventions; (2) integrating quantitative and qualitative studies and assessing strength of evidence; (3) incorporating expert…
A debate on approaches to impact evaluation has raged in development circles in recent years. This paper makes a contribution to this debate through discussion of four issues. First, I point out that there are two definitions of impact evaluation. Neither is right or wrong, but they refer to completely different things. There is no point in methodological debates unless they agree a common star…
This article proposes a methodology to assess the efficiency of the European Union Structural Funds (SF) operations using two alternative approaches: first, by a unit cost analysis of the programme’s output indicators; second, by the comparative study of the tendering, contract-awarding and implementation costs of a sample of projects. This methodology has been applied to the mid-term evaluatio…
The purpose of this article is to discuss what is meant by multicultural competence in evaluation and how policies and programmes aiming at multicultural awareness and ‘validity’ can be evaluated. The article discusses three main ways of understanding multiculturalism and how multicultural competence in evaluation can be defined. It also develops evaluation criteria that can be used for assessi…
This article presents a model to evaluate national diplomatic interventions in multilateral decision-making arenas. The performance-driven evaluation model that is commonly used in new public management settings does not do justice to the trade of diplomacy as a distinctive profession. This article argues that when evaluating diplomatic interventions in multilateral settings the process-driven …
Peace and conflict impact assessment (PCIA) is a tool that potentially can improve the quality of development work in conflict zones. PCIA’s conceptual strengths and weaknesses are much debated but few studies to date have examined its use in practice. For this article, PCIA was used to structure research on conflict and peace dynamics in post-war Mozambique. The findings address both local pea…
Most of the efforts undertaken to build national evaluation capacity in Romania are directed towards institutionalizing evaluation. These efforts might lead to the establishment of a national evaluation system. But a national evaluation culture — not easily created by regulations, formal tools and institutional hierarchies alone — is nevertheless emerging and is already having an impact on the …
Asymmetric relations among stakeholders create challenges in participatory evaluation processes. Power and conflict may hinder equal and genuine communication about the value of the practices evaluated. Guidelines to deal with these sociopolitical challenges are scarce and usually focus on the need for evaluators to remain distant or to handle resistance. In this article we discuss the use of a…
While parliaments and politicians increasingly call for the evaluation of legislation, the methodological implications of statutes as an evaluand have not received much interest. This article explores the challenges and methods of evaluating statutes. It describes the methodology and the costs of eight all-encompassing evaluations of statutes at Swiss federal level. It then presents approaches …
This article investigates the dissemination of evaluation as it appears from a Swedish and to a lesser extent an Atlantic vantage point since 1960. Four waves have deposited sediments, which form present-day evaluative activities. The scientific wave entailed that academics should test, through two-group experimentation, appropriate means to reach externally set, admittedly subjective, goals. P…
This article argues that evaluation-specific logic, which involves the identification of relevant criteria and performance standards for the evaluand (considered as a whole or in its single dimensions), can benefit from research on the causal chains leading from programme inputs to the final outcomes; and on the contexts and mechanisms responsible, which are normally investigated when adopting …
International development assistance tackles sociopolitical and socioeconomic problems, typically with formal host government and population buy-in (or acquiescence), in settings with complex interrelated challenges. A new subset of contemporary interventions, however, faces all of the usual development challenges plus security threats affecting foreign and local staff, host country leadership,…