This article presents a case study of engaging stakeholders in the early stages of an impact evaluation of educational development work in a UK university. The rationale for undertaking participative impact evaluation is outlined in relation to the national and local context. The aim is to contribute to wider knowledge about appropriate methodology to lead to a better understanding of change pr…
In the context of a growing evaluation community in Romania, networking in evaluation tends to be not only a topic for analysis and research, but mainly a practical tool for reaching the generic community of evaluation practitioners. Furthermore, this is a process requiring resources (time, dedicated people, information and funding) which are scarce. Therefore the better documented the process …
In this article we develop a ‘Stakeholder Mapping’ approach to ex-ante policy evaluation. The proposed framework helps to assess policy implementation activities by applying several tests to evaluate the completeness, non-redundancy, proper assignment and internal consistency of a policy design. We illustrate the method by applying it to the analysis of e-commerce policy in Iran. We conclude by…
Spain’s entry into the European Union (EU), in 1986, represented a great challenge in several areas, including the evaluation of public policy. In 1988 Structural Fund regulations were amended to include mandatory evaluation of European structural policies by both the Commission and the member states, and the 1999 reform strengthened evaluation requirements. This contribution discusses the majo…
Based on the example of the evaluation of service organization models, this article shows how a configurational approach overcomes the limits of traditional methods which for the most part have studied the individual components of various models considered independently of one another. These traditional methods have led to results (observed effects) that are difficult to interpret. The configur…
Laws are important vehicles for policy. They are generally complex and involve various interventions. Despite their pervasive presence and numerous evaluation studies, laws have not been a topic of meta-analytical interest among social scientists. As a result, we lack an overall picture of the type(s) of interventions involved in laws, of the chains of events these are expected to set off and o…
At the outset of a policy evaluation, it is often necessary to gather data from the few personnel charged with responsibility for the policy in order to better understand the policy and its objectives. The objective hermeneutics method provides a tool for deriving the maximum benefit from several core text sequences of such interviews. Two examples of interviews with administrative officials ar…
Comparative performance evaluation has taken different forms depending upon the purposes of performance monitoring and the types of measures available. This paper investigates the different performance measurement systems in place in the social care setting, in particular for older people receiving community care services. In England, earlier systems to assist performance management within orga…
International comparison of performance has become an influential lever for change in the provision of public services. For health care, patients’ views and opinions are increasingly being recognized as legitimate means for assessing the provision of services, to stimulate quality improvements, and more recently, in evaluating system performance. This has shifted the focus of analyses towards t…
The Netherlands and England are near neighbours whose health care systems have much in common and whose health policy communities have also usually been well aware of what is going on in the other country. Nevertheless, for the two decades from 1982, England adopted and repeatedly redeveloped performance indicator (PI) systems in the health care field while the Netherlands virtually shunned the…
This article discusses a comparative study of how local actors tackle health inequalities in England, Scotland and Wales. The main method used in this study was a thematic analysis of 200 interview transcripts. Its focus was on how health inequalities are framed for intervention by performance assessment systems and the challenge for these systems that their nature as a ‘wicked issue’ presents.…
Over the last decade, external assessments of organizational effectiveness have played an increasingly pivotal role in the regulation of UK local authorities. Unlike most approaches to evaluation, which are concerned with the implementation and outcomes of policies, programmes or interventions, these assessments are concerned primarily with internal organizational processes. And in contrast to …
This article presents and discusses five challenges encountered in conducting a knowledge synthesis on primary healthcare, commissioned by the Canadian Health Services Research Foundation. These challenges are (1) conceptualizing, defining and operationalizing complex interventions; (2) integrating quantitative and qualitative studies and assessing strength of evidence; (3) incorporating expert…
A debate on approaches to impact evaluation has raged in development circles in recent years. This paper makes a contribution to this debate through discussion of four issues. First, I point out that there are two definitions of impact evaluation. Neither is right or wrong, but they refer to completely different things. There is no point in methodological debates unless they agree a common star…
This article proposes a methodology to assess the efficiency of the European Union Structural Funds (SF) operations using two alternative approaches: first, by a unit cost analysis of the programme’s output indicators; second, by the comparative study of the tendering, contract-awarding and implementation costs of a sample of projects. This methodology has been applied to the mid-term evaluatio…
The purpose of this article is to discuss what is meant by multicultural competence in evaluation and how policies and programmes aiming at multicultural awareness and ‘validity’ can be evaluated. The article discusses three main ways of understanding multiculturalism and how multicultural competence in evaluation can be defined. It also develops evaluation criteria that can be used for assessi…
This article presents a model to evaluate national diplomatic interventions in multilateral decision-making arenas. The performance-driven evaluation model that is commonly used in new public management settings does not do justice to the trade of diplomacy as a distinctive profession. This article argues that when evaluating diplomatic interventions in multilateral settings the process-driven …
Peace and conflict impact assessment (PCIA) is a tool that potentially can improve the quality of development work in conflict zones. PCIA’s conceptual strengths and weaknesses are much debated but few studies to date have examined its use in practice. For this article, PCIA was used to structure research on conflict and peace dynamics in post-war Mozambique. The findings address both local pea…
Most of the efforts undertaken to build national evaluation capacity in Romania are directed towards institutionalizing evaluation. These efforts might lead to the establishment of a national evaluation system. But a national evaluation culture — not easily created by regulations, formal tools and institutional hierarchies alone — is nevertheless emerging and is already having an impact on the …
Asymmetric relations among stakeholders create challenges in participatory evaluation processes. Power and conflict may hinder equal and genuine communication about the value of the practices evaluated. Guidelines to deal with these sociopolitical challenges are scarce and usually focus on the need for evaluators to remain distant or to handle resistance. In this article we discuss the use of a…