Approaches to participation in evaluation rely on the principle of active participation by major stakeholders, including less organized groups, as fundamental to good evaluation practice. This process offers a number of advantages but implementation requires certain prerequisites. The goal of our article is to weigh up the advantages of participation and to examine the conditions necessary for …
The current debate around the emergence of participatory approaches in evaluation practice suggests that participatory evaluation may be considered an organizational learning praxis, one which facilitates the development of a holistic process of intentional change. Through critical reflection on how participatory evaluation has been conceptualized, this article offers an overview of some of the…
The main function of program evaluations is to describe programs in order to generate judgments of value. To be considered credible, judgments should be both legitimate and justified. The research presented in this article posed the following question: do program evaluation practitioners generate legitimate and justified judgments? A meta-analysis of 40 program evaluation reports was carried ou…
Among parliamentary democracies there is a widespread belief that above and beyond the occasional opportunity to vote, citizens should be allowed to participate in decisions that affect them. Governments at all levels are now going further and supporting more active forms of citizenship in which various decision processes are open to more public participation. While this principle may be widely…
This article presents a case study of engaging stakeholders in the early stages of an impact evaluation of educational development work in a UK university. The rationale for undertaking participative impact evaluation is outlined in relation to the national and local context. The aim is to contribute to wider knowledge about appropriate methodology to lead to a better understanding of change pr…
In the context of a growing evaluation community in Romania, networking in evaluation tends to be not only a topic for analysis and research, but mainly a practical tool for reaching the generic community of evaluation practitioners. Furthermore, this is a process requiring resources (time, dedicated people, information and funding) which are scarce. Therefore the better documented the process …
In this article we develop a ‘Stakeholder Mapping’ approach to ex-ante policy evaluation. The proposed framework helps to assess policy implementation activities by applying several tests to evaluate the completeness, non-redundancy, proper assignment and internal consistency of a policy design. We illustrate the method by applying it to the analysis of e-commerce policy in Iran. We conclude by…
Spain’s entry into the European Union (EU), in 1986, represented a great challenge in several areas, including the evaluation of public policy. In 1988 Structural Fund regulations were amended to include mandatory evaluation of European structural policies by both the Commission and the member states, and the 1999 reform strengthened evaluation requirements. This contribution discusses the majo…
Based on the example of the evaluation of service organization models, this article shows how a configurational approach overcomes the limits of traditional methods which for the most part have studied the individual components of various models considered independently of one another. These traditional methods have led to results (observed effects) that are difficult to interpret. The configur…
Laws are important vehicles for policy. They are generally complex and involve various interventions. Despite their pervasive presence and numerous evaluation studies, laws have not been a topic of meta-analytical interest among social scientists. As a result, we lack an overall picture of the type(s) of interventions involved in laws, of the chains of events these are expected to set off and o…
At the outset of a policy evaluation, it is often necessary to gather data from the few personnel charged with responsibility for the policy in order to better understand the policy and its objectives. The objective hermeneutics method provides a tool for deriving the maximum benefit from several core text sequences of such interviews. Two examples of interviews with administrative officials ar…
Comparative performance evaluation has taken different forms depending upon the purposes of performance monitoring and the types of measures available. This paper investigates the different performance measurement systems in place in the social care setting, in particular for older people receiving community care services. In England, earlier systems to assist performance management within orga…
International comparison of performance has become an influential lever for change in the provision of public services. For health care, patients’ views and opinions are increasingly being recognized as legitimate means for assessing the provision of services, to stimulate quality improvements, and more recently, in evaluating system performance. This has shifted the focus of analyses towards t…
The Netherlands and England are near neighbours whose health care systems have much in common and whose health policy communities have also usually been well aware of what is going on in the other country. Nevertheless, for the two decades from 1982, England adopted and repeatedly redeveloped performance indicator (PI) systems in the health care field while the Netherlands virtually shunned the…
This article discusses a comparative study of how local actors tackle health inequalities in England, Scotland and Wales. The main method used in this study was a thematic analysis of 200 interview transcripts. Its focus was on how health inequalities are framed for intervention by performance assessment systems and the challenge for these systems that their nature as a ‘wicked issue’ presents.…
Over the last decade, external assessments of organizational effectiveness have played an increasingly pivotal role in the regulation of UK local authorities. Unlike most approaches to evaluation, which are concerned with the implementation and outcomes of policies, programmes or interventions, these assessments are concerned primarily with internal organizational processes. And in contrast to …
This article presents and discusses five challenges encountered in conducting a knowledge synthesis on primary healthcare, commissioned by the Canadian Health Services Research Foundation. These challenges are (1) conceptualizing, defining and operationalizing complex interventions; (2) integrating quantitative and qualitative studies and assessing strength of evidence; (3) incorporating expert…
A debate on approaches to impact evaluation has raged in development circles in recent years. This paper makes a contribution to this debate through discussion of four issues. First, I point out that there are two definitions of impact evaluation. Neither is right or wrong, but they refer to completely different things. There is no point in methodological debates unless they agree a common star…
This article proposes a methodology to assess the efficiency of the European Union Structural Funds (SF) operations using two alternative approaches: first, by a unit cost analysis of the programme’s output indicators; second, by the comparative study of the tendering, contract-awarding and implementation costs of a sample of projects. This methodology has been applied to the mid-term evaluatio…
The purpose of this article is to discuss what is meant by multicultural competence in evaluation and how policies and programmes aiming at multicultural awareness and ‘validity’ can be evaluated. The article discusses three main ways of understanding multiculturalism and how multicultural competence in evaluation can be defined. It also develops evaluation criteria that can be used for assessi…