This article is a case study of the emergence of an evaluation culture in the public sector and particularly in education in Ireland. It suggests that the emergence of this culture was strongly influenced by external bodies, particularly the EU and, to a lesser but significant degree, the OECD. It is further argued that the continuation of systematic evaluation is still dependent on external fo…
Evidence-based evaluation of adult learning (training and capacity development) programs is essential for guiding investment decisions regarding such programs. But standard practice for evaluation of training and development programs relies on participants' self-reports regarding the quality, effectiveness, and impact of such training. More rigorous evaluation methods — including reliable measu…
Embedded in the critical realist tradition, this article aims to explore the potentialities of the case study for evaluation purposes when complexity and specificity are moderate.Three objectives are pursued. First, it is stated that the focus of the evaluation effort is not necessarily the programme itself, but can be one premise upon which the programme is built. Second, taking into account D…
This article provides some reflective thoughts on current research on regulatory impact assessment (RIA) in Europe.The narrative draws on the author's fieldwork diaries. It starts from the methodological problems of identifying and analysing regulatory impact assessment in the UK, Denmark, the Netherlands and Sweden.This leads to more fundamental theoretical questions about research design, the…
The objective was to assess the impact of new guidance on the conduct of narrative synthesis in systematic reviews of effectiveness, by means of a blinded comparison of guidance-led narrative synthesis against a meta-analysis of the same study data.The conclusions of the two syntheses were broadly similar. However, differences between the approaches meant that conclusions about the impact of mo…
This article examines the relationship between the gross and net employment outcomes from the largest EU structural funding programme in the UK, the Objective 1 West Wales and the Valleys programme. The article demonstrates the practical difficulties of moving from gross to net outputs. It is suggested that these difficulties are not always explicitly considered in EU programme evaluations. One…
In Europe, the aim of achieving a knowledge-based society makes the quality of training systems a topic of crucial importance. Italy is also interested in working out strategies and tools for improving its training supply. To this end the Ministry of Labour asked a public research institute, the Institute for the Development of Workers' Vocational Training (ISFOL), to design a self-assessment t…
Despite the rich literature on networks, publications on the evaluation of networks are scarce. The aim of the article is to present the core concepts of network evaluation in the context of local and regional development, and a case study in Finnish North-Karelia. It is argued that network evaluation from the everyday life perspective (NEELP) is a special case. It requires an integrative desig…
Models that shift more responsibility onto researchers for the process of incorporating research results into decision-making have greatly gained in popularity during the past two decades. This shift has created a new area of research to identify the best ways to transfer academic results into the organizational and political arenas. However, evaluating the utilization of information coming out…
The evidence movement and the idea of systematic reviews, defined as summaries of the results of already existing evaluation and research projects, have gained considerable support in recent years as many international as well as national evidence-producing organizations have been established. This article analysis how the idea is practiced in the areas of health, social welfare and education a…
Approaches to participation in evaluation rely on the principle of active participation by major stakeholders, including less organized groups, as fundamental to good evaluation practice. This process offers a number of advantages but implementation requires certain prerequisites. The goal of our article is to weigh up the advantages of participation and to examine the conditions necessary for …
The current debate around the emergence of participatory approaches in evaluation practice suggests that participatory evaluation may be considered an organizational learning praxis, one which facilitates the development of a holistic process of intentional change. Through critical reflection on how participatory evaluation has been conceptualized, this article offers an overview of some of the…
The main function of program evaluations is to describe programs in order to generate judgments of value. To be considered credible, judgments should be both legitimate and justified. The research presented in this article posed the following question: do program evaluation practitioners generate legitimate and justified judgments? A meta-analysis of 40 program evaluation reports was carried ou…
Among parliamentary democracies there is a widespread belief that above and beyond the occasional opportunity to vote, citizens should be allowed to participate in decisions that affect them. Governments at all levels are now going further and supporting more active forms of citizenship in which various decision processes are open to more public participation. While this principle may be widely…
This article presents a case study of engaging stakeholders in the early stages of an impact evaluation of educational development work in a UK university. The rationale for undertaking participative impact evaluation is outlined in relation to the national and local context. The aim is to contribute to wider knowledge about appropriate methodology to lead to a better understanding of change pr…
In the context of a growing evaluation community in Romania, networking in evaluation tends to be not only a topic for analysis and research, but mainly a practical tool for reaching the generic community of evaluation practitioners. Furthermore, this is a process requiring resources (time, dedicated people, information and funding) which are scarce. Therefore the better documented the process …
In this article we develop a ‘Stakeholder Mapping’ approach to ex-ante policy evaluation. The proposed framework helps to assess policy implementation activities by applying several tests to evaluate the completeness, non-redundancy, proper assignment and internal consistency of a policy design. We illustrate the method by applying it to the analysis of e-commerce policy in Iran. We conclude by…
Spain’s entry into the European Union (EU), in 1986, represented a great challenge in several areas, including the evaluation of public policy. In 1988 Structural Fund regulations were amended to include mandatory evaluation of European structural policies by both the Commission and the member states, and the 1999 reform strengthened evaluation requirements. This contribution discusses the majo…
Based on the example of the evaluation of service organization models, this article shows how a configurational approach overcomes the limits of traditional methods which for the most part have studied the individual components of various models considered independently of one another. These traditional methods have led to results (observed effects) that are difficult to interpret. The configur…
Laws are important vehicles for policy. They are generally complex and involve various interventions. Despite their pervasive presence and numerous evaluation studies, laws have not been a topic of meta-analytical interest among social scientists. As a result, we lack an overall picture of the type(s) of interventions involved in laws, of the chains of events these are expected to set off and o…