Peer Evaluation and its Blurred Boundaries: Results from a Meta-evaluation in…

Peer evaluation has proved to be a popular approach with both peers and the evaluated, but there has been considerable variation in the ways in which peer evaluations have been implemented. There are different forms, purposes and ways of organizing peer evaluation. Peer evaluations are also not uniform and links to other evaluations can be made. After providing a general overview of different k…

Unpacking the Politics of Evaluation: A Dramaturgical Analysis

This article draws on a four-year evaluation that assessed the delivery of support services by 15 British hospices and social agencies to family carers of terminally ill people. It aims to examine the politics of evaluation research. Three main arguments are posited: first, that evaluation research is distinguishable from ‘quick and dirty’ evaluations, which are insufficiently resourced and not…

To Be or Not to Be a Profession: Pros, Cons and Challenges for Evaluation

Debates on the professionalization of evaluation regularly fuel controversies. Evaluation literature contains varied points of view in favour of or against means of restricting access to the profession and quality control mechanisms. This article examines the aims pursued (e.g. institutionalization, quality improvement, ethical practice) and challenges faced by the promoters of the professional…

What Doesn’t Work? Three Failures, Many Answers

Current debates on impact evaluation have addressed the question ‘what works and what doesn’t?’ mainly focussing on methodology failures when providing evidence of impact. In order to answer that question, this article contrasts different approaches to evaluation in terms of the way they address different kinds of possible failures. First, there is more to be debated than simply methodological …

Gender Similarities and Differences in the Treatment, Relapse, and Recovery C…

This study explores the influence of gender on changes in recovery status among participants in a longitudinal study. The study sample (N = 1,202; 60% female) is recruited on referral to treatment, and annual interviews are conducted from Years 2 to 6 following intake. At each annual observation, participants are classified into one of four statuses (recovery, treatment, incarcerated, and using…

Longitudinal Research That Can Inform Dynamic Models for the Treatment of Add…

Edition

Vol. 32, No. 1, February 2008. Page 3 - 6

ISBN/ISSN

0193841X

Collation

-

Series Title

Evaluation Review

Call Number

-

Patterns of Crime and Drug Use Trajectories in Relation to Treatment Initiati…

Drug abusers vary considerably in their drug use and criminal behavior over time, and these trajectories are likely to influence drug treatment participation and treatment outcomes. Drawing on longitudinal natural history data from three samples of adult male drug users, we identify four groups with distinctive drug use and crime trajectories during the 5 years prior to their first treatment ep…

Longitudinal HIV Risk Behavior Among the Drug Abuse Treatment Outcome Studies…

Longitudinal trajectories for HIV risk were examined over 5 years following treatment among 1,393 patients who participated in the nationwide Drug Abuse Treatment Outcome Studies. Both injection drug use and sexual risk behavior declined over time, with most of the decline occurring between intake and the first-year follow-up. However, results of the application of growth mixture models for bot…

Nine-Year Psychiatric Trajectories and Substance Use Outcomes: An Application…

This study identifies longitudinal psychiatric trajectories of 934 adult individuals entering chemical dependency treatment in a private, managed care health plan and examines the relationship of these trajectories with substance use (SU) outcomes. The authors apply a group-based modeling approach to identify trajectory groups based on repeated measures of psychiatric severity for 9 years and i…

The Interaction of Co-Occurring Mental Disorders and Recovery Management Chec…

This article examines the effectiveness of quarterly Recovery Management Checkups (RMCs) for people with substance disorders by level of co-occurring mental disorders (34% none, 27% internalizing disorders, and 39% internalizing and externalizing) across two randomized experiments with 92% to 97% follow-up. The 865 participants are 82% African American, 53% female, and age 37 on average. …

Designing a Household Survey to Address Seasonality in Child Care Arrangements

In household telephone surveys, a long field period may be required to maximize the response rate and achieve adequate sample sizes. However, long field periods can be problematic when measures of seasonally affected behavior are sought. Surveys of child care use are one example because child care arrangements vary by season. Options include varying the questions posed about school-year and sum…

The Truncation-by-Death Problem: What To Do in an Experimental Evaluation Whe…

Although experiments are viewed as the gold standard for evaluation, some of their benefits may be lost when, as is common, outcomes are not defined for some sample members. In evaluations of marriage interventions, for example, a key outcome—relationship quality—is undefined when a couple splits up. This article shows how treatment-control differences in mean outcomes can be misleadi…

Mixed Results in a Transitional Planning Program for Alternative School Students

Disciplinary alternative schools have a reputation as gateways to the juvenile and criminal justice systems. The authors conducted an evaluation of an intervention (Strategies for Success) designed to divert seventh-, eighth-, and ninth-grade alternative school students from this gateway. They used propensity score matching and a multivariate random effects model to estimate program impacts and…

Using an Empirical Binomial Hierarchical Bayesian Model as an Alternative to …

This article explores the statistical methodologies used in demonstration and effectiveness studies when the treatments are applied across multiple settings. The importance of evaluating and how to evaluate these types of studies are discussed. As an alternative to standard methodology, the authors of this article offer an empirical binomial hierarchical Bayesian model as a way to effectively e…

Mental Health and Firearms in Community-Based Surveys: Implications for Suici…

Suicide rates are higher among those who own or live in a household with a hand gun. This article examines the association between hand gun ownership and mental health, another risk factor for suicide. Data from the General Social Survey, a series of surveys of U.S. adults, are analyzed to compare general emotional and mental health, sadness and depression, functional mental health, and mental …

The Impact of Child Obesity on Active Parental Consent in School-Based Survey…

Previous studies have shown that active consent procedures result in sampling bias in surveys dealing with adolescent risk behaviors such as cigarette smoking and illicit drug use. To examine sampling bias from active consent procedures when the survey topic pertains to childhood obesity and associated health behaviors, the authors pair data obtained from both active and passive consent p…

Rates of Missing Responses in Personal Digital Assistant (PDA) Versus Paper A…

This article describes rates of missing item responses in personal digital assistant (PDA) assessments as compared to paper assessments. Data come from the evaluation of a classroom-based leisure, life skills, and sexuality education program delivered to high school students in Cape Town, South Africa. Analyses show that the paper assessments had much higher rates of missing-ness than PDA a…

Tightening Income Documentation in a Means-Tested Program: Who Stays Away?

Programs using means tests to identify low-income households face a trade-off between promoting access and ensuring program integrity. The authors use a comparison-district design to estimate the effects of a pilot program to improve the accuracy of the process of certifying students for free or reduced-price meals in the National School Lunch Program. This pilot program required households to …

Active Parental Consent in School-Based Research: How Much Is Enough and How …

Active parental consent policies have been blamed for low participation rates and selection bias (i.e., loss of "high-risk" youths) in school-based studies. In this article, the authors describe active consent procedures that produced an overall active consent rate of 79% in a sample of more than 4,500 middle school students attending 29 schools in seven cities across the United States. Consent…

Weighting Regressions by Propensity Scores

Regressions can be weighted by propensity scores in order to reduce bias. However, weighting is likely to increase random error in the estimates, and to bias the estimated standard errors downward, even when selection mechanisms are well understood. Moreover, in some cases, weighting will increase the bias in estimated causal parameters. If investigators have a good causal model, it seems bette…