Improvement Science in Evaluation: Methods and Uses

While improvement science has experienced a surge of interest over the past 30 years, applications of it are rare in the evaluation literature. This issue promotes the cross-fertilization of ideas, techniques, and tools between evaluation and improvement science. There are at least four areas where this cross-fertilization is particularly relevant: learning from error, examining variation, appreciating context, and focusing on systems change. This volume considers: the conceptual similarities and distinctions between improvement science and evaluation; the intellectual foundations, methods, and tools that collectively comprise improvement science; and case chapters that offer an inspiring review of state-of-the-art improvement science applications. Cutting across all of these applications is a shared grounding in systems thinking, a determination to capture and better understand variation and contextual complexity, as well as a sustained commitment to generative learning about projects and programs—all issues of great concern to evaluators. The issue offers producers and users of evaluations the potential benefits of a closer engagement with improvement science. This is the 153rd issue in the New Directions for Evaluation series from Jossey-Bass. It is an official publication ofthe American Evaluation Association.

Editors’ Notes 7 Christina A. Christie, Moira Inkelas, Sebastian Lemire 1. Understanding the Similarities and Distinctions Between Improvement Science and Evaluation 11 Christina A. Christie, Sebastian Lemire, Moira Inkelas This chapter considers the similarities and distinctions between improvement science and evaluation according to use, valuing, and methods. 2. The Methods and Tools of Improvement Science 23 Sebastian Lemire, Christina A. Christie, Moira Inkelas This chapter introduces the intellectual foundation, core principles, and selected tools of improvement science. 3. Timely and Appropriate Healthcare Access for Newborns: A Neighborhood-Based, Improvement Science Approach 35 Courtney M. Brown, Robert S. Kahn, Neera K. Goyal This chapter illustrates the application of quality improvement science techniques to iteratively address family- and system-level barriers to primary care. 4. Improvement for a Community Population: The Magnolia Community Initiative 51 Moira Inkelas, Patricia Bowie, Lila Guirguis This chapter illustrates how a network of diverse organizations can use iterative learning cycles to come up with promising ideas, test and prototype these ideas, and spread and sustain what is found to work for a community population. 5. Breaking the “Adopt, Attack, Abandon” Cycle: A Case for Improvement Science in K–12 Education 65 Kristen Rohanna This chapter describes the implementation of rapid cycles of evaluations (Plan–Do–Study–Act cycles) to adapt interventions to local school contexts. 6. Online Learning as a Wind Tunnel for Improving Teaching 79 James W. Stigler, Karen B. Givvin This chapter considers the potential value of combining improvement science and online learning. 7. Value and Opportunity for Improvement Science in Evaluation 93 Moira Inkelas, Christina A. Christie, Sebastian Lemire This concluding chapter reflects on the case examples comprising this volume and considers the major benefits and implications of integrating improvement science more firmly in evaluation. Index 103