The ABCs of Evaluation

The ABCs of Evaluation: Timeless Techniques for Program and Project Managers, 3rd Edition Thoroughly revised and updated, this third edition of the bestselling book offers an introduction to program evaluation including information on stakeholder relationships, program design, data collection and analysis, reporting results, and other important steps. This edition expands coverage of evaluation models and logic models. Written for any program type or setting, The ABCs of Evaluation shows how to select participants for the evaluation and how to deal with multiple goals and objectives—including those of the organization, the staff, and the client. All references and pedagogical features including learning objectives, key terms, and questions for discussion have been updated.

Preface vii The Authors xv 1 What is Evaluation? 1 Common Denominators 3 Two Definitions of Evaluation 4 Evaluating Efficiency, Effectiveness, and Impact 5 Evaluating Alternatives 8 Identifying Areas to Improve 8 Other Levels 10 Formal Reasons to Evaluate 12 Practical Application 15 Formative and Summative Evaluations 16 Putting It All Together 16 Key Words and Concepts 24 Questions and Exercises 24 Further Reading 27 2 Why Evaluate? 29 Grants and Grantsmanship 31 Benefits 33 Limitations 35 Considerations 36 Putting It All Together 40 Key Words and Concepts 40 Questions and Exercise 43 Further Reading 46 3 Decision Making: Whom to Involve, How, and Why? 47 Program and Project Cycles 49 Evaluation and Monitoring 52 Program Planning Cycle and Evaluation 57 Putting It All Together 61 Key Words and Concepts 65 Questions and Exercises 65 Further Reading 73 4 Starting Point: The Evaluator’s Program Description 75 What an Evaluator’s Program Description Can Do 77 Developing an Evaluator’s Program Description 78 Putting It All Together 83 Key Words and Concepts 84 Questions and Exercises 84 Further Reading 96 5 Choosing an Evaluation Model 97 More Than One Way to Evaluate 98 Four Evaluation Models 101 Other Models 108 Choosing the Right Model 110 Evaluation Design Format 111 Other Considerations 114 Putting It All Together 114 Key Words and Concepts 115 Questions and Exercise 115 Further Reading 117 6 Data Sources 119 Starting Point 120 Qualitative and Quantitative Data Sources 124 Four Levels of Data 125 Nature of Data Collection 127 Existing Data 128 Newly Collected Data 130 Putting It All Together 141 Key Words and Concepts 141 Questions and Exercise 141 Further Reading 144 7 Data Analysis 147 Data Analysis 148 Data 150 Terms 151 Levels of Data 157 Putting It All Together 164 Key Words and Concepts 164 Questions and Exercise 165 Further Reading 167 Software for Personal Computers 168 8 Is It Evaluation or Is It Research? 169 Determining Evaluation or Research from Intent 170 Determining Evaluation or Research from Design 172 Qualitative and Quantitative Characteristics 174 Sampling 175 Sampling and Surveys 180 Putting It All Together 181 Key Words and Concepts 183 Questions and Exercise 183 Further Reading 184 9 Writing the Evaluation Report 185 Communicating Your Findings 187 Evaluation Reports 188 The Report Outline 192 Putting It All Together 201 Key Words and Concepts 202 Questions and Exercise 202 Further Reading 203 10 Evaluation as a Business 205 Is There a Market for Program Evaluation? 206 How Do You Find the Business? 211 Proactive Resources 213 Reactive Resources 216 Emergency Resources 216 How Do You Negotiate a Contract? 217 Keeping the Business Alive 222 Putting It All Together 224 Key Words and Concepts 225 Further Reading 225 A: More on Data Analysis 227 Analyses for Each Level 227 Significance Testing 231 Statistical and Practical Significance 235 B: Sample Evaluation Report 237 C: Sample Evaluation Contract 271 References 277 Index 281