"How-to" Guides for Conducting EE Evaluation (OLD)
Guides for planning, implementing, and reporting evaluations of environmental education programs
In this section, there are a number of guides that provide step-by-step instructions, or practical information, for planning and conducting an evaluation. There are three types of guides below:
-
EE Evaluation Specific Guides
These guides provide instructions on how to perform an environmental education evaluation. They provide steps, hints, and examples related to EE specifically. - Topic Specific Guides
These guides provide information about a specific topic (for example, logic models). The topics often have nuances not covered in other guides. - General Evaluation Guides
These guides are excellent resources for all evaluators to consider, regardless of field.
For each guide, there is a designation of beginner, intermediate, and/or advanced. This indicates the skill level that each guide targets and should help in selecting the most appropriate resource(s). Most of the guides are available free of charge.
EE Evaluation Specific Guides (5 guides)
- Evaluating EE in Schools: A Practical Guide for Teachers.
Bennett, D.B. (1984). UNESCO-UNEP.
Available at http://unesdoc.unesco.org/images/0006/000661/066120eo.pdf
beginner intermediate
Although one of the first guides written on EE evaluation, this resource provides timeless advice for planning your evaluation, implementing it, and making the most of your data. Particularly useful are Bennett's "action verbs" for developing measures of knowledge and understanding, thinking skills, and action skills. The guide also provides easy to understand explanations of different evaluation designs (e.g. post-test only vs. pre-test and post-test), instrument types, and research methods. Step-by-step instructions for collecting and analyzing data help readers avoid common mistakes. - Does your project make a difference? A guide to evaluating environmental education projects and programs.
Sydney: Department of Environment and Conservation, Australia. (2004)
Available at http://www.epa.nsw.gov.au/community/projecteval.htm
beginner
Information in this guide is laid out in an easy to digest manner, making it very user-friendly for evaluation newcomers. In addition to describing important considerations in each step of evaluation, Does your project make a difference? uses hypothetical case studies to help readers think through the logic of their programs, the types of questions they might ask about their program's impact, and how those questions can be measured using indicators. The guide also provides an "Outcomes Hierarchy Framework Template" for planning evaluations, which some readers may find easier to complete than traditional logic models. - Designing Evaluation for Education Projects.
NOAA Office of Education and Sustainable Development (2004).
Available at http://wateroutreach.uwex.edu/use/documents/ NOAAEvalmanualFINAL.pdf
beginner
This is a useful and easy-to-read primer for learning the basics of evaluation. In addition to outlining the steps of planning your evaluation, the guide provides brief explanations of the different types of evaluations, the questions each type might help you answer, and a review of what different evaluation tools can help you accomplish. Particularly useful are the appendices which include tips on writing objectives, selecting a sample, and choosing appropriate evaluation tools. - Evaluating Environmental Education.
Stokking et. al.. IUCN Commission on Education and Communication (1996).
Available at http://app.iucn.org/webfiles/doc/CEC/Public/ Electronic/CEC/ Books/Evaluating_Environmental_Ed.pdf
beginner intermediate
Evaluating Environmental Education provides an in-depth look at conducting an evaluation. For each step of the process (e.g. formulating objectives, selecting instruments, collecting data, etc.), the guide provides clear explanations of the choices an evaluator has as well as important factors to consider when choosing the option best suited to his or her situation. Numerous examples are used throughout to make the concepts more concrete. - Measuring the Success of EE Programs.
Thomson, G. and J. Hoffman. Canadian Parks and Wilderness Society (2003).
Available at http://www.cpawscalgary.org/education/pics/ee-success.pdf
beginner intermediate
This guide, focused on outcome evaluation, outlines practical strategies for planning and implementing EE evaluations. Illustrative examples are drawn from existing programs and special attention is paid to measuring challenging outcomes such as changes in attitude, behavior, and benefit to the environment. Of particular note are the sample questionnaires and interview protocols included in the appendices.
Topic Specific Guides (3 guides)
- Evaluation Publications available at the University of Wisconsin Extension Program Development and Evaluation Unit. http://www.uwex.edu/ces/pdande/evaluation/evaldocs.html
beginner
The Evaluation Publications section of the UW-Extension site offers user-friendly guides on various topics such as planning an evaluation, questionnaire design, sampling, collecting data, analyzing data using Excel, using graphics to report results, and more! Although focused on evaluating Extension programs, the information is applicable to almost any evaluation setting. - W.K. Kellogg Foundation Logic Model Development Guide.
W.K. Kellogg Foundation (2004).
Available at http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf
beginner intermediate
This guide describes what a logic model is and how it can be used to direct your evaluation efforts. Fictitious examples are used throughout to help readers understand the processes of both developing a logic model and using it to frame your evaluation questions. Helpful tips are also provided for establishing indicators to measure success. The appendix offers logic model templates as well as checklists of important things to consider when constructing each part of the model. - Using a Logic Model to Review and Analyze an Environmental Education Program.
Marcinkowski, T.J. (2004). NAAEE.
Available for purchase at http://naaee.org/pages/publications/index.html
intermediate
This paper demonstrates how logic models provide "conceptual guidance and visual support" in analyzing and comparing environmental education programs. To illustrate this point, Marcinkowski guides readers step-by-step through a logic model-based examination of Hungerford et al.'s Issue and Action Orientation program.
General Evaluation Guides (5 guides)
- The 2002 User Friendly Handbook for Project Evaluation.
Frechtling, J.. National Science Foundation (2002).
Available at http://www.nsf.gov/pubs/2002/nsf02057/nsf02057.pdf
beginner
The 2002 User Friendly Handbook for Project Evaluation targets managers who need information on what evaluations are, the main components of an evaluation, and the important issues to consider when planning one. The guide covers a range of topics including evaluation design, sampling, and reporting and also provides worksheets for several phases of the planning process. In addition, the handbook dedicates a chapter to the complexities of being culturally responsive in evaluation. - Evaluator's Handbook.
Herman, J. L., Morris, L. L., Fitz-Gibbon, C. T. . Newbury Park, CA: SAGE Publications (1984).
Available for purchase at Amazon.com Evaluator's Handbook
intermediate advanced
This guide is written in a handbook/workbook format to provide evaluators with practical advice, information, and samples on how to conceptualize, plan, and conduct evaluations. The workbook provides step-by-step guides as well as questions to ask stakeholders, sample budget planners, and worksheets for each step of the evaluation process. Seperate sections and worksheets are included for summative and formative evaluations. - Participatory Evaluation Essentials 2003: A Guide for Non-Profit Organizations and their Evaluation Partners.
Bruner Foundation (2003).
Available at http://www.brunerfoundation.org/ei/docs/ guide_for_nonprofits.pdf
beginner intermediate
Written in a handbook format, Participatory Evaluation Essentials 2003 provides an excellent overview of critical questions to consider as you plan your evaluation. Guidance is provided on topics such as identifying and working with evaluation stakeholders, developing a logic model, selecting appropriate data collection techniques, and writing the evaluation report. For each of these, worksheets are provided to help readers apply the information to their respective programs. Particularly noteworthy is the section on data collection methods, which reviews multiple ways to conduct surveys, interviews, and other techniques. - User-Friendly Handbook for Mixed Methods Evaluations
Frechtling, J. and Sharp, L. (1997). National Science Foundation .
Available at http://www.ehr.nsf.gov/EHR/REC/pubs/ NSF97-153/pdf/mm_eval.pdf
intermediate
This publication is designed to help evaluators learn about mixed methods evaluations and how to plan and conduct them. Specifically, it provides information on how to include qualitative information with a quantitative study. The handbook includes worksheets for determining what questions to focus on and what methods to use in responding to those questions. - W.K. Kellogg Foundation Evaluation Handbook
W.K. Kellogg Foundation (1998).
Available at http://www.wkkf.org/Pubs/Tools/Evaluation/Pub770.pdf
beginner
Part II of this handbook, "Blueprint for Conducting Project-Level Evaluation," provides a brief overview of different types of evaluations (e.g., needs assessment, implementation/formative evaluation, and summative/outcome evaluation) as well as three techniques for developing a logic model. Chapter Five provides an overview of important considerations at different stages of the evaluation process. Particularly helpful are the worksheets and advice for budgeting an evaluation and choosing an evaluator. The handbook also discusses various methods of data collection and gives advice for selecting the methods most appropriate to different needs.



This page is so helpful. Excellent descriptions of the resources.
Under the description of “topic specific guides” at the top, I would edit text as follows: “THESE MANUALS (NOT the topics) often have nuances not covered in other guides.
Under "General Evaluation Guides", (or maybe "topic specific guides" I wonder why you don't include EMI's new evaluation guide, called "Measuring Progress: A guide to the development and implementation of a conservation evaluation plan ". Seems like it would be good to promote an in-house publication, even though it's specific to conservation. I'm quite sure it's been used to evaluate EE programs. Here's website to download for free: http://www.snre.umich.edu/ecomgt//evaluation/guide/download.htm Here's a short description of the guide: http://www.snre.umich.edu/ecomgt//evaluation/guide/stages.htm
Also, it seems like you should be able to find a few more good ones to add under the "general eval guides" section.
I like the pop out definitions of words like "outcomes"