Environmental Program Evaluation
(An Online Course)
Program evaluation is one mechanism for establishing accountability in agencies and organizations. This online grduate course focuses on the techniques and application of evaluation methods to assess the effectiveness of organizations and agencies envolved in environmental services. It does so by exposing the student to a variety of evaluation frameworks employed by such organizations as NSF, EPA, NOAA, WHO, the W.W. Kellogg Foundation and more. Participants will learn evaluation techniques currently used to determine whether their organization's programs are having the intended effects. Participants will also learn how to develop and implement cutting-edge performance measurement strategies. Topics include:
- Program evaluation and accountability
- Practical program evaluation techniques
- Performance management strategies for effective and efficient programming
- Access to EPA, NOAA, NSF, WHO, CDC, Kelogg and other program evaluation procedures and processes.
This graduate course in environmental program evaluation is designed to:
- Enable students to conduct environmental program evaluations within a variety of organizational and programatic settings.
- Facilitate student capacity to identify and engage stakeholders in the evaluative process.
- Prepare students to conduct program evaluation directed toward program and organizational "processes" as well as program "outcomes."
- Enable students to develop and/or improve upon measures of program performance.
- Provide students with the skills to understand the relationship between program activities and results.
- Assist students in generating useful data on program results that may not be regularly collected by an organization.
- Aid students in insuring that outcome and process data are of high quality and utility.
Boulmetis, John and Dutwin, Phyllis (2011) The ABC's of Program Evaluation. San Francisco, CA: Jossey-Bass. (Available on Amazon Kindle)
Emison, Gerald A. (2007) Practical Program Evaluations: Getting From Ideas to Outcomes. Washington, DC: CQ Press. (Available on Amazon Kindle)
CDC (2014) Developing and Effective Evaluative Plan - Setting the Course for Effective Program Evaluation. Atlanta, GA: Centers for Disease Control & Prevention.
CDC (2001) "Program Operations: Guidelines for STD Prevention," Atlanta, GA: Centers for Disease Control & Prefvention
CDC (2001) "Types of Evaluation," Atlanta, GA: Centers for Disease Control & Prevention.
CDC (1999) "Framework for Program Evaluation in Public Health," Mortality and Morbidity Weekly Report (MMWR). Vol. 48, No. RR11 (September 17).
Frechtling, Joy and Sharp Westat, Joy (1997) User Friendly Handbook for Mixed-Method Evaluations. Washington, DC: National Science Foundation.
Government Accountability Office (GAO) (1992) Quantitative Data Analysis: An Introduction. Washington, DC: GAO.
Holm-Hansen, Cheryl (2007) "Ethical Issues: Tips for Conducting Program Evaluation," Minnestoa Office of Justice Programs.
Kellogg Foundation (2017) The Step-By-Step Guide to Evaluation. Battle Creek, MI: W.K. Kellogg Foundation.
Kellogg Foundation (2004) W.K. Kellogg Foundation Evaluation Handbook. Battle Creek, MI: W.K. KelloggFoundation.
Malkinson, Trevor (2010) "To What End Are We Living?- Instrumental Reason and the Problem of the Good Life in Modern Times," Beams and Struts. (July). Accessed October 7, 2017 http://beamsandstruts.com/essays/item/85-to-what-end-are-we-living?-instrumental-reason-and-the-problem-of-the-good-life-in-modern-times
Michaud, Mary (2002) "Tips on Preparing a Conference Poster," Madison, WI: Uiversity of Wisconsin.
Muriel J. Bebeau (1995) "Moral Reasoning in Scientific Research: Cases for Teaching and Assessment." Bloomington, Indiana: Poynter Center.
National Center for Environmental Innovation (NCEI) (2009) Guidelines for Evaluating an EPA Partnership Program. Washington, DC: USEPA.
National Center for Environmental Innovation (NCEI) (2009) "EPA Program Evaluation Worksheet," Washington, DC: USEPA.
National Oceanopgraphic and Atomospheric Administration (NOAA) (2014) "Program Evaluation Guide," Washington, DC: NOAA.
Preskill, Hallie and Jones, Nathalie (2009) A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions. Princeton, NJ: Robert Wood Johnson Foundation.
Rogers, Patricia (2014) "Rainbow Program Evaluation Framework," RMIT University, Melbourne, Australia.
Taylor-Powell, Ellen and Henert, Ellen (2008) "Developing a Logical Model: Teaching and Training Guide," (plus Logical Model Slides) Madison, WI: University of Wisconsin Extension.
University of Wisconsin-Extension, (2014) Developing a Logical Model p. 1-140. Mikwaukee, WI.
World Health Organization (WHO) (2000)Workbook 1: Planning Evaluations. Wasington DC: WHO.
World Health Organization (WHO) (2000) Workbook 2: Implementing Evaluations. Wasington DC: WHO.
Yumi, Sara and Beaudry, Susan (2007) "Monitoring & Evaluation," World Bank Small Grants Program.
- Weekly Submission of Homework Questions. (50% of grade)
- Evaluation Plan Proposal For Students Enrolled in the M.A. Program in Environmental Studies: (1) Thoroughly review one of the environmental cases linked below and research related mateiral pertinent to your chosen case, (2) Choose one of the evaluation paradigms discussed in this course, (3) Develop a program evaluation plan based upon your chosen paradigm that will be implemented next semester in concert with your professional internship. All students must submit an intership site program evaluation proposal (for the following semester) by no later than March 25. The initial internship site program evaluation proposal will answer the following questions:
- What are you going to evaluate and where?
- What is the purpose of the evaluation?
- Who will use the evaluation?
- How will they use it?
- What questions will the evaluation seek to answer?
- What information do you need to answer the questions?
- When is the evaluation needed?
- What resources do you need—time, money, people?
Students will consult with their instructor and receive approval before proceeding to develop a program evaluation plan. Students will subsequently have until April 16 to complete a program evaluation plan for their interniship site and submit it to the instructor via Canvas email. The plan must be based upon one of the program evaluation paradigms emphasized in this class. It must address the following issues:
- What sources of information will you use?
- What data collection method(s) will you use?
- What collection procedures will you use?
- How will the data be analyzed?
- How will the information be interpreted and by whom?
- How will the evaluation be communicated and shared?
- How will you implement the plan over time?
- Who will be responsible for each part of the implementation timeline and what will their specific responsibilities entail?
- When will the final program evaluation be completed?
The completed program evaluation plan will be worth 50% of the class credit.
Environmental Program Evaluation Cases:
- Calderon, Margaret M. and Nawir, Ani A. (2006) "An Evaluation of the Feasiblity and Benefits of Forest Partnerships to Develop Tree Plantations: Case Studies in the Phillipines," CIFOR Working Paper No. 27, Bogor, Indonesia: Center for International Forestry Research.
- Centers for Disease Control and Prevention (CDC) (2011) "Introduction to Program Evaluation for Public Health Programs," (See this guide and its associated Case Studies)
- Chen-Hsuan Cheng, Judith, Monroe, Martha C. and Gordon, Alison (2010) "Measuring the Effectiveness of Lagoon Quest: A Case Study in Environmental Education Program Evaluation," Gainesville, FL: University of Florida Institute for Food and Agricultural Science.
- Enviro-Vision and Herrera Environmental Consultants (2006) "Evaluation of Washington's Industrial Stormwater General Permit," (November) Washington Department of Ecology Contract No. C0600124, Washington, DC: Office of Policy, Planning and Evaluation, USEPA.
- Martin, Shiela A. et al. (1996) "Eco-Industrial Parks: A Case Study and Analysis of Economic, Environmental, Technical and Regulatory Issues," RTI Project No. 6050FR,
- Matirui, Violet and Stewart, Howard M. (2007) "Country Program Case Study: Kenya," Joint Evaluation of the GEF Small Grants Programme, UNDP Evaluation Office (June).
- Mendoza, Patricia B. (2007) "Country Program Case Study: Belize," Joint Evaluation of the GEF Small Grants Programme, ABEN Services and Consulting Limited, Belize City, Belize (June).
- Minkler, Meredith et. al. (2008) "Promoting Healthy Public Policy Through Community Based Participatory Research: Ten Case Studies," A project of the University of California, Berkeley, School of Public Health and PolicyLink, funded by a grant from the W. K. Kellogg Foundation.
- U.S. Envirionmental Protection Agency (USEPA) (2012) "Measuring the Effectiveness of the Ocean Dumping Management Program," Washington, DC: USEPA.
- Worlen, Christine (2011) "Meta Evaluation of Climate Mitigation - Case Study: Poland's Heating Sector," Berlin, GR: Areopo Consultants
59% and Below
Homework submitted later than 5:00 p.m. on the final date of each weekly class session will will be considered late and will not be credited. Distance learning courses are considered to operate over a seven day week period. Missing (i.e. nonparticipation or inadequate participation in class activities on Canvas via email) for 3 classes or more without instructor permission will result in students being withdrawn from the class.
NOTE: Course assignments and schedule, course objectives, and grading criteria, distributions and weights may change as circumstances dictate and at the discretion of the instructor.
APA Documentation Method:
strongly encourage every student to take the time to complete
the following online
APA Citation and Reference Tutorial. One of the objectives of this class is that you
learn how to use the APA documentation style. You can learn
everything you need to know about how to use this method by
studying the fmaterial at the following sites. Please understand
that any of your discussion questions or your student
presentation papers that are lacking in complete APA styled
documentation will be returned to you and will not be graded
until they are in order.
APA Documentation Aids:
Guide for Writing Research
Papers in APA Style
University: Using American
APA Citation Styles - University of Toledo
APA Style Sheet (Cornell)
Quick Guide APA Style (UNC)
APA Guide to Electronic Resources
APA Style for Videos
APA Style for Court Decisions
Frequently Asked Questions Regarding APA Style