Formative evaluation
From EvaluationWiki
Formative Evaluation [1]
Formative evaluation begins during project development and continues throughout the life of the project. Its intent is to assess ongoing project activities and provide information to monitor and improve the project. It is done at several points in the developmental life of a project and its activities. According to evaluation theorist Bob Stake, “When the cook tastes the soup, that’s formative; When the guests taste the soup, that’s summative.”
The following are some examples of formative evaluation:
Example A - "Why didn't they show up?" At the beginning of the school year, Smalltown High School decided to provide the first part of a two-part teacher professional development series on the integration of technology into the classroom. While the teachers attending the professional development were satisfied and seemed to benefit from it, the turnout was lower than expected. Upon further investigation, the evaluators found out that, since the professional development was offered after school, many teachers who were involved in leading extra-curricular activities could not attend. Learning this, evaluators surveyed teachers about their preferences and availability. From the survey results, the decision was made to have both Saturday training and training during the summer.
Example B - "Things don't always go as planned." During the summer, Big City Elementary completed all its wiring. The school's plan was to install equipment in the fall and provide professional development for teachers that would teach them how to use the new equipment in the spring. In December, as part of its overall evaluation plan, the school assessed whether the equipment had been installed as intended. It turned out that most of the computer equipment was backordered and would not be installed in time for the professional development. Evaluators determined that, because of the delay, prices had dropped and better equipment could be obtained at a similar cost. The school quickly updated specifications and was able not only to receive the equipment in time for the spring professional development, but to purchase better equipment than originally intended.
Formative evaluation has two components: implementation evaluation and progress evaluation. Implementation Evaluation. The purpose of implementation evaluation is to assess whether the project is being conducted as planned. This type of evaluation, sometimes called “process evaluation,” may occur once or several times during the life of the program. The underlying principle is that before you can
evaluate the outcomes or impact of a program, you must make sure the program and its components are really
operating and, if they, are operating according to the proposed plan or description.
A series of implementation questions guides an implementation evaluation. For example, questions that might be posed for the NSF
Louis Stokes Alliances for Minority Participation (LSAMP) are as follows:
- Were appropriate students selected? Were students with deficits in precollege preparation included as well as ones with stronger records? Was the makeup of the participant group consistent with NSF’s goal of developing a more diverse workforce?
- Were appropriate recruitment strategies used? Were students identified early enough in their undergraduate careers to provide the transitional supports needed?
- Do the activities and strategies match those described in the plan? Were students given both academic and personal supports? To what extent were meaningful opportunities to conduct research provided?
- Was a solid project management plan developed and followed?
Sometimes the terms “implementation evaluation” and “monitoring evaluation” are confused. They are not the same. An implementation evaluation is an early check by the project staff, or the evaluator, to see if all essential elements are in place and operating. Monitoring is an external check. The monitor typically comes from the funding agency and is responsible for determining progress and compliance on a contract or grant for the project. Although the two differ, implementation evaluation, if effective, can facilitate project implementation and ensure that there are no unwelcome surprises during monitoring.
The purpose of a progress evaluation is
to assess progress in meeting the goals of the program and the
project. It involves collecting information to learn whether or
not the benchmarks of participant progress were met and to
point out unexpected developments. Progress evaluation
collects information to determine what the impact of the
activities and strategies is on participants, curriculum, or
institutions at various stages of the intervention. By measuring
progress, program staff can eliminate the risk of waiting until
participants have experienced the entire program to assess likely
outcomes. If the data collected as part of the progress evaluation fail
to show expected changes, the information can be used to fine tune
the project. Data collected as part of a progress evaluation can also
contribute to, or form the basis for, a summative evaluation conducted
at some future date. In a progress evaluation of the LSAMP program,
the following questions can be addressed:
- Are the participants moving toward the anticipated goals of the project? Are they enhancing their academic skills? Are they gaining confidence in themselves as successful learners? Are they improving their understanding of the research process?
- Are the numbers of students reached increasing? How do changes in project participation relate to changes in the overall enrollments in mathematics, science, and technology areas at their institutions? Are students being retained in their programs at an increasing rate?
- Does student progress seem sufficient in light of the long range goals of the program and project to increase the number of traditionally underrepresented students who receive degrees in science, mathematics, or technology?
Progress evaluation is useful throughout the life of the project, but is
most vital during the early stages when activities are piloted and their
individual effectiveness or articulation with other project components
is unknown.
References
- ↑ User-Friendly Handbook for Project Evaluation, National Science Foundation. (2002)
