1. What are the strengths and weaknesses of each of the following designs: posttest-only, pretest/posttest with comparison group, and pretest/posttest only?
Evaluation study results are an important measure of a training’s effectiveness. Trainers use these results as a guideline for improving training procedures and as proof that training is working. Trainers want to prove to organizations that their training is effective and to do this, trainers must eliminate the question of whether evaluation results are valid. Pretests, post-tests, and the use of comparison groups, are ways that trainers eliminate the threat of validity.
The post-test only training measure collects evaluation results only after the training is complete. No other evaluation methods are performed prior to the training. According to Meera’s website, one of the advantages of the post-test only design is that it is less time consuming than the other designs. It is also useful when participants are not available for pretesting, as in the case of new hires. The biggest disadvantage of the post-test only evaluation, is that there is no data in which to compare the results. Therefore, there is no way to determine if the outcome, of the post-test results, are due to the training or if participants had prior knowledge of training material, from some other source. Post-test only evaluations should only be used when there are time constraints or when there is no way to perform a pre-test, in which to compare results.
Pretest/posttest only evaluations collect data before and after training, in order to determine training effectiveness. In this design, the same testing method is used before and after training. With this design, evaluators are able to compare test scores and therefore determine what trainees have learned, based on the participants’ prior knowledge, which was evaluated in the pretest. Since there is a control, the previous test results, the results of this method are more reliable than the post-test only method. As the text noted, one of the downsides of this method is that it can not account for knowledge acquired elsewhere.
Pretest/posttest with a comparison group is the most accurate, of the three designs. Just as with the pretest/posttest only design, data is collected before and after training. In addition, once training is complete, data is also collected from a group that did not participate in the training. This evaluation design is more time consuming than the other designs. In addition, the comparison group must consist of people from similar backgrounds as those that completed the training. As the text notes, if the results of the posttest are better for the group that took the training, this proves that the training is responsible for the trainees’ new knowledge.
As with any evaluation method, there are pros and cons. Trainers must choose their evaluation methods wisely, to reduce the threat of validity. Evaluation results assist trainers with ways to improve training, and they are also a method of ensuring that organizations will choose their training, in the future. The more accurate the evaluation results, the better for the trainer, trainees, and the organization.
References
Noe, R. A. (2017). Employee training and development (7th ed.). New York, NY: McGraw-Hill. ISBN: 9780078112850.
Zint, M. (2019). Types of Evaluation Designs. Retrieved from http://meera.snre.umich.edu/types-evaluation-designs
