There are instances where the creation of a rubric template may be used by multiple evaluators. In such cases, calibration of the rubric is critical in order to maintain reliability and validity. Following these guidelines can help you ensure appropriate calibration of campus rubrics before using them to assess individuals.
- Gather all evaluators and thoroughly discuss each component of the rubric at hand. For example, what particular components distinguish one achievement level from another?
- Whenever possible, provide work samples that showcase the achievement level differences on each dimension. Work together as a group using think aloud strategies to encourage discussion.
- First, focus your attention on rubric components that resulted in consistent ratings among the group. Once this discussion has concluded, follow-up with a more in-depth analysis and discussion around those inconsistently rated dimensions.
- It is ideal to have evaluators assess a few work samples on their own, coming together with another evaluator to compare ratings after the fact.
- Only once evaluators have gained a solid understanding of the specific dimensions and achievement levels, and feel comfortable using the tool, should evaluators assess individuals using the rubric.
- As with any assessment instrument, be prepared to analyze the results and devise a plan for improvement, if needed, before continuing to use the rubric for future programs or courses.