Participation

Training

Scorers will be trained on the tasks, rubrics, and scoring process in training sessions held at our Hadley, MA facility. Each training session will focus on defining what is expected from each candidate, and what to look for at each level of proficiency. Scorers will be trained until they meet calibration and reliability standards on the exemplars and based on selected benchmarks for the score scale(s).

Once a scorer is certified, he or she will use the web-based assessment system to evaluate and score candidate work products, which will maintain a record of all scores. Scoring staff will use these data to monitor scorer performance to ensure the accurate and reliable assignment of scores. The system will also include materials for system training and support for scorers.

Confidentiality

Scorers will be assigned unique codes to track information about the reliability of their scoring and to ensure confidentiality. Candidate confidentiality will be maintained: trainers and scorers will only have access to the de-identified final products candidates submit, for candidates with whom they have no affiliation.

Scorers are required to maintain the security of the training and scoring materials, and of individual candidate products and scores. They must agree to maintain security by not copying, discussing or disclosing and confidential information related to the performance assessment system. The scoring application is secure and web-based, permitting multiple raters to view and score an individual candidate's submission independently for the purpose of monitoring rater reliability. Scorers who do not consistently meet the scoring standard will be recalibrated and those who repeatedly cannot meet the standard will be released.