In large science courses, teaching assistants do most of the grading, and their standards rarely match. One TA gives full credit for enthusiasm while another penalizes any imprecision, and students notice quickly. Writing assignments based on a book like Joe Schwarcz's "Radar, Hula Hoops, and Playful Pigs" make the problem more visible, since open-ended responses leave more room for different interpretations than a numerical answer. A deliberate calibration process is the best protection against unfair variation.

Calibration starts with a rubric whose rows can be observed in the text. If graders must guess what "strong understanding" looks like, they will guess differently. Describe each level using features they can point to, such as the presence of an accurate explanation, a named principle, or a specific example.

Then, before grading begins, have every TA read and score the same five or six sample papers independently. The sample should include a range of quality, including at least one paper that is difficult to score. Difficult cases reveal differences in interpretation that easy papers hide.

Running the calibration meeting

Collect everyone's scores before the meeting and display them side by side. Start with the papers where scores diverge the most, and ask each grader to explain their reasoning aloud. The conversation usually reveals that the disagreement comes from a specific row, such as how much weight to give mechanical errors when the science is sound.

• Have each TA score the sample papers independently before seeing anyone else's marks.
• Display scores in a table and identify papers with the largest disagreement first.
• Discuss the reasoning behind each score and resolve differences by pointing to rubric language.
• Revise any rubric row that two graders interpreted differently and note the clarification.
• Save the agreed scores and comments as anchor papers for future reference.

Keeping alignment from drifting during the term

Alignment erodes over a semester as graders get tired and develop personal shortcuts. Schedule a brief recalibration at the midpoint using fresh sample papers, and monitor score distributions by grader. If one TA's average is a full level higher or lower than the others, a conversation is warranted, though differences can also reflect which sections they were assigned.

Spot-check a few papers from each TA every assignment cycle. This does not need to be a formal audit; reading five papers and comparing them to your own judgment is enough to catch problems early. It also signals that consistency matters, which tends to improve the care graders take.

Training TAs to give useful feedback

Scores are only part of the picture; the comments matter just as much. Provide a short guide with examples of strong and weak feedback, and encourage TAs to prioritize one or two key suggestions per paper. A TA who writes a sentence naming a specific strength and a specific next step provides more value than one who fills the margins with minor corrections.

Pair new TAs with experienced ones for their first batch. Reading each other's graded papers builds a shared sense of tone and standards, and it lightens the load on the instructor, who would otherwise have to answer the same questions again and again.

Using AI as a consistent reference point

AI grading tools apply the same rubric to every paper without fatigue, which makes them useful as a reference point in a calibration program. Comparing a TA's scores to the tool's scores can highlight where a grader is systematically harsher or more lenient than the rubric intends. The goal is not to replace TAs but to give them and you a steady benchmark.

When disagreements arise between a TA and the tool, treat them as learning opportunities. Sometimes the tool misses context a person would catch, and sometimes the TA has drifted from the rubric. Either way, examining the case together sharpens everyone's understanding of the standard.