Start with the anchor, not the score
A rubric without anchors leaves the meaning of a score to the reviewer. Write a short description for each level: what a strong answer contains, what a partial answer contains and what is missing. Use the same words the interview questions use, so the reviewer can connect the answer to the criterion.
An anchor can be a sentence and an example. For a planning criterion, the strong anchor might ask for a sequence, a risk and a review point. The partial anchor might ask for a sequence without a risk. The missing anchor describes an answer that stays general.
Score the same recording separately
Pick a recording that the team has already reviewed. Ask each manager to score it alone with the rubric. Do not show the scores before everyone has written a note. When the scores are on the table, ask each person to point to the sentence in the transcript that supports the score.
This exercise shows where the rubric is vague. One manager may read a general answer as strong because it sounds confident. Another may read the same answer as partial because it has no example. The difference is not a character flaw. It is a gap in the anchor.
Compare the evidence, not the person
When reviewers disagree, ask what evidence they used and which criterion it belongs to. Keep the conversation on the recording. Do not turn the session into a review of the manager's judgement. The question is whether the rubric gives both reviewers a shared way to read the answer.
If the evidence is the same and the scores differ, the anchor needs a decision rule. If the evidence differs, one reviewer may have missed a part of the transcript. Rewind, read the passage and score again with the rule in mind.
Write the rule down
After the session, update the rubric with the examples the team discussed. Add a short note for the case that caused the disagreement. Keep the note next to the criterion so a new reviewer can read it before the next interview.
Record the decision in a shared place. A rule that lives only in the memory of the people in the room will disappear when a manager changes team. The written rule also helps when a candidate or a colleague asks why a score was given.
Look for drift in the review notes
Read a sample of review notes and look for phrases that do not refer to the rubric. Comments about likeability, accent or presentation are a sign that the reviewer has left the criteria. Comments that repeat the question without evidence are a sign that the note was written from memory.
Ask reviewers to quote or paraphrase the part of the answer they scored. That habit makes the note useful in a discussion and it makes drift visible before it reaches a decision.
Keep the routine small
A calibration session does not need a long agenda. One recording, one rubric and one question about the evidence is enough to start. Choose a moment when a new role, a new rubric or a new reviewer arrives, and add a short check after a decision that was hard to explain.
Consistent scoring is a practice, not a certificate. The team gets there by reading the same evidence against the same anchors and by writing down what it learned. Each session adds a little clarity to the next review.