Choose one recording and one criterion
A session with a full interview and every criterion becomes a general discussion. Pick one recording and one criterion that the team finds difficult. Tell the reviewers which part of the rubric they will use and give them the transcript or the link before the session.
Choose a recording where the answer is not obviously strong or weak. A clear case teaches less than an answer that two reviewers could read in different ways. If the team already knows the candidate, choose a recording from another role.
Score alone before the meeting
Ask each reviewer to submit a score and a short note before the session. The note should name the evidence, not only the conclusion. A sentence such as the answer gave a sequence but no risk is useful. A sentence such as good answer is not.
Collect the notes without showing them to the group. At the start of the session, show the scores and the notes together. The reviewers can then see whether they used the same evidence or read the same evidence in different ways.
Ask for the evidence first
Go around the room and ask each reviewer to point to the part of the answer that supports the score. Do not start with the score. Starting with the number turns the session into a negotiation. Starting with the evidence makes clear where the reading diverged.
When two reviewers point to the same sentence, ask which criterion it belongs to. When they point to different sentences, read both passages aloud. The group can then decide whether the rubric covers the difference or whether the anchor is missing a case.
Separate the rubric problem from the candidate problem
A disagreement can come from a vague anchor, a missed passage or a different understanding of the role. Name the cause before you discuss the final score. If the anchor is vague, write an example. If a passage was missed, read it again. If the role understanding differs, ask the hiring manager to explain the criterion in plain words.
Do not settle the session by averaging the scores. An average hides the reason for the gap. A written rule or a clearer example gives the next reviewer something they can use.
Write one rule and add it to the rubric
End the session with one rule that the group can apply. Keep it short and put it next to the criterion. For example: an answer that names a sequence but no risk is partial for planning. Add the example that made the rule clear.
Send the updated rubric to everyone who joined and to anyone who reviews the role later. A rule that stays in the meeting notes will not reach the next interview.
Plan the next session around a real moment
Choose the next session when a new rubric, a new role or a new reviewer arrives. Add a short check after a decision that was hard to explain. A session tied to a real moment is easier to justify than a recurring meeting that nobody prepares for.
Keep a small log of the rules the team has added. Read it before a new role starts. Over time, the log becomes the practical memory of how the team reads its own rubric.