Monitoring Jury Progress

Once a round is open, you need to know whether the panel will finish on time and whether anyone is scoring in a way that will distort the result. Two pages answer those questions: the Rounds tab on the Jury page for how much work is done, and the call's Analytics page for how the panel is scoring. This guide covers both, plus reminders and what to do when a juror drops out.

How far along is the round

Open Admin → Calls → your call → Jury and go to the Rounds tab. Each round card shows its dates, its status, and Scores Submitted for the round as a whole. That number against your submission count and panel size tells you whether the round is on track.

Jurors can score only while a round is Open. If the round is still in Configuring or Assigning, nobody is scoring yet, and the count staying at zero is the state machine, not the panel.

How is the panel scoring

Open Admin → Calls → your call → Analytics and use the Jury Analytics tab. Four cards summarise the round:

• Total Reviews, with an average per juror.
• Completion Rate across the panel.
• Average Score, with the scale it sits on.
• Inter-Rater Reliability, how closely your jurors agree with each other.

Below them, Bias Detection raises alerts for jurors scoring consistently high or low, using an extreme spread, or scoring inconsistently. An alert is a prompt to look, not a verdict. A juror flagged as a low scorer may simply be applying the rubric strictly while the rest of the panel is generous, which is exactly what score normalization exists to handle. See Score Normalization and Results.

The Juror Performance tab gives you the per-juror table: Juror, Reviews, Avg Score, Std Dev, Consistency, Completion and Bias. This is where a juror who has not started shows up as a completion figure near zero.

Reminders

Crafted Call emails jurors who still have work outstanding as a round's end date approaches, at roughly a week, three days, one day and four hours out. Each juror gets each reminder once. The email names how many submissions they have left and links straight into the review screen.