How to Assess Judgment and Prioritisation in Candidates (with a Scorecard Template)
Judgment is the ability to make a sound, proportionate decision with incomplete information, and prioritisation is the ability to rank competing work against a goal and say no to the rest. You assess both the same way: write a scorecard with anchored 1–4 descriptions before the first interview, ask every candidate the same behavioural and situational questions, review real decision artifacts, and have every interviewer score independently before the debrief. The template below is for a hypothetical senior product manager; it adapts easily to engineering or design leads.
What are judgment and prioritisation as hiring competencies?
A competency is a capability the job needs that you can observe and score. For senior roles, five do most of the work:
- Judgment: making a sound, proportionate decision with incomplete information, weighing evidence, risk and reversibility, and knowing when to decide, escalate or wait.
- Prioritisation: ranking competing work against a stated goal, cutting what doesn't make the list, and re-ranking when the facts change.
- Reasoning: breaking a problem into parts, stating assumptions, and estimating sensibly.
- Communication: making the point clearly to the audience in front of you, in speech and writing.
- Execution: turning a plan into a shipped outcome and owning what happens after launch.
Judgment and prioritisation are the hardest to see on a CV. A CV shows outcomes, not the decisions behind them, and never shows what the person chose not to do.
Why don't CV screens and coding tests capture judgment?
They measure the wrong things. Years of experience is the classic CV proxy, and in the revised estimates by Sackett, Zhang, Berry and Lievens its mean operational validity (the correlation with later job performance, after statistical corrections) is .07. Structured interviews come out at .42, work sample tests at .33 and unstructured interviews at .19 (Sackett et al., 2023, Table 1).
Timed coding tests check whether a senior engineer can solve a contained puzzle, not whether they pick the right problem, accept the right trade-off or know when to stop. For engineers, use the same scorecard with questions about architecture decisions, incident calls and migrations, plus a walkthrough of work they have already shipped.
How do you assess judgment and prioritisation in practice?
Use three sources of evidence, scored against the same anchors.
- Behavioural questions. The U.S. Office of Personnel Management says these "ask applicants to describe a past behavior demonstrated in a situation relevant to the competency of interest" (OPM). Push for specifics: what did they know, assume and cut?
- Situational questions. These ask what applicants "would do or how they would behave in a situation similar to those encountered on the job" (OPM). They matter most when past titles don't match the role.
- Decision artifacts. A roadmap with a "not doing" list, a decision note, a PRD that records trade-offs. Ask the candidate to walk you through one: it shows judgment under real constraints.
Scorecard template: senior product manager (hypothetical)
Copy this into a doc or spreadsheet. The weights are an example; set your own from the brief before you meet anyone.
Question bank (examples only; scenarios are hypothetical). B = behavioural, S = situational.
How do you compare shortlisted candidates side by side?
Put every finalist in one comparison grid: one row per competency, one column per candidate, each cell holding the score and a link to the evidence. A hypothetical example using the weights above:
The totals sit within a tenth of a point, which is normal. The grid's job is to show where candidates differ: B is the strongest decision-maker but the weakest communicator; C prioritises well but scored 2 on judgment. That tells the panel what to discuss and what to check next.
Three kinds of tool can hold the grid:
- A shared spreadsheet, enough for a team hiring a few people a year.
- ATS scorecards. In Greenhouse, "Scorecards allow your hiring team to review all of a job's candidates using a set of predetermined criteria", with an overall recommendation of Definitely Not, No, Yes or Strong Yes (Greenhouse).
- Evidence platforms, which attach artifacts and verified claims to each competency, so the grid links to proof rather than interview notes alone.
How should the debrief run?
OPM notes that structured interviews are usually scored by a panel in which each member "rates applicant responses individually and then participates in a group discussion to resolve significant scoring discrepancies" (OPM). A workable protocol:
- Score alone. Each interviewer submits scores and evidence the same day, before seeing anyone else's.
- Read the grid. Everyone reads the comparison grid and linked evidence before the meeting.
- Start with the biggest gaps. Discuss competencies where scores differ by two points or more.
- Argue from evidence. Each point cites an answer, artifact or verified claim.
- Decide and record. The hiring manager decides and writes the reason against the scorecard.
If the panel keeps disagreeing on one competency, fix the anchor. Read AI's playbook: "Disagreements among hiring panels usually mean the rubric is ambiguous, not that one interviewer is right" (Read AI).
Can AI assess judgment and communication, or only hard skills?
AI is useful around the assessment, not as the judge. It can capture and transcribe interviews (a human checks the summary), structure evidence by mapping answers and artifacts to the rubric's competencies, flag missing evidence for competencies nobody has probed, and keep the record: one audit-ready trail of scores, evidence and the decision.
It cannot reliably decide whether a decision was sound in its context. That depends on what the person knew at the time, which a model reading a transcript doesn't have, and fluent answers can sound like good judgment when they aren't. The principle is human in the loop: AI gathers and organises; the hiring manager decides and owns the reason. Read AI says AI "should capture and structure evidence rather than make the selection call" (Read AI); HackerEarth says automated scores "should inform recruiter and hiring manager decisions, not replace them, especially at borderline thresholds" (HackerEarth).
Regulation points the same way. For roles in New York City, Local Law 144 prohibits using an automated employment decision tool unless it has had a bias audit within one year of use, information about the audit is publicly available, and candidates have received notices; enforcement began on July 5, 2023 (NYC DCWP).
How does this help candidates from non-traditional backgrounds?
A scorecard built on competencies, not titles, lets you hire someone who has never been called a product manager. A founder who wound down a startup, an operations lead who ran a city launch, or an engineer who owned a roadmap informally can all score 3 or 4 on judgment and prioritisation. HackerEarth's guide notes that evidence-based hiring "allows recruiters to uncover talent from non-traditional sources" (HackerEarth).
To keep it fair: lean on situational questions when past roles don't map neatly; score the reasoning, not the vocabulary (someone who ranks five requests against a clear goal without naming a framework deserves a 3); and accept any credible artifact, from a spreadsheet model to an ops playbook. For the method behind this, read What is evidence-based hiring?
How does CareerPassport build this matrix for you?
CareerPassport starts where this guide does. The hiring manager writes one brief, and CareerPassport turns it into a hiring blueprint: technology requirements, expertise level, and an evaluation matrix chosen from nine competencies (Reasoning, Prioritization, Judgment, Expertise, Execution, Initiative, Communication, Collaboration, Alignment).
An agent then runs the role. It draws candidates from the company's own database, recruitment partners and CareerPassport profiles, approaches them, and collects evidence before anyone books an interview. Every claim is confirmed by the organisation named on it. The hiring manager gets a shortlist that is cleared to meet and stays in control: invite, reject, or view the evaluation summary.
To see an evaluation matrix built from one of your own briefs, book a free 30-minute demo.