Ask three interviewers why they liked a candidate and you will often get three different answers: one liked the confidence, one liked the portfolio, one simply got on with them. None of those is wrong, but none of them is the job. Without a shared definition of what good looks like, an interview panel is three separate opinions averaged into a decision.
A weighted hiring scorecard fixes that before the first candidate walks in: the same criteria, the same weights and the same scale for everyone. This guide shows how to build one, how to combine a panel's scores, and how to use their disagreements. The free Weighted Hiring Scorecard does the maths and the comparison.
Key takeaways
- Pick four to six criteria from what the job actually requires, and weight them so they total 100%.
- Anchor the scale: write what a 1, a 3 and a 5 look like for each criterion, before any interviews.
- Ask every candidate the same questions for each criterion, and score from evidence.
- Interviewers score independently, before they discuss anyone.
- A two-point gap between interviewers on one criterion is a signal to compare evidence, not to split the difference.
Step 1: choose the criteria
Start from the job, not from a generic list. What will this person need to do well in their first six months? For an account manager at a small agency, that might be:
- Client communication: explains clearly, listens, handles difficult conversations.
- Problem solving: works out what is really wrong and what to do about it.
- Commercial awareness: understands budgets, scope and what the agency needs to earn.
- Organisation: keeps several projects and deadlines moving at once.
- Collaboration: works well with designers, developers and colleagues.
Four to six criteria is enough. More than that and interviewers stop distinguishing between them. Be wary of vague criteria like "culture fit": if you use it, define the specific behaviours you mean, or it becomes a polite word for "similar to us".
Step 2: weight the criteria to 100%
Not everything matters equally. Weights make that explicit:
| Criterion | Weight |
|---|---|
| Client communication | 30% |
| Problem solving | 25% |
| Commercial awareness | 20% |
| Organisation | 15% |
| Collaboration | 10% |
| Total | 100% |
Agree the weights with everyone on the panel before the interviews. Arguing about weights after meeting the candidates is how a favourite gets a scorecard built around them.
Step 3: anchor the 1 to 5 scale
"4 out of 5 for communication" means nothing unless everyone agrees what a 4 is. For each criterion, write short anchors:
- 1: no evidence, or evidence of a real concern.
- 3: meets the bar for the role. Clear, relevant examples.
- 5: exceptional. Specific examples with results, going beyond what the role needs.
For client communication, a 5 might be "described handling an unhappy client step by step, including what they said and what changed afterwards"; a 3, "gave a clear example of a routine client update". Twos and fours fall in between.
Step 4: ask the same questions, and write down the evidence
Give each criterion one or two questions, and ask every candidate the same ones. "Tell me about a time a client was unhappy with the work. What did you do?" is better than "Are you good with clients?", because it produces evidence you can score against the anchors. Interviewers note what the candidate said, not how they felt about it.
Step 5: score independently, then combine
Each interviewer scores each criterion from 1 to 5 before the panel talks, so the first or loudest opinion does not set everyone else's. Then combine them:
- Average each criterion across the interviewers.
- Weighted contribution = average ÷ 5 × weight.
- Weighted total = the sum of the contributions, out of 100.
For one candidate, scored by three interviewers:
| Criterion | Weight | Scores | Average | Contribution |
|---|---|---|---|---|
| Client communication | 30% | 4, 5, 4 | 4.33 | 26.00 |
| Problem solving | 25% | 4, 3, 4 | 3.67 | 18.33 |
| Commercial awareness | 20% | 3, 3, 3 | 3.00 | 12.00 |
| Organisation | 15% | 4, 4, 4 | 4.00 | 12.00 |
| Collaboration | 10% | 5, 4, 5 | 4.67 | 9.33 |
| Weighted total | 77.7 / 100 |
The total makes candidates comparable, and the rows show why: this candidate scores well on client communication, the criterion with the most weight, and is solid but unremarkable on commercial awareness, which could be developed in the role.
Use disagreement as a signal
Suppose a second candidate's problem-solving scores are 2, 5 and 4. The average, 3.67, hides the fact that one interviewer saw a serious weakness and another saw real strength. A gap of two points or more between the highest and lowest score on a criterion is worth stopping for. Usually one of two things happened: the interviewers heard different evidence, or they read the scale differently. Compare the notes, not the conclusions. Sometimes the low score was right; sometimes it came from one poor answer early in the interview.
The biases a scorecard helps with
- The halo effect: one strong impression colours every score. Scoring criterion by criterion, from notes, weakens it.
- Similar to me: interviewers warm to people like themselves. Fixed criteria keep attention on the job.
- Contrast: an average candidate looks brilliant after a weak one. Anchored scales compare each candidate with the standard, not with the previous interview.
- Overall impression: "I just liked her" is not a criterion. If it matters, it belongs on the scorecard with a definition and a weight.
Build it in the Weighted Hiring Scorecard
The free Weighted Hiring Scorecard is built around this process:
- Name the evaluation and the role, and add criteria with weights; it checks they total 100%.
- Each interviewer scores each candidate on each criterion from 1 to 5.
- It averages the panel's scores, works out each criterion's weighted contribution and every candidate's weighted total out of 100, and ranks the candidates side by side.
- Panel alignment flags any criterion where the interviewers' scores are further apart than your disagreement threshold, two points to begin with.
- It tells you whether the evaluation is complete, and exports an Excel-compatible XLSX and a recap image for the hiring decision.
Everything runs in your browser; nothing is uploaded. If you are hiring because the business is growing, the 12-Month Cash Flow Forecast shows whether the new salary fits, and our break-even guide shows how much more you need to sell to cover it.
Frequently asked questions
How many criteria should a scorecard have?
Four to six. Enough to cover what the job needs, few enough that interviewers can keep them distinct during an interview.
Should all criteria have the same weight?
Only if they really matter equally, which is rare. Weighting the two or three most important criteria more heavily stops a strong score on a minor criterion from carrying the decision.
Is a 1 to 5 scale enough?
Yes. Interviewers use a five-point scale more consistently than a ten-point one, especially when each point has a written anchor.
What if the interviewers disagree?
Compare the evidence each of them wrote down for that criterion. If they heard different things, the candidate may be inconsistent; if they heard the same thing and scored it differently, agree what the anchors mean before the next interview.
Does a scorecard make hiring fair?
It helps you apply the same standard to every candidate and record why you decided as you did. It does not replace judgment, and you should still follow employment law and your own organisation's policies.
