Skip to content
Convergo360

How to choose raters for a 360 review

Pick one manager, all direct reports, about 4 peers and a few other staff. An independent administrator should approve the list, never the person being rated.

· 7 min read · Convergo360 team

Choose 360 raters from the people who work with the leader every week. Include one manager, all direct reports, about 4 peers and a few other staff, and make sure each anonymous group can reach at least 3 responses. An independent administrator should build and approve the list, and the person being rated should play no part in choosing who rates them.

The rest of this guide covers each rater group, what to do in small teams, how to stop any one person drowning in surveys, and a step by step process you can follow.

Who should rate a leader in a 360 review?

A leadership 360 uses five rater groups. Each sees a different side of the leader, which is why you need all of them.

Rater group Who belongs here How many Anonymous in the report
Self The person being rated 1 Shown separately
Manager The person they report to 1 No, labelled not anonymous
Peers Leaders at a similar level who work with them often 4 Yes, with at least 3 responses
Direct Reports Everyone who reports to them All Yes, with at least 3 responses
Other Staff People in other teams or lower levels who deal with them regularly 3 to 5 Yes, with at least 3 responses

Self, Manager, Peers and Direct Reports answer the full leadership survey. Other Staff answer a shorter survey of 8 statements, because they see less of the leader's daily work. You can read both sets of questions in our list of 21 leadership 360 questions.

Manager

There is only one manager, so their feedback cannot be anonymous. Tell managers this before the survey opens. Most write more useful comments once they know the leader will read them as theirs.

Direct reports

Invite all of them. Leaving some out makes the remaining reports easier to identify and raises the question of who chose whom. If a leader has 20 or more reports, inviting all of them still works, because each person completes only one survey about this leader.

Peers

Pick peers who work with the leader often enough to see how they make decisions, share information and handle disagreement. Sitting on the same leadership team is a good start. Sharing a project or a client is better.

Other Staff

These are people who rely on the leader without reporting to them, such as a payroll officer who chases their approvals or a team leader two levels down. Department lists are a good source. Three to five gives you a group large enough to stay anonymous.

How many raters do you need for a 360?

A typical manager with 6 direct reports ends up with about 14 raters plus their self review:

Group Raters
Manager 1
Direct Reports 6
Peers 4
Other Staff 3
Total 14

The number that matters most is 3. Any anonymous group with fewer than 3 responses is merged into a combined "Others" group for that leader, and its comments merge too. That rule protects raters, but it also means the leader loses the separate view from that group. Invite more than 3 wherever possible, because someone is always on leave or misses the deadline.

Our article on 360 feedback anonymity explains why 3 is the minimum and how comments are checked before release.

Should the person being rated choose their own raters?

They should stay out of it. A leader choosing their own raters will lean towards people they get on with, even without meaning to. Colleagues who were left out notice, and the leader's own manager will discount scores that look too generous.

Build the lists from reporting lines instead, and let an independent administrator approve them. The administrator can be someone in HR who is outside the leader's reporting line or an external consultant. Their job is to check that every list is fair, swap pairings that do not work and keep a record of why they made changes.

If a leader wants to suggest a peer who knows their work well, the administrator can take that into account. The final decision stays with the administrator.

How do you avoid survey fatigue?

Survey fatigue happens when the same people get asked over and over. Senior managers are most at risk, because they rate their peers, their own manager and every direct report who is also in the cycle.

Take a senior manager who rates 4 peers, has 2 direct reports in the cycle and is also asked to rate their own boss. That person has 7 surveys before any Other Staff requests arrive. Set a limit of 5 surveys per person and act when anyone goes over it.

Ways to keep the load down:

  1. Balance peer pairings so each leader gives 4 peer surveys and receives 4. Convergo360 assigns peers this way automatically and warns the administrator when anyone has more than 5 surveys.
  2. Spread Other Staff requests across a department instead of asking the same helpful person every time.
  3. Send one email per rater that lists how many surveys they have, rather than one email per survey.
  4. Give people two to three weeks to finish, with reminders only to those who have surveys outstanding.
  5. Run large organisations in waves by division if the numbers still look heavy.

What do you do with small teams?

Small teams are where most rater problems show up. A leader with 2 direct reports cannot have an anonymous Direct Reports group, because 2 is below the minimum of 3.

You have three options:

  1. Accept the merge. The 2 direct report responses join the peer responses in a combined "Others" group. The leader still gets the feedback, without a separate Direct Reports column.
  2. Add raters to other groups. More peers or Other Staff make the "Others" group larger and harder to read back to any individual.
  3. Tell the direct reports in advance that their answers will be combined with other groups. People write more freely once they know a group of 2 will not be shown on its own.

Avoid padding a small group with people who barely know the leader just to reach 3. Their Not observed answers and guesses add little, and anonymity is already handled by the merge.

When should you swap peers?

Automatic pairings get most peers right, but the administrator should check every list for these situations and swap where needed:

Situation Why it matters
The two people rarely work together Ratings will be guesses
There is an open grievance or disciplinary matter between them Feedback could become a weapon
They are close family or partners Ratings will lean one way
One reports to the other in a matrix structure The pair belongs in Manager or Direct Reports
The peer is leaving the company during the cycle The response is unlikely to arrive

Record each swap and the reason. If a leader questions their rater list later, you can show that changes were made for work reasons.

What about new starters?

Set a minimum time in the role before someone rates or is rated, and apply it to everyone. Three months is a sensible starting point, because a newer colleague has seen too little to rate most statements.

A new starter who does join the rater list will use Not observed often, which is fine. Those answers are left out of the averages. A leader who is new to their role should usually wait for the next cycle, since their scores would describe their first weeks more than their leadership.

How to choose 360 raters, step by step

  1. Load your people list with roles and reporting lines. A spreadsheet from payroll or your HR system is enough for a first cycle.
  2. Generate Self, Manager and Direct Reports from reporting lines.
  3. Assign 4 peers to each leader, balanced so each gives 4 peer surveys.
  4. Suggest Other Staff by department, aiming for 3 to 5 per leader.
  5. Apply your minimum time in role to remove very new starters.
  6. Have the independent administrator review every list, swap peers where needed and record reasons.
  7. Check survey counts and move surveys away from anyone with more than 5.
  8. Check every anonymous group can reach 3 responses, and decide in advance how you will explain merges to small teams.
  9. Approve the lists and send each rater a single email with their survey count and a single use link.

Rater lists contain personal information, so handle them under your usual POPIA controls. Our guide to 360 feedback and POPIA covers what to tell raters.

Next step

Pull your current reporting lines into a spreadsheet and count, for each leader, how many direct reports, likely peers and Other Staff they have. Any group under 3 shows you where to plan for a merge. If you want to see how Convergo360 builds and checks these lists, look at how it works or get in touch.

Common questions

How many raters should a 360 review have?

Aim for one manager, all direct reports, 4 peers and a few other staff. Each anonymous group needs at least 3 responses, so invite more than 3 where you can to allow for people who do not finish.

Should employees choose their own 360 raters?

No. When the person being rated picks their raters, they tend to choose people who like them, and the results lose credibility. Build lists from reporting lines and let an independent administrator approve them.

What happens if a group has fewer than 3 raters?

Its responses and comments merge into a combined Others group so that no individual's answers can be picked out. The manager's feedback is the exception and is shown on its own, labelled as not anonymous.

How many 360 surveys is too many for one rater?

More than 5 surveys in one cycle is a sensible warning point. Beyond that, people tend to rush their answers, so move some of that person's surveys to someone else.

See a sample report and how a cycle runs

A 30-minute walkthrough with someone who has run 360s before.

Book a demo