Reviewer Updates: October 2022

jmpld40

Reviewers play an important role in the experience of our members and customers. In turn, you play a critical role in Topcoder’s mission of connecting members and customers to produce high-quality, timely outcomes.

The purpose of this note is to share an update to the Reviewer Selection process (how we assign Review opportunities to Review applicants). We are making this update to better align the Reviewer Selection process with our mission to produce high-quality, timely outcomes.

Effective immediately:

• Reviewer selection will be based on applications received within the first 24 hours of posting

  • Goals: Support timeliness. Support the timezones of a global community.

• Applicants will be selected based on fit & qualification. This includes relevant skill match and performance history (timeliness and quality of past reviews). This article provides an overview and some examples.

  • Goal: Support high-quality.

• If 2 applicants are equally qualified, the assignment will go to the candidate who applied first.

  • Goal: Support timeliness.

Unless a data point clearly relates to your timeliness or work quality, it’s not a determining factor in your eligibility to get a Review assignment. Therefore please note that tenure, number of projects, and payment-related details are not considered selection criteria.

Lastly, thank you again for being a special part of what makes Topcoder. And we want to hear from you! Reply here with your feedback.

FAQ
Does this mean all the work will go to just a few Reviewers?
Not necessarily. It means that the most qualified and most responsive members will receive the most assignments. It also means a member won’t be denied (or given) work assignments based on their activity level or workload.

What if a challenge launches when I’m sleeping?
Challenges launch at different times. On whole, this means certain time zones have advantages for certain challenges, based on when those challenges are launched. To balance this:

• Applicants have 24 hours to apply and be considered for assignment
• Assignments are decided first & primarily by applicant qualifications and performance history. Speed/time of application is a tie-breaker.

What if 1 person gets assigned all the Review opportunities and projects suffer because they have too much work?
High-quality comes first in our mission. Each assignment represents a commitment to perform high-quality, timely work. We will respond to poor quality and/or late work with individual members in accordance with Development Review Role & Responsibilities.

Thank you!

Ghostar

We are going to amend the guidelines above a bit to include a few clarifications to the review assignments, in the case of more reviewer applications than available spots in a challenge:

1. Number of Reviews in Last 60 Days: If the skill match, qualification, and performance history fail to be a deciding factor, comparison would be performed on the number of reviews performed by applicant in the last 60 days (about 2 months). Applicants with a smaller number of reviews in the last 60 days (about 2 months) will be prioritized for assignment.
2. If there is still no way to easily decide who to assign, we will then fall back to the review application time/date as a tie-breaker, with the earlier review application being approved.

NOTE - "Performance history" above refers to complaints about reviewers from members, copilots, and PMs that have been deemed to be accurate. These are all fully investigated and tracked via Topcoder's internal SalesForce.

As we work through some changes to the new platform, expect to see more traditional reviews in the coming months (hopefully).