Proctor Quality and Learning Manager, Scaling Operations
London · New York, NY · Pittsburgh · GBP 90,630-122,617 · 5+ years experience · Remote · Posted Sep 30, 2026
At a glance
- Experience
- 5+ years experience
- Workplace
- Remote
- Location
- London · New York, NY · Pittsburgh
- Salary
- GBP 90,630-122,617 · Base salary
About the role
- As the Proctor Quality and Learning Manager (L3) in Scaling Operations at Duolingo, you will own the end-to-end quality standard and learning system for the global proctor network of the Duolingo English Test (DET).
- You will define what correct decisions look like, run the calibration and adjudication program, and build ongoing training that measurably improves decision quality.
- This is an individual-contributor role within DET Proctoring Operations, with functional leadership over Training Specialists and adjudicators executing the program.
Responsibilities
- Own the DET proctoring quality standard by maintaining rubrics, decision rules, examples, and guidance that define accurate decisions across routine and difficult edge cases.
- Lead calibration and adjudication by operating recurring Proctor Certification Test cycles, resolving disputed decisions, identifying patterns of disagreement, and ensuring consistent and auditable outcomes.
- Build and run an ongoing learning program, publishing a forward-looking training calendar, delivering monthly interventions, and launching targeted training in response to quality signals, threats, or workflow changes.
- Measure effectiveness using calibration results, audits, and performance trends to identify learning needs, evaluate training impact, and decide when to clarify standards, retrain, or update guidance.
- Guide specialist execution by setting expectations for Training Specialists and adjudicators, assigning deliverables, reviewing outputs, and operating onboarding and certification that ramp new specialists quickly.
- Maintain durable sources of truth by keeping calibration, adjudication, and training runbooks current so critical processes are documented and auditable.
- Evolve quality alongside proctoring by translating new threats and workflow changes into clear standards, training, and quality controls in partnership with Psychometrics, Security, Operations, Product, and Engineering.
- Represent the proctoring craft by bringing frontline proctor evidence into product experiments and workflow decisions and assessing whether launched changes preserve decision quality.
- Connect human and technology-assisted quality by establishing benchmarks for AI-assisted workflows and validating automated capabilities against real sessions before deployment.
Requirements
- 5+ years in assessment development, scoring/rater quality, certification, quality assurance, or a field involving complex, consequential judgments, including experience aligning reviewers to a shared standard through calibration programs.
- Adult-learning program experience spanning onboarding, certification, ongoing training, remediation, and effectiveness measurement.
- Exceptional attention to detail and strong judgment in ambiguous cases, with the ability to produce clear, defensible decisions and guidance.
- Analytical fluency to interpret quality data, disagreement patterns, and performance trends and translate them into practical interventions.
- Clear written and verbal communication, with the ability to turn complex standards into precise, usable guidance and explain difficult calls.
- Cross-functional leadership without formal authority, including experience directing contractors or partners toward clear accountability and outcomes.
- Demonstrated skill in quality standards and rubric design, building living systems with decision logs, change histories, and controls for auditability.
- Experience defining quality benchmarks and validation methods for human-technology workflows, including AI-assisted processes.
- Adaptability and comfort revising standards and programs as risks, products, and technology change.
