or apply directly on LawZero's site. We never take the application ourselves.
Is this posting real?
- This role has been open
- 81 days LawZero's roles stay open a median of 81 days
- Reposted
- No
- Salary listed
- No 0% of LawZero's roles list one
- Ghost-job risk at LawZero
- high 15 stale, 0 reposted of 16 open
- Hiring momentum
- 17 roles opened in the last 90 days ↑ up vs. the prior 90 days
- Last confirmed on the employer's board
- 2026-10-23
Measured from postings appearing on and disappearing from LawZero's own greenhouse board since 2026-08-03. Full hiring picture for LawZero.
About this role
The Director of Evaluations at LawZero will lead the establishment and growth of the Evaluations Team, focusing on the independent assessment of AI systems' capabilities and safety. Responsibilities include defining evaluation strategies, building a team, designing benchmarks and datasets, overseeing red-teaming efforts, and communicating evaluation results to stakeholders. This role is crucial for ensuring the integrity and trustworthiness of AI safety claims within the organization and the broader community.
- benefits
- 4/5
- freshness
- 1/5
- career value
- 5/5
- role clarity
- 5/5
- pay transparency
- 0/5
Scored from the posting itself — how clearly the role is described, how much it says about pay and benefits, and how recently it was listed. Not a judgement of LawZero as an employer.
What you need
- An advanced degree (MSc or higher) in machine learning, computer science, or a closely related field.
- 10+ years of experience in machine learning, with at least 5 years in a leadership role building or scaling technical teams working on real-world ML products.
- Hands-on expertise in designing and running large-scale evaluations of LLMs or other frontier ML systems across capabilities, safety, and adversarial robustness.
- A track record of building evaluation datasets, benchmarks, or interactive environments from scratch, including for safety-relevant properties such as honesty, sycophancy, refusal behaviour, and adversarial robustness.
- Strong written and verbal communication skills, including the ability to translate technical results for non-technical audiences such as executives, funders, and policymakers.
- Comfortable operating in a research-driven, fast-moving environment with significant ambiguity, and able to bring structure to it without slowing it down.
Nice to have
- Experience leading red-teaming exercises (automated, manual, or both) and working with third-party evaluation or red-teaming partners is a bonus.
- Experience working with third-party partners for benchmark and dataset creation
- Experience releasing open-source datasets, benchmarks, or evaluation tooling is a bonus.
- Familiarity with current AI safety policy and standards work (UK AISI, CAISI, NIST, EU AI Act, etc.) is a bonus.
- Experience contributing to or coordinating with external safety institutes, grant funders, or government bodies is a bonus.
What you get
- The opportunity to contribute to a unique mission with a major impact
- Comprehensive health benefits
- A minimum of 20 days vacation per year upon start
- A minimum retirement savings employer contribution of 4%
- Generous flexible benefits designed to contribute to your well-being
- A team of passionate experts in their field
Worth weighing
- This is a foundational hire, which may involve significant responsibility and ambiguity in defining the evaluation strategy.
- The role requires building a team from scratch, which could be challenging depending on the available talent pool.
- The focus on independent evaluations may lead to conflicts with other teams, requiring careful management of relationships.
- No salary or compensation details are provided in the posting.
Summarised from LawZero's posting. Read the full original.
Listed by LawZero on their greenhouse job board, last confirmed open on 2026-10-23. PitchMeAI is not the employer.
More roles at LawZero
- Machine Learning ManagerMontreal
- Senior ML Research DeveloperMontreal
- Senior ML Data Processing DeveloperMontreal
- Mathematical Scientist for AI Safety ResearchMontreal
- Senior Distributed ML EngineerMontreal
- ML Research Scientist (probabilistic inference)Montreal
- Senior ML Research ScientistMontreal
- Senior Manager, Talent AcquisitionMontreal