Senior Product Operations Manager, Evaluation
Harvey · San Francisco, CA
- On site
- Full-time
- $150,000 / year
- San Francisco, CA
Email the hiring manager to get a response.
Get their verified email + an intro that's ready to send.
Subject: Interested in the Senior Product Operations Manager, Evaluation role at Harvey
Hi Riley — I came across the Senior Product Operations Manager, Evaluation opening and wanted to reach out directly. I've spent the last few years doing exactly this kind of work, and Harvey stood out because…
✎ Personalized to your résumé after sign-up.
- ✓ Verified email of the hiring manager
- ✓ Intro email personalized to your résumé
- ✓ $9/mo = unlimited — any job link
Secure checkout · cancel anytime
Job highlights
- Build and scale AI evaluation systems.
- Operationalize evaluation methodologies for product.
- Manage data providers and legal experts.
- Improve evaluation tooling and automation.
- Ensure model accuracy and trust globally.
About the role
About Harvey
At Harvey, we’re transforming how legal and professional services operate. By combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise, we’re reshaping how critical knowledge work gets done for decades to come.
This is a rare chance to help build a generational company at a true inflection point. With 1500+ customers in 60+ countries, strong product-market fit, and world-class investor support, we’re scaling fast and defining a new category in real time. The work is ambitious, the bar is high, and the opportunity for growth — personal, professional, and financial — is unmatched.
Our team moves fast, takes ownership, and is deeply committed to the mission — operating with intensity, staying close to our customers, and pushing each other for excellence. We live by three values: Decisiveness, Simplicity, and Job's Not Finished. We act quickly on clear judgment over perfect information, we believe simplicity is what scales, and we're never satisfied with where we are. If you want to do the best work of your career alongside people who share that drive, we'd love to build with you.
At Harvey, the future of professional services is being written today — and we’re just getting started.
Role Overview
We’re looking for a technical, systems-minded operator to build and scale the evaluation engine behind Harvey’s platform. As we expand globally, ensuring our models behave reliably, accurately, and jurisdictionally correctly is mission-critical—and evaluation complexity is increasing 10x.
As a member of our Product Operations team, you’ll work closely with Applied Legal Researchers, Product, Engineering, AI Research, and human data providers to operationalize evaluation methodologies and embed them into our product development lifecycle. You’ll create the workflows, systems, and tooling that make evaluation a first-class product capability at Harvey.
This is a high-ownership role for someone who thrives in ambiguity, loves building structure, and wants to help scale the evaluation infrastructure of a global AI company.
What You'll Do
- Build and scale the systems that power model and product evaluations across Harvey
- Run intake, triage, and prioritization for the evaluation request queue, routing capacity to the highest-value coverage gaps
- Embed evaluation workflows and readiness checkpoints into the product development lifecycle
- Create the single source of truth for evaluation status, results, history, and launch readiness
- Turn expert-designed evaluation methodologies into scalable, repeatable operational processes
- Manage human data providers and stand up our internal contract-attorney pipeline, ensuring evaluation quality meets legal standards
- Work with Engineering and Research to improve evaluation tooling, automation, and dashboards
- Drive evaluation readiness for major product and model launches across geographies and jurisdictions
- Document and operationalize evaluation governance as complexity increases
- Help define how Harvey ensures model accuracy, reliability, and trust at global scale
What You Have
- 4–7+ years in technical program management, product operations, research operations, or evaluation/benchmarking roles
- Experience working with ML/AI evaluations, benchmarking frameworks, or scientific workflows
- Comfort with statistical methodologies and SQL or Python, or similar tools to interpret evaluation data (either natively or with AI tool support)
- Strong business acumen with an ability to apply an ROI-focused mindset to scaling
- Ability to work deeply with legal experts and operationalize complex evaluation methodologies
- Strong cross-functional coordination skills across Product, Engineering, Research, and data providers/vendors
- High attention to detail and a bias toward clarity, rigor, and reproducibility
- Ability to navigate an evolving landscape and bring order to complex systems
- Strong communication skills and comfort translating technical nuance for diverse stakeholders
- Desire to do whatever it takes to make evaluation systems successful—from writing documentation to diagnosing pipeline issues
Depending on your location, an Applicant Privacy Notice may apply to you. You can find all of our Applicant Privacy Notices [here].
Harvey is an equal opportunity employer and does not discriminate on the basis of race, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition, or any other basis protected by law.
We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made by emailing accommodations@harvey.ai
Key skills/competency
- Senior Product Operations Manager
- AI Evaluation
- ML Evaluation
- Product Operations
- Technical Program Management
- Research Operations
- SQL
- Python
- Scalability
- Workflow Automation
Skills & topics
- Senior Product Operations Manager
- Product Operations
- Evaluation
- AI
- Machine Learning
- Technical Program Management
- Research Operations
- SQL
- Python
- Scalability
How to get hired
- Tailor your resume: Highlight experience in AI evaluation, technical program management, and operationalizing complex systems.
- Showcase technical skills: Emphasize proficiency in SQL, Python, and ML/AI benchmarking frameworks.
- Demonstrate operational expertise: Detail your experience in building scalable workflows and managing cross-functional teams.
- Understand Harvey's values: Align your application with their focus on decisiveness, simplicity, and continuous improvement.
- Prepare for technical and behavioral questions: Be ready to discuss your experience with AI model evaluation and handling ambiguity.
Technical preparation
Behavioral questions
Frequently asked questions
- What is the primary focus of the Senior Product Operations Manager, Evaluation role at Harvey?
- The Senior Product Operations Manager, Evaluation at Harvey will focus on building and scaling the engine that evaluates Harvey's AI models and platform. This involves operationalizing evaluation methodologies, creating scalable processes, and ensuring the reliability and accuracy of AI models across different jurisdictions.
- What kind of technical background is required for this role?
- The role requires a technical, systems-minded operator with experience in technical program management, product operations, or evaluation/benchmarking. Comfort with statistical methodologies and tools like SQL or Python for interpreting evaluation data is essential. Experience with ML/AI evaluations and benchmarking frameworks is also key.
- How does Harvey approach product development and evaluation?
- Harvey embeds evaluation workflows and readiness checkpoints into their product development lifecycle. They aim to make evaluation a first-class product capability, with a focus on scalability, repeatability, and ensuring model accuracy, reliability, and trust at a global scale.
- What are Harvey's core values, and how do they influence the work environment?
- Harvey's core values are Decisiveness, Simplicity, and 'Job's Not Finished.' These values drive a fast-paced environment where the team acts quickly, prioritizes simplicity for scalability, and is never satisfied with the status quo. They foster a culture of high ownership and continuous improvement.
- What opportunities for growth are available at Harvey?
- Harvey offers significant opportunities for personal, professional, and financial growth. As a company defining a new category in AI for professional services and scaling rapidly, employees have the chance to grow alongside the company and potentially do the best work of their careers.
- What is Harvey's commitment to diversity and inclusion?
- Harvey is an equal opportunity employer committed to diversity and inclusion. They do not discriminate based on race, gender, sexual orientation, national origin, disability, age, or other protected statuses. They also provide reasonable accommodations for applicants with disabilities.
- What does 'operationalize evaluation methodologies' mean in the context of this role?
- It means taking the methodologies designed by experts (like Applied Legal Researchers) and turning them into scalable, repeatable processes and systems that can be integrated into the product development lifecycle. This includes defining workflows, creating tooling, and ensuring quality control for AI model evaluations.
Similar roles
Open positions we recommend based on this role.