Data Science Expert - AI Evaluation
Mercor · United States
- Hybrid
- Part-time
- $170,000 / year
- United States
Tailored resume — keyword-matched to this role.
Hiring manager — we find who's hiring.
Intro email — drafted to reach them directly.
Job highlights
- Design grading criteria for data science deliverables.
- Score AI and human work against criteria.
- Ensure reproducible, evidence-based judgments.
- Incorporate feedback and iterate quickly.
- Improve AI model performance independently.
About the role
Data Science Expert at Mercor
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Role Responsibilities
- Design precise, task-specific grading criteria for real-world data science deliverables such as analyses, models, dashboards, and experiment readouts.
- Score AI-generated and human work samples against criteria with detailed written justifications for every score.
- Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate quickly on your work.
- Work independently and asynchronously to meet deadlines while improving AI model performance.
Qualifications
Must-Have
- 5+ years of professional data science experience in industry.
- Background in business operations, product, or growth data science at top-tier technology companies.
- Deep fluency in experiment design and A/B testing, metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
- Exceptionally strong written communication.
- Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.
Preferred
- Prior experience with AI training, evaluation, or human-data projects.
Application Process
The application process takes 20–30 minutes to complete. Submit your resume or relevant technical background to get started. Qualified applicants may be asked to complete a brief technical assessment or submit additional information.
Resources & Support
For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
For any help or support, reach out to: support@mercor.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Key skills/competency
- Data Science
- AI Evaluation
- Experiment Design
- A/B Testing
- SQL
- Python
- Metric Definition
- Written Communication
- Data Analysis
- Product Analytics
Skills & topics
- Data Science
- AI Evaluation
- Data Analysis
- Experiment Design
- A/B Testing
- SQL
- Python
- Metric Definition
- Remote Work
- Contract Role
- Data Scientist
- Machine Learning Evaluation
- Product Analytics
- Growth Analytics
- Business Operations
How to get hired
- Tailor your resume: Highlight 5+ years of data science experience, business operations, product, or growth analytics at top tech firms.
- Showcase expertise: Emphasize fluency in experiment design, A/B testing, metric definition, SQL/Python analysis, and executive communication.
- Demonstrate communication skills: Provide examples of exceptionally strong written communication and detail-oriented work.
- Prepare for assessments: Be ready for a technical assessment and calibration against peer judgment.
- Follow the process: Complete all steps in the Mercor application and AI interview.
Technical preparation
Master SQL and Python for data analysis.,Practice experiment design and A/B testing.,Define key metrics for business outcomes.,Prepare to justify scoring decisions.
Behavioral questions
Describe a complex data analysis you led.,How do you ensure consistency in your work?,How do you handle feedback on your judgment?,Explain technical findings to non-technical stakeholders.
Frequently asked questions
- What is the application process for a Data Science Expert at Mercor?
- The application process for the Data Science Expert role at Mercor takes approximately 20-30 minutes. It involves submitting your resume or technical background, and potentially completing a brief technical assessment or providing additional information. Mercor's team reviews applications daily.
- What are the core responsibilities of a Data Science Expert at Mercor?
- As a Data Science Expert at Mercor, you will design grading criteria for data science deliverables, score AI and human work against these criteria with detailed justifications, and ensure your judgments are consistent and reproducible. You'll also incorporate feedback and work to improve AI model performance.
- What qualifications are essential for the Data Science Expert position at Mercor?
- Mercor requires at least 5 years of professional data science experience, with a background in business operations, product, or growth analytics from top technology companies. Strong proficiency in experiment design, A/B testing, metric definition, SQL/Python analysis, and exceptional written communication skills are crucial.
- Is there an opportunity for remote work as a Data Science Expert at Mercor?
- Yes, the Data Science Expert position at Mercor is a remote role, allowing you to work from anywhere.
- What is the compensation for a Data Science Expert at Mercor?
- The compensation for the Data Science Expert role at Mercor is between $120-$170 per hour on a contract basis.
- Does Mercor have preferred qualifications for this Data Science Expert role?
- Yes, Mercor prefers candidates who have prior experience with AI training, evaluation, or human-data projects for the Data Science Expert position.
- How can I get support during the Mercor application process for the Data Science Expert role?
- For any help or support regarding the application process for the Data Science Expert role, you can reach out to support@mercor.com. Additionally, you can find information about the interview process at https://talent.docs.mercor.com/welcome.