Data Evaluation Expert
Mercor
Job Overview
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
About Mercor
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Mercor is a fully remote company.
Position Overview: Data Evaluation Expert
This is a contract position focused on generalist data evaluation, offering compensation between $30–$40 per hour.
Role Responsibilities
- Review, evaluate, and annotate AI outputs with extreme attention to detail.
- Follow complex guidelines precisely and apply them consistently across tasks.
- Identify subtle errors, edge cases, and quality issues others might miss.
- Provide clear, structured feedback to improve downstream AI training quality.
- Maintain high accuracy across repetitive, high-volume workflows.
- Flag unclear instructions or ambiguous cases proactively.
Qualifications
Must-Have
- Exceptionally detail-oriented and quality-obsessed mindset.
- Prior experience as a CB / reviewer / rater in human data or AI training pipelines.
- Proven ability to follow nuanced instructions without deviation.
- Strong written communication and reasoning skills.
- High reliability, consistency, and comfort with repetitive precision work.
- Ability to work independently and maintain focus over long task batches.
Preferred
- Experience with large-scale AI data annotation or evaluation programs.
- Background in QA, operations, trust & safety, or structured review work.
- Familiarity with multi-step rubric-based evaluations.
- Comfort giving calibrated feedback aligned with strict quality bars.
Application Process
The application process takes approximately 20–30 minutes to complete:
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
For any help or support, reach out to: support@mercor.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Key skills/competency
- Data Evaluation
- AI Output Annotation
- Quality Assurance
- Attention to Detail
- Guideline Adherence
- Error Identification
- Structured Feedback
- Precision Work
- Repetitive Task Management
- Problem-Solving
How to Get Hired at Mercor
- Research Mercor's culture: Study their mission, values, recent news, and employee testimonials on LinkedIn and Glassdoor, focusing on their AI and talent network.
- Tailor your resume: Customize your application to highlight experience in data evaluation, AI training pipelines, or quality assurance roles.
- Prepare for the AI interview: Practice articulating experiences demonstrating extreme attention to detail, adherence to complex guidelines, and strong written communication.
- Showcase precision & consistency: Emphasize your ability to work independently, maintain focus during repetitive tasks, and provide calibrated, structured feedback.
- Highlight relevant experience: Detail any background in large-scale data annotation, QA, operations, or trust & safety, showcasing your structured review capabilities.
Frequently Asked Questions
Find answers to common questions about this job opportunity
Explore similar opportunities that match your background