Data Science Expert (Train AI Models Part Time!)
hackajob · United States
- Hybrid
- Full-time
- $80,000 / year
- United States
Tailored resume — keyword-matched to this role.
Hiring manager — we find who's hiring.
Intro email — drafted to reach them directly.
Job highlights
- Evaluate AI system performance on data science tasks.
- Design grading criteria for data science deliverables.
- Score AI and human work samples rigorously.
- Provide detailed written justifications for scores.
- Collaborate with senior reviewers for feedback.
About the role
Role Overview
Mercor is partnering with a leading AI research organization to engage experienced data scientists for a project focused on evaluating how well AI systems perform real-world data science work. Rather than producing deliverables yourself, you will define what excellent work looks like: designing task-specific grading criteria and scoring completed work samples with rigorous, well-reasoned written justifications.Key Responsibilities
- Design precise, task-specific grading criteria for real-world data science deliverables (analyses, models, dashboards, experiment readouts, and written recommendations)
- Score AI-generated and human work samples against those criteria, with detailed written justifications for every score
- Apply consistent, evidence-based judgment so that scores are reproducible and defensible
- Incorporate structured feedback from senior reviewers and iterate quickly on your work
Ideal Qualifications
- 5+ years of professional data science experience in industry
- Background in business operations, product, or growth data science at top-tier technology companies
- Deep fluency in experiment design and A/B testing, metric definition, SQL/Python analysis, and communicating findings to executive stakeholders
- Exceptionally strong written communication
- Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers
- Prior experience with AI training, evaluation, or human-data projects is a strong plus
Application Process
- Submit your resume or relevant technical background to get started
- Qualified applicants may be asked to complete a brief technical assessment or submit additional information
Key skills/competency
- Data Science
- AI Model Evaluation
- Grading Criteria Design
- Scoring Work Samples
- Written Justifications
- Experiment Design
- A/B Testing
- SQL
- Python Analysis
- Communication Skills
Skills & topics
- Data Science
- AI Evaluation
- Machine Learning
- Model Scoring
- Data Analysis
- Python
- SQL
- Experiment Design
- A/B Testing
- Technical Writing
How to get hired
- Tailor your resume: Highlight 5+ years of data science experience, business operations, product, or growth focus, and strong written communication skills.
- Showcase expertise: Emphasize proficiency in experiment design, A/B testing, metric definition, SQL/Python analysis, and executive communication.
- Address AI experience: Clearly state any prior involvement in AI training, evaluation, or human-data projects as a significant advantage.
- Prepare for assessment: Be ready for a potential technical assessment that may test your evaluation and justification abilities.
Technical preparation
Review AI evaluation methodologies.,Practice writing clear scoring justifications.,Refresh SQL and Python analysis skills.,Study experiment design principles.
Behavioral questions
How do you ensure scoring consistency?,Describe a time judgment was reviewed.,How do you incorporate critical feedback?,How do you communicate complex findings?
Frequently asked questions
- What is the primary focus of the Data Science Expert role at Mercor for hackajob?
- The primary focus for a Data Science Expert at Mercor, through hackajob, is to evaluate the performance of AI systems in completing real-world data science tasks. This involves designing grading criteria and scoring AI-generated work, rather than producing deliverables yourself.
- What are the key responsibilities for this Data Science Expert position?
- Key responsibilities include designing specific grading criteria for data science outputs, scoring AI and human work samples against these criteria with detailed written justifications, applying consistent judgment, and incorporating feedback from senior reviewers.
- What experience is essential for the Data Science Expert role with Mercor?
- Essential qualifications include at least 5 years of professional data science experience in industry, ideally with a background in business operations, product, or growth at top-tier tech companies. Strong written communication and expertise in experiment design, A/B testing, SQL, and Python analysis are also crucial.
- Is prior experience with AI beneficial for this Data Science Expert job?
- Yes, prior experience with AI training, evaluation, or human-data projects is considered a strong plus for this Data Science Expert role, demonstrating relevant domain knowledge.
- How does the application process work for the Data Science Expert role via hackajob?
- The application process begins with submitting your resume or relevant technical background. Qualified candidates may then be invited to complete a brief technical assessment or provide additional information.
- What technical skills are most important for this Data Science Expert position?
- Key technical skills include deep fluency in experiment design, A/B testing, metric definition, SQL/Python analysis. While not producing deliverables, a strong analytical foundation is vital for evaluation.
- How will my judgment be assessed as a Data Science Expert?
- Your judgment will be assessed based on its consistency, evidence-based reasoning, and reproducibility. You should be comfortable having your scoring and justifications reviewed and calibrated against peers.
- Is this a full-time or part-time role for a Data Science Expert?