10 days ago

Rubric Design Expert

Mercor

Hybrid
Part Time
$88,400
Hybrid

Job Overview

Job TitleRubric Design Expert
Job TypePart Time
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$88,400
LocationHybrid

Who's the hiring manager?

Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.

Uncover Hiring Manager

Job Description

About the Rubric Design Expert Role at Mercor

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

This contract position for a Rubric Design Expert is fully remote, requiring a commitment of 30–40 hours per week. Compensation is competitive, ranging from $40–$45 per hour.

Role Responsibilities

  • Create and evaluate rubrics for user-generated shopping tasks to enhance AI model performance.
  • Simulate realistic consumer behavior through prompt design to improve model outputs.
  • Ensure outputs align with quality standards and task objectives for consistent results.
  • Collaborate asynchronously with the AI lab to refine rubric coverage and ensure comprehensive evaluation.

Qualifications

To be successful as a Rubric Design Expert, candidates must possess:

Must-Have
  • Prior experience designing or using structured rubrics for data collection or evaluation.
  • Familiarity with AI training, evaluation, or prompt engineering tasks.
  • Excellent attention to detail and ability to assess nuanced task quality.
  • Strong written communication and organizational skills.

Start Date & Application Process

Immediate availability is required for this role. The application process typically takes 20–30 minutes to complete:

  • Express interest by confirming availability and experience.
  • Expect a brief discussion or call to align on project expectations.
  • Selected candidates will begin task work within 24 hours.

For more details about the interview process and platform information, please visit: talent.docs.mercor.com/welcome/welcome. For any help or support, reach out to: support@mercor.com.

Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Key skills/competency

  • Rubric Design
  • AI Model Evaluation
  • Prompt Engineering
  • Data Quality Assurance
  • User Behavior Simulation
  • Written Communication
  • Organizational Skills
  • Attention to Detail
  • Asynchronous Collaboration
  • Contract Work

Tags:

Rubric Design Expert
Rubric design
AI evaluation
prompt engineering
data quality
user behavior
task analysis
model performance
asynchronous collaboration
quality assurance
data collection
AI training
machine learning
prompt engineering
data evaluation
AI models
natural language processing
data analysis
task automation
quality assurance tools
project management software

Share Job:

How to Get Hired at Mercor

  • Research Mercor's mission: Understand Mercor's vision in connecting AI talent with research labs.
  • Tailor your resume: Highlight rubric design, AI evaluation, and prompt engineering experience.
  • Showcase attention to detail: Provide specific examples of quality assurance and meticulous work.
  • Prepare for asynchronous collaboration: Emphasize strong written communication and organizational skills.
  • Complete all application steps: Ensure your AI interview and application are fully submitted.

Frequently Asked Questions

Find answers to common questions about this job opportunity

Explore similar opportunities that match your background