9 days ago

Rubric Creation Expert

Mercor

Hybrid
Part Time
$77,500
Hybrid

Job Overview

Job TitleRubric Creation Expert
Job TypePart Time
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$77,500
LocationHybrid

Who's the hiring manager?

Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.

Uncover Hiring Manager

Job Description

About The Job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

This is a Contract position for a Rubric Creation Expert, offering compensation of $40–$45/hour and requiring a commitment of 30–40 hours/week. The role is fully Remote.

Role Responsibilities

  • Create and evaluate rubrics for user-generated shopping tasks to enhance AI model performance.
  • Simulate realistic consumer behavior through prompt design to improve model outputs.
  • Ensure outputs align with quality standards and task objectives for consistent results.
  • Collaborate asynchronously with the AI lab to refine rubric coverage and ensure comprehensive evaluation.

Qualifications

Must-Have
  • Prior experience designing or using structured rubrics for data collection or evaluation.
  • Familiarity with AI training, evaluation, or prompt engineering tasks.
  • Excellent attention to detail and ability to assess nuanced task quality.
  • Strong written communication and organizational skills.

Start Date

Immediate availability required.

Application Process (Takes 20–30 mins to complete)

  • Express interest by confirming availability and experience.
  • Expect a brief discussion or call to align on project expectations.
  • Selected candidates will begin task work within 24 hours.

Resources & Support

For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome

For any help or support, reach out to: support@mercor.com

Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Key skills/competency

  • Rubric creation
  • AI model evaluation
  • Prompt engineering
  • Data quality assurance
  • Structured evaluation
  • Asynchronous collaboration
  • Written communication
  • Organizational skills
  • Attention to detail
  • Consumer behavior simulation

Tags:

Rubric Creation Expert
Rubric creation
AI evaluation
Prompt engineering
Data quality
Consumer behavior
Asynchronous collaboration
Quality assurance
Task objectives
Model performance
Structured evaluation
AI
Machine Learning
Data Annotation
NLP
Prompt Design
Large Language Models (LLMs)
Data Collection
Quality Control
Model Training
AI Systems

Share Job:

How to Get Hired at Mercor

  • Research Mercor's mission: Understand how Mercor connects talent with AI labs and their investor backing.
  • Tailor your resume: Highlight rubric design, AI evaluation, and prompt engineering experience for Mercor.
  • Demonstrate attention to detail: Showcase your meticulousness in communication and application materials for Mercor.
  • Prepare for a quick turnaround: Be ready for an immediate start and a streamlined interview process with Mercor.
  • Complete AI interview: Follow Mercor's explicit instructions for the AI interview and application steps promptly.

Frequently Asked Questions

Find answers to common questions about this job opportunity

Explore similar opportunities that match your background