Rubrics Generalist Expert

Role Overview

The Rubrics Generalist Expert role involves creating and evaluating rubrics to assess AI model performance on real-world shopping tasks. This project is short-term with an immediate start and potential for ongoing collaboration.

Key Responsibilities

• Create and evaluate rubrics for user-generated shopping tasks.
• Simulate realistic consumer behavior through prompt and scenario design.
• Ensure AI outputs align with quality standards and task objectives.
• Collaborate asynchronously with the AI research team to refine rubric coverage.

Ideal Qualifications

• Experience designing or using structured rubrics for data collection or evaluation.
• Familiarity with AI training, evaluation, or prompt engineering tasks.
• Excellent attention to detail and ability to assess nuanced task quality.
• Strong written communication and organizational skills.

Application Process

Express interest by confirming availability and relevant experience. Participate in a brief discussion to align on project expectations. Selected candidates will begin within 24 hours.

Contract and Payment Terms

• Engagement as an independent contractor.
• Fully remote role with flexible scheduling.
• Projects may be extended, shortened, or concluded early based on performance.
• Weekly payments via Stripe or Wise based on hours worked.
• H1-B or STEM OPT candidates cannot be supported.

Key skills/competency

rubric creation, evaluation, AI research, prompt design, consumer behavior, independent contractor, remote work, structured data, attention to detail, communication