Want to get hired at Crossing Hurdles?

Generalist Evaluator Expert

Crossing Hurdles

HybridHybrid

Original Job Summary

Position Overview

The Generalist Evaluator Expert role at Crossing Hurdles is an hourly contract position supporting top AI research labs. In this role, you will design and optimize prompts, define evaluation standards, and rate AI model outputs.

Role Responsibilities

  • Design and optimize detailed prompts with multiple constraints for language model evaluation.
  • Define evaluation standards and develop comprehensive rubrics for consumer contexts.
  • Test and grade AI outputs against set expectations.
  • Support benchmarking and quality assurance to maintain prompt rigor.
  • Collaborate with team members in quality assurance review processes.

Requirements

  • BS or BA degree from a reputable institution (completed or in progress).
  • Strong writing skills, critical thinking, and instructional clarity.
  • Ability to work independently and meet deadlines.
  • Familiarity with ChatGPT or similar language tools.
  • Experience in teaching or research is preferred.
  • Access to a desktop or laptop computer (Chromebooks are not supported).

Application Process

  • Complete an AI-led interview (approximately 15 minutes).
  • Complete a 45-minute written assessment focused on writing rubrics.
  • If selected, you will be onboarded to the project.

Key skills/competency

  • Prompts
  • Evaluation
  • Rubrics
  • Quality Assurance
  • Benchmarking
  • AI Models
  • ChatGPT
  • Research
  • Writing
  • Collaboration

How to Get Hired at Crossing Hurdles

🎯 Tips for Getting Hired

  • Customize your resume: Highlight AI prompt design and evaluation skills.
  • Showcase technical expertise: Emphasize your experience with language models.
  • Prepare examples: Detail independent work and meeting deadlines.
  • Practice interviews: Review AI, rubric development and assessment questions.

📝 Interview Preparation Advice

Technical Preparation

Review language model functionalities.
Practice designing structured AI prompts.
Study evaluation rubric development.
Test ChatGPT and similar AI platforms.

Behavioral Questions

Describe independent project experiences.
Explain meeting tight deadlines.
Share collaboration in quality assurance.
Discuss critical problem-solving in tasks.