15 hours ago
Prompt Evaluator
North Star Staffing
Hybrid
Contractor
$40,000
Hybrid
Job Overview
Job TitlePrompt Evaluator
Job TypeContractor
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$40,000
LocationHybrid
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
About the Prompt Evaluator Role at North Star Staffing
North Star Staffing is seeking a Generalist Evaluator Expert to support a high-impact AI research initiative. This remote, hourly contract role focuses on crafting structured prompt–answer pairs and evaluation rubrics to train and benchmark advanced language models. It's an ideal short-term, flexible engagement for strong writers and analytical thinkers who excel at transforming complex ideas into clear, structured instructions.
Key Responsibilities
- Design detailed prompts with multiple constraints and structured instructions.
- Author high-quality "golden answers" aligned to defined expectations.
- Develop comprehensive evaluation rubrics and grading criteria.
- Run prompts through AI systems and assess outputs against standards.
- Participate in quality assurance cycles to ensure rigor and consistency.
- Contribute to benchmarking workflows before dataset integration.
Ideal Qualifications
- BS or BA from a reputable institution (completed or in progress).
- Strong writing, editing, and critical thinking skills.
- Ability to work independently and meet structured deadlines.
- Familiarity with ChatGPT or similar AI tools for personal or professional use.
- Experience in teaching or research preferred.
- Must be based in the United States or Canada.
Project Details
- Commitment of at least 20 hours per week.
- Expected duration of approximately 2 months.
- Structured project environment with clear goals and tooling.
More About the Opportunity
- Fully remote, flexible schedule.
- Independent contractor engagement.
- Weekly payments via Stripe or Wise.
- Project scope may be extended based on performance.
Key skills/competency
- Prompt Engineering
- AI Evaluation
- Technical Writing
- Critical Thinking
- Language Models
- ChatGPT
- Rubric Design
- Quality Assurance
- Analytical Skills
- Research
How to Get Hired at North Star Staffing
- Research North Star Staffing's culture: Study their mission, values, recent news, and employee testimonials on LinkedIn and Glassdoor.
- Tailor your resume: Highlight writing, critical thinking, and AI familiarity. Customize for prompt evaluation roles.
- Prepare for the AI interview: Practice articulating your analytical and writing skills concisely for North Star Staffing.
- Master rubric design: Sharpen your ability to create clear, structured evaluation criteria for the written assessment.
- Showcase your AI experience: Be ready to discuss your personal or professional use of ChatGPT or similar AI tools.
Frequently Asked Questions
Find answers to common questions about this job opportunity
01What does a Prompt Evaluator at North Star Staffing do?
02Is the Prompt Evaluator role remote, and where are candidates expected to be located?
03What is the expected time commitment and duration for this Prompt Evaluator contract?
04What qualifications are North Star Staffing looking for in a Prompt Evaluator?
05How will I be compensated for the Prompt Evaluator role?
06What does the application process involve for the Prompt Evaluator position?
07Can you elaborate on the AI research initiative this Prompt Evaluator will support?
08What kind of project environment can I expect as a Prompt Evaluator?
Explore similar opportunities that match your background