AI Conversation Evaluator
Talent Wall
Job Overview
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
AI Conversation Evaluator at Talent Wall
Mercor is seeking highly skilled Generalists (Bachelor’s degree minimum) to support a fast-moving project with one of the world’s leading AI labs. This initiative focuses on improving AI systems designed for therapeutic-style conversations by evaluating, comparing, and refining model-generated responses.
Your Responsibilities
- Review pairs of AI-generated replies and assess them against structured criteria (e.g., empathy, emotional insight, communication quality, and appropriateness for supportive dialogue).
- Produce clear, improved rewrites that better meet the project’s standards.
- Directly support research teams as they strengthen safe, supportive, and human-aligned AI behavior in mental-health-oriented conversation settings.
Project Details
This project has an accelerated timeline and requires contributors who can begin immediately following a mandatory onboarding session.
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Contract and Payment Terms
- You will be engaged as an independent contractor.
- This is a fully remote role that can be completed on your own schedule.
- Projects can be extended, shortened, or concluded early depending on needs and performance.
- Your work at Mercor will not involve access to confidential or proprietary information from any employer, client, or institution.
- Payments are weekly on Stripe or Wise based on services rendered.
- Please note: We are unable to support H1-B or STEM OPT candidates at this time.
Key skills/competency
- AI Evaluation
- Therapeutic Communication
- Empathy Assessment
- Emotional Intelligence
- Communication Quality
- Content Refinement
- Generative AI
- Natural Language Processing
- Linguistic Analysis
- AI Ethics
How to Get Hired at Talent Wall
- Understand Talent Wall's mission: Explore their focus on connecting talent with innovative projects and ethical AI development.
- Tailor your resume: Highlight experience in AI evaluation, linguistic analysis, content refinement, or therapeutic communication, specifically for the AI Conversation Evaluator role.
- Showcase critical thinking: Prepare to discuss instances where you've critically assessed complex information or provided constructive, criterion-based feedback.
- Demonstrate adaptability: Emphasize your ability to quickly learn new guidelines, adapt to fast-paced project timelines, and work independently in a remote setting.
- Highlight AI interest: Express genuine interest in contributing to ethical, safe, and human-centric AI development, especially in sensitive domains.
Frequently Asked Questions
Find answers to common questions about this job opportunity
Explore similar opportunities that match your background