AI Model Trainer - English & Hindi

About the Role

hackajob is partnering with Mercor to find exceptional tech professionals for the AI Model Trainer - English & Hindi role. This position is a contract opportunity, available for full-time or part-time engagement, with geography restricted to India and the USA.

Why This Role Exists

Mercor collaborates with leading AI teams to enhance the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are integral to various everyday and professional scenarios, and their effectiveness is directly tied to how clearly, accurately, and helpfully they respond to real user questions. This project specifically focuses on evaluating and improving the general chat behavior of large language models (LLMs). As an AI Model Trainer, you will assess model-generated responses across diverse topics, provide high-quality human feedback, and ensure AI systems communicate accurately, with sound reasoning, and in alignment with human expectations.

What You’ll Do

• Evaluate LLM-generated responses based on their effectiveness in answering user queries.
• Conduct thorough fact-checking using reliable public sources and external tools.
• Generate high-quality human evaluation data by annotating response strengths, identifying areas for improvement, and pinpointing factual inaccuracies.
• Assess the reasoning quality, clarity, tone, and completeness of responses.
• Ensure model responses adhere to expected conversational behavior and established system guidelines.
• Apply consistent annotations by diligently following clear taxonomies, benchmarks, and detailed evaluation guidelines.

Who You Are

• You possess a Bachelor’s degree.
• You are a native speaker or have primary fluency (ILR 5/C2 CEFR) in Hindi.
• You have significant experience using large language models (LLMs) and understand their typical use cases.
• You demonstrate excellent writing skills, capable of articulating nuanced feedback clearly.
• You have strong attention to detail, consistently noticing subtle issues.
• You are adaptable and comfortable working across various topics, domains, and customer requirements.
• You have a background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering).
• You possess excellent college-level mathematics skills.

Nice-to-Have Specialties

• Prior experience with RLHF, model evaluation, or data annotation work.
• Experience writing or editing high-quality written content.
• Experience comparing multiple outputs and making fine-grained qualitative judgments.
• Familiarity with evaluation rubrics, benchmarks, or quality scoring systems.

What Success Looks Like

• Identifying factual inaccuracies, reasoning errors, and communication gaps in model responses.
• Producing clear, consistent, and reproducible evaluation artifacts.
• Your feedback leads to measurable improvements in response quality and user experience.
• Mercor customers trust their AI systems due to your pre-release issue identification.

Why Join Mercor

At Mercor, you will operate at the cutting edge of human-in-the-loop AI development, directly influencing the behavior of advanced language models globally. This role offers flexible, remote contract work and a meaningful opportunity to contribute to AI systems used by millions. Competitive contract rates are offered, aligning with required expertise and scope of work.

Key skills/competency

• Large Language Models (LLMs)
• AI Model Evaluation
• Natural Language Processing (NLP)
• Fact-Checking
• Data Annotation
• Analytical Thinking
• Conversational AI
• Quality Assurance
• Linguistics
• Human Feedback (RLHF)