English (U.S. Native) AI Trainer & Evaluator (Remote, Hourly Contrator)
CNTXT AI · Brooklyn, NY
- Hybrid
- Full-time
- $30,000 / year
- Brooklyn, NY
This role may have been filled. Drop your résumé and we'll check if it's still open — or find you similar roles.
Job highlights
- Remote hourly contractor role for AI projects.
- Generate content, annotate data, evaluate AI responses.
- Requires native American English speaker.
- Focus on accuracy, tone, and cultural nuance.
- Flexible hours, project-based work.
About the role
AI Trainer and Evaluator
This is a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible hour basis. Project scopes vary and may include:
- Content generation: writing high-quality prompts and model responses, or recording high-quality voice samples, to guide AI learning across diverse topics.
- Data annotation: labeling, classifying, and structuring documents, tables, and other content to support AI training datasets.
- LLM evaluation: reviewing AI-generated responses for accuracy, reasoning quality, coherence, and cultural/linguistic appropriateness.
- Localization QA: ensuring terminology, tone, cultural nuance, and locale-specific details (units, references, names, dates) are consistently accurate across outputs.
This position exclusively seeks native speakers of American English born and raised in the United States.
Profile Requirements
- Native speakers of American English born and raised in the United States.
- Excellent editorial judgment in register, tone, punctuation, inclusivity, and cultural nuance, with extreme attention to detail.
- Ability to identify meaning drift, ambiguity, locale inconsistencies, and subtle errors, and to explain corrections clearly in writing.
- Ability to rigorously fact-check localized content (units, references, names, dates) using reliable sources and consistent reasoning.
- Ability to identify reasoning gaps, methodological errors, and unclear explanations even when language is fluent.
- Reliable, self-directed, and able to deliver consistent quality with clear communication and responsiveness across time zones.
Preferred Experience
- Familiarity with MQM/LQA concepts (severity, category, and root-cause thinking) for consistent quality decisions.
- Familiarity with QA workflows.
- Previous experience with AI data training, annotation, or evaluation.
About CNTXT AI
CNTXT AI builds artificial intelligence products and data solutions with a focus on making AI accurate, safe, and globally relevant for impact. Our work spans data services, custom AI solutions, and proprietary AI products, with deep expertise in Arabic-native and secure, sovereign solutions.
Key skills/competency
- Native American English Speaker
- Content Generation
- Data Annotation
- LLM Evaluation
- Localization QA
- Editorial Judgment
- Fact-Checking
- Attention to Detail
- AI Training
- Quality Assurance
Skills & topics
- AI Trainer
- AI Evaluator
- Content Generation
- Data Annotation
- LLM Evaluation
- Localization QA
- Remote
- Contractor
- American English
- Native Speaker
How to get hired
- Tailor your resume: Highlight experience in AI training, data annotation, LLM evaluation, and localization QA.
- Craft a compelling cover letter: Emphasize your native American English fluency and keen editorial judgment.
- Showcase attention to detail: Provide examples of identifying subtle errors and explaining corrections clearly.
- Demonstrate self-direction: Mention your ability to work reliably and communicate effectively across time zones.
- Highlight preferred experience: If applicable, detail your familiarity with MQM/LQA concepts and QA workflows.
Technical preparation
Practice writing clear, concise prompts.,Review AI response evaluation guidelines.,Familiarize with data annotation tools.,Study localization and QA principles.
Behavioral questions
Describe a time you found a subtle error.,How do you ensure accuracy in your work?,How do you manage flexible work hours?,Explain a complex topic clearly in writing.
Frequently asked questions
- What are the primary responsibilities of an AI Trainer and Evaluator at CNTXT AI?
- As an AI Trainer and Evaluator, you'll be responsible for tasks such as generating content (prompts, responses, voice samples), annotating data for AI training sets, evaluating AI-generated responses for accuracy and appropriateness, and performing localization QA to ensure cultural and linguistic consistency. This role requires a native American English speaker with excellent editorial judgment.
- What specific qualifications are essential for the AI Trainer and Evaluator role at CNTXT AI?
- The most critical qualification is being a native speaker of American English, born and raised in the United States. Additionally, you need excellent editorial judgment, extreme attention to detail, the ability to identify subtle errors and meaning drift, and the skill to rigorously fact-check content. Self-direction and clear communication are also key.
- Is this AI Trainer and Evaluator position remote, and what are the working hours?
- Yes, this is a fully remote, hourly contractor role. The hours are flexible and project-based, allowing you to work on your own schedule within project requirements and across different time zones.
- What does 'LLM evaluation' entail for an AI Trainer and Evaluator at CNTXT AI?
- LLM evaluation involves reviewing responses generated by large language models (LLMs). You'll assess their accuracy, the quality of their reasoning, how coherent they are, and whether they are culturally and linguistically appropriate for the target audience. This requires a sharp editorial eye and an understanding of nuanced language.
- How important is localization QA in the AI Trainer and Evaluator role?
- Localization QA is very important. It ensures that AI outputs are not only accurate in terms of terminology and tone but also culturally nuanced and locally relevant. This includes verifying locale-specific details like units, references, names, and dates to maintain consistency across different regions.
- Does CNTXT AI prefer candidates with specific QA or AI training experience for the AI Trainer and Evaluator role?
- While not strictly required, CNTXT AI prefers candidates with familiarity with MQM/LQA concepts (severity, category, root-cause thinking) for quality decisions, general QA workflows, and prior experience in AI data training, annotation, or evaluation. This experience can give you an edge.
Similar roles
Open positions we recommend based on this role.