LLM Data Quality Specialist

Role Overview: LLM Data Quality Specialist

A global AI-focused organization, Elevate Recruitment, is seeking an LLM Data Quality Specialist to support large language model training initiatives. This role focuses on structured data validation and lightweight tooling development, blending hands-on quality control with practical tool-building to improve dataset reliability, clarity, and evaluation standards across both STEM and humanities domains.

You will collaborate closely with internal experts and AI teams to ensure training data meets rigorous quality standards, while also developing simple, effective workflows to scale review processes.

Key Responsibilities

• Perform quality checks (QC) on generated datasets, including Q&A pairs, rubric-based datasets, evaluation samples, and agentic workflows.
• Review content across both STEM and humanities domains for logical consistency, accuracy, clarity, and completeness.
• Develop clear, actionable QC guidelines to facilitate independent review by domain experts.
• Design and build lightweight Python-based utilities (primarily in Google Colab) to streamline validation workflows.
• Translate quality requirements into structured review checklists, scripts, and semi-automated tools.
• Identify recurring data issues and recommend process improvements to enhance efficiency.
• Communicate findings clearly and professionally to cross-functional stakeholders.

Required Qualifications

• Strong analytical thinking and meticulous attention to detail.
• Demonstrated experience performing structured content validation or quality assurance.
• Proficiency in Python, particularly for lightweight scripting and automation tasks.
• Comfortable working in Google Colab or similar notebook environments.
• Ability to convert ambiguous QC requirements into practical workflows and effective tools.
• Excellent written communication skills for documentation and stakeholder interactions.
• Ability to commit to full-time work with required PST overlap.

Engagement Details

• This is an independent contractor assignment; it does not include medical benefits or paid leave.
• Requires a full-time commitment, with 4 hours of overlap with Pacific Standard Time (PST).
• The contract duration is 10 months, with potential for extension based on performance.
• This is a fully remote position.

Key skills/competency

• Data Quality
• LLM Training
• Python Scripting
• Google Colab
• Data Validation
• Quality Assurance
• Content Review
• Workflow Automation
• Process Improvement
• Analytical Thinking