7 days ago
LLM Data Quality Specialist
North Star Staffing
Hybrid
Contractor
$180,000
Hybrid
Job Overview
Job TitleLLM Data Quality Specialist
Job TypeContractor
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$180,000
LocationHybrid
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
LLM Data Quality Specialist at North Star Staffing
A global AI-focused organization is seeking an LLM Data Quality Specialist to support large language model training initiatives through structured data validation and lightweight tooling development. This role blends hands-on quality control with practical tool-building to improve dataset reliability, clarity, and evaluation standards across STEM and humanities domains.
You will work closely with internal experts and AI teams to ensure training data meets rigorous quality standards while building simple, effective workflows to scale review processes.
Key Responsibilities
- Perform quality checks (QC) on generated datasets including Q&A pairs, rubric-based datasets, evaluation samples, and agentic workflows.
- Review content across both STEM and humanities domains for logical consistency, accuracy, clarity, and completeness.
- Develop clear, actionable QC guidelines to help domain experts independently review their samples.
- Design and build lightweight Python-based utilities (primarily in Google Colab) to streamline validation workflows.
- Translate quality requirements into structured review checklists, scripts, and semi-automated tools.
- Identify recurring data issues and recommend process improvements.
- Communicate findings clearly and professionally to cross-functional stakeholders.
Required Qualifications
- Strong analytical thinking and attention to detail.
- Experience performing structured content validation or quality assurance.
- Proficiency in Python (especially for lightweight scripting and automation).
- Comfortable working in Google Colab or similar notebook environments.
- Ability to convert ambiguous QC requirements into practical workflows and tools.
- Excellent written communication skills.
- Ability to work full-time with required PST overlap.
Engagement Details
- Contractor assignment (no medical/paid leave).
- Full-time commitment with 4 hours of overlap with PST.
- Duration: 10 months.
- Fully remote.
- Potential for contract extension based on performance.
Evaluation Process
- Job interest form.
- Interview.
- Selection and onboarding.
Key skills/competency
- LLM Data Quality
- Data Validation
- Python Scripting
- Google Colab
- Quality Assurance
- AI Training Data
- Content Review
- Workflow Automation
- Analytical Thinking
- Communication Skills
How to Get Hired at North Star Staffing
- Research North Star Staffing's culture: Study their mission, values, recent news, and employee testimonials on LinkedIn and Glassdoor.
- Tailor your resume meticulously: Highlight experience in data quality, Python scripting, and LLM-related projects to align with the LLM Data Quality Specialist role.
- Master technical proficiencies: Showcase your expertise in Python for scripting, Google Colab, and data validation techniques relevant to large language models.
- Prepare for structured interviews: Practice communicating your analytical thinking, problem-solving approaches, and experience in building lightweight tools effectively.
- Emphasize remote work readiness: Articulate your ability to collaborate effectively remotely, especially with Pacific Standard Time overlap requirements.
Frequently Asked Questions
Find answers to common questions about this job opportunity
01What does the LLM Data Quality Specialist role at North Star Staffing involve?
02What specific technical skills are crucial for this LLM Data Quality position?
03Is the LLM Data Quality Specialist position at North Star Staffing fully remote?
04What is the typical duration of the LLM Data Quality contract with North Star Staffing?
05How will my performance be evaluated as an LLM Data Quality Specialist contractor?
06What kind of datasets will I be working with as an LLM Data Quality Specialist?
Explore similar opportunities that match your background