Data Labeler, LearnWith.AI (Remote) - $30,000/year USD
Crossover · Austin, TX
- Hybrid
- Full-time
- $30,000 / year
- Austin, TX
Job highlights
- Label student videos for AI training data.
- Evaluate and correct LLM pre-annotations.
- Ensure high accuracy and temporal precision.
- Work remotely in a full-time role.
- Contribute directly to AI model improvement.
About the role
About The Role
If precision matters more to you than pace, this position will suit you. The labels you create serve as training data for AI systems used daily by thousands of students. Accurate labeling makes the product smarter. Inconsistent labeling teaches the model incorrect patterns.
LearnWith.AI develops AI-driven learning experiences through learning science, data analytics, and subject matter expertise. This position converts raw video recordings of student sessions into high-fidelity, rubric-based labels the team can rely on. You will review recorded student sessions, pinpoint critical behavioral events, and apply defined rules to categorize what occurred and when. You will also evaluate LLM pre-annotations, correct errors, and record edge cases so engineers can refine the system.
This is not gig-based, scattered annotation work. It is a consistent workflow within one product area, with direct feedback, calibration against reference standards, and advancement tied to accuracy and reliability. If you value transparent expectations, quantifiable quality, and tasks that directly affect model outcomes, we would like to hear from you.
What You Will Be Doing
- Label student session videos by detecting, categorizing, and timestamping behavioral events according to a comprehensive rubric
- Evaluate and amend LLM pre-annotations by eliminating false positives, inserting overlooked events, and refining timestamps
- Document reasoning for ambiguous decisions, including rubric citations and the logic you applied
- Record edge cases and clarification requests for unclear scenarios and maintain an annotation log with session metadata
- Participate in calibration exercises, incorporate QA feedback, and implement rubric revisions to enhance accuracy progressively
What You Won’t Be Doing
- Develop AI models, conduct experiments, or perform research on student behavior patterns
- Create the annotation rubric or alter category definitions based on subjective interpretation
- Prioritize speed over accuracy, consistency, or timestamp precision
- Handle sporadic, disconnected tasks across unrelated fields without context or quality feedback
Data Labeler Key Responsibilities
This role ensures that student session videos are transformed into ≥95%-accurate, temporally precise labeled datasets that dependably indicate when model performance advances or declines.
Basic Requirements
- A minimum of 1 year of experience in data annotation, content moderation, QA evaluation, or comparable rubric-based review work
- Excellent English reading comprehension and the capacity to adhere to detailed written instructions without deviation from the guidelines
- Capacity to maintain concentration and precision throughout 4–6 hours of video-based work daily
- Ability to detect subtle on-screen and visual behavioral signals and categorize them uniformly across numerous sessions
- Solid written communication skills for describing edge cases, reasoning, and clarification inquiries
- Dependable internet connection suitable for video streaming
- Comfort reviewing, correcting, and enhancing AI/LLM-produced annotations
About LearnWith.AI
LearnWith.AI is an edtech startup that leverages AI and subject matter experts to cultivate a new way of learning. Our unique approach leverages 50+ years of learning science, cutting-edge data analytics and AI-powered coaching. In doing so, we can help students learn more, learn faster, and learn better - and have fun while doing it. We are a remote-first company that hires globally via Crossover.
There is so much to cover for this exciting role, and space here is limited. Hit the Apply button if you found this interesting and want to learn more. We look forward to meeting you!
Working with Us
This is a full-time (40 hours per week), long-term position. The position is immediately available and requires entering into an independent contractor agreement with Crossover as a Contractor of Record. The compensation level for this role is $15 USD/hour, which equates to $30,000 USD/year assuming 40 hours per week and 50 weeks per year. The payment period is weekly. Consult www.crossover.com/help-and-faqs for more details on this topic.
Key skills/competency
- Data Annotation
- AI Training Data
- LLM Evaluation
- Quality Assurance
- Video Analysis
- Rubric Application
- Attention to Detail
- English Comprehension
- Problem Solving
- EdTech
Skills & topics
- Data Labeler
- AI
- Machine Learning
- Data Annotation
- EdTech
- Quality Assurance
- Remote
- Video Analysis
- LLM
- Training Data
How to get hired
- Tailor your resume: Highlight data annotation, QA, or content moderation experience. Quantify achievements using numbers.
- Emphasize attention to detail: Showcase your ability to follow complex instructions precisely.
- Prepare for technical assessment: Demonstrate your understanding of annotation rubrics and quality control.
- Highlight remote work skills: Show self-discipline, a reliable internet connection, and strong communication.
- Understand the company: Research LearnWith.AI's mission in edtech and AI.
Technical preparation
Behavioral questions
Frequently asked questions
- What is the primary responsibility of a Data Labeler at LearnWith.AI?
- The primary responsibility of a Data Labeler at LearnWith.AI is to meticulously review and label student session videos, ensuring high accuracy and temporal precision for AI training data. This involves detecting, categorizing, and timestamping behavioral events according to a defined rubric, as well as evaluating and correcting LLM pre-annotations.
- What kind of experience is required for the Data Labeler role at LearnWith.AI?
- The Data Labeler role requires a minimum of 1 year of experience in data annotation, content moderation, QA evaluation, or comparable rubric-based review work. Strong English reading comprehension and the ability to adhere strictly to detailed instructions are also essential.
- Is the Data Labeler position at LearnWith.AI remote?
- Yes, the Data Labeler position at LearnWith.AI is a remote-first role. LearnWith.AI hires globally via Crossover, allowing candidates from various locations to apply and work from home.
- What is the compensation for the Data Labeler role at LearnWith.AI?
- The compensation for the Data Labeler role is $15 USD per hour, equating to $30,000 USD per year, assuming a 40-hour work week and 50 weeks per year. Payments are made weekly.
- What will a Data Labeler NOT be doing at LearnWith.AI?
- A Data Labeler at LearnWith.AI will not be developing AI models, conducting experiments, performing research on student behavior patterns, creating annotation rubrics, or prioritizing speed over accuracy. They will not handle sporadic, disconnected tasks across unrelated fields.
- How does LearnWith.AI ensure data quality for its AI models?
- LearnWith.AI ensures data quality through a rigorous labeling process that requires over 95% accuracy and temporal precision. This includes direct feedback, calibration against reference standards, ongoing QA evaluation, and progressive implementation of rubric revisions to enhance accuracy.
- What technical skills are important for a Data Labeler at LearnWith.AI?
- Key technical skills include comfort with reviewing and correcting AI/LLM-produced annotations, the ability to detect subtle visual signals, and maintaining precision with timestamps. A dependable internet connection for video streaming is also crucial.
- What kind of work arrangement is the Data Labeler position at LearnWith.AI?
- The Data Labeler position is a full-time, long-term role requiring a consistent workflow within one product area. It is a remote arrangement, allowing individuals to work from home.