Data Scientist – AI Training & Evaluation
Alignerr · New York, NY
- Hybrid
- Contract
- $75,000 / year
- New York, NY
Job highlights
- Evaluate AI model outputs for accuracy.
- Design data-driven evaluation criteria.
- Analyze AI responses for errors.
- Create high-quality training data.
- Provide feedback to improve models.
About the role
Data Scientist – AI Training & Evaluation
Alignerr is seeking data scientists to train, evaluate, and improve next-generation AI systems. Your expertise in statistics, machine learning, and data analysis will directly shape how AI reasons, interprets data, and delivers results, making a real and measurable impact on the future of AI.
About the Role
Organization: Alignerr
Type: Hourly Contract
Location: Remote (US-based)
Commitment: 10–40 hours/week
What You'll Do
- Evaluate AI model outputs for accuracy, reasoning quality, and statistical soundness.
- Design and apply data-driven evaluation criteria and scoring rubrics.
- Analyze patterns in AI-generated responses to identify systematic errors or biases.
- Create high-quality training data including prompts, solutions, and annotations in your areas of expertise.
- Provide structured, detailed feedback to improve model performance across data science and analytical tasks.
- Review AI-generated code, visualizations, and statistical analyses for correctness.
- Work independently and asynchronously on your own schedule.
Who You Are
- Based in the United States.
- Degree in Data Science, Statistics, Computer Science, Mathematics, or a related quantitative field (MS or PhD preferred).
- Strong foundation in statistics, probability, and machine learning concepts.
- Proficient in Python, R, SQL, or similar data analysis tools.
- Experienced with data wrangling, exploratory data analysis, and model evaluation.
- Excellent analytical thinking and sharp attention to detail.
- Clear written communicator — able to explain complex technical concepts simply.
- Self-motivated and comfortable working independently.
Nice to Have
- Experience with deep learning frameworks (PyTorch, TensorFlow).
- Familiarity with NLP, LLMs, or AI evaluation workflows.
- Published research or industry experience in applied machine learning.
- Background in A/B testing, causal inference, or experimental design.
Why Join Us
- Work on cutting-edge AI projects shaping the future of technology.
- Collaborate with top research labs and AI teams globally.
- Fully remote and flexible — work on your own schedule, your own way.
- Gain deep, hands-on exposure to how state-of-the-art LLMs are trained and evaluated.
- Be part of a growing community of expert contributors making AI smarter and more reliable.
- Potential for ongoing work and long-term contract extension.
Key skills/competency
- Data Science
- AI Training
- Model Evaluation
- Machine Learning
- Statistics
- Python
- R
- SQL
- Data Analysis
- AI Systems
Skills & topics
- Data Scientist
- AI
- Machine Learning
- Statistics
- Python
- R
- SQL
- Model Evaluation
- Data Analysis
- Remote
How to get hired
- Tailor your resume: Highlight your AI, machine learning, and data science skills.
- Showcase your experience: Detail your proficiency in Python, R, SQL, and model evaluation.
- Emphasize remote work skills: Demonstrate self-motivation and independent work capabilities.
- Prepare for technical questions: Review statistics, probability, and ML concepts thoroughly.
- Highlight your quantitative background: Ensure your degree and experience align with the role.
Technical preparation
Review statistical and machine learning principles.,Practice Python, R, and SQL coding.,Prepare for data wrangling exercises.,Familiarize with model evaluation techniques.
Behavioral questions
How do you ensure accuracy in AI model evaluation?,Describe a time you identified bias in data.,How do you provide constructive feedback?,How do you manage your work independently?
Frequently asked questions
- What is the work arrangement for the Data Scientist AI Training and Evaluation role at Alignerr?
- This Data Scientist AI Training and Evaluation position at Alignerr is fully remote, allowing you to work from anywhere within the United States on your own schedule.
- What are the primary responsibilities of a Data Scientist AI Training and Evaluation at Alignerr?
- As a Data Scientist AI Training and Evaluation at Alignerr, you will focus on evaluating AI model outputs, designing evaluation criteria, analyzing AI responses for errors, creating training data, and providing feedback to enhance model performance.
- What qualifications are essential for the Data Scientist AI Training and Evaluation role at Alignerr?
- To be considered for the Data Scientist AI Training and Evaluation role at Alignerr, you need a degree in a quantitative field (MS/PhD preferred), a strong foundation in statistics, machine learning, and proficiency in tools like Python, R, or SQL.
- Does Alignerr offer long-term opportunities for Data Scientists in AI Training and Evaluation?
- Yes, Alignerr mentions a potential for ongoing work and long-term contract extensions for successful Data Scientists in AI Training and Evaluation who contribute effectively.
- What kind of impact can I make as a Data Scientist AI Training and Evaluation at Alignerr?
- As a Data Scientist AI Training and Evaluation at Alignerr, you will directly influence how AI systems reason and deliver results, contributing to the advancement and reliability of next-generation AI.
- How flexible is the Data Scientist AI Training and Evaluation role at Alignerr?
- The Data Scientist AI Training and Evaluation role at Alignerr offers significant flexibility, with commitment ranging from 10-40 hours per week, allowing you to work independently and asynchronously on your own schedule.