Data Scientist AI Training and Evaluation

Data Scientist – AI Training & Evaluation

Alignerr is seeking talented Data Scientists based in the New York metro area to contribute to the training, evaluation, and enhancement of advanced AI systems. This role involves applying your expertise in statistics, machine learning, and data analysis to meticulously assess AI model outputs, develop robust evaluation frameworks, and ensure the accuracy and reliability of AI systems.

What You'll Do

• Evaluate AI model outputs for accuracy, reasoning quality, and statistical soundness.
• Design and implement data-driven evaluation criteria and scoring rubrics.
• Analyze patterns in AI-generated responses to identify systematic errors or biases.
• Create high-quality training data, including prompts, solutions, and annotations.
• Provide structured, detailed feedback to improve model performance.
• Review AI-generated code, visualizations, and statistical analyses for correctness.
• Work independently and asynchronously on your own schedule.

Who You Are

• Based in the United States.
• Degree in Data Science, Statistics, Computer Science, Mathematics, or a related quantitative field (MS or PhD preferred).
• Strong foundation in statistics, probability, and machine learning concepts.
• Proficiency in Python, R, SQL, or similar data analysis tools.
• Experience with data wrangling, exploratory data analysis, and model evaluation.
• Excellent analytical thinking and attention to detail.
• Clear written communication skills, with the ability to simplify complex technical concepts.
• Self-motivated and comfortable working independently.

Nice to Have

• Experience with deep learning frameworks like PyTorch or TensorFlow.
• Familiarity with NLP, LLMs, or AI evaluation workflows.
• Published research or industry experience in applied machine learning.
• Background in A/B testing, causal inference, or experimental design.

Why Join Us

• Work on cutting-edge AI projects that are shaping the future of technology.
• Collaborate with leading research labs and AI teams globally.
• Enjoy freelance perks such as full autonomy, flexible scheduling, and a remote-first culture.
• Gain deep exposure to the training and evaluation processes of state-of-the-art LLMs.
• Potential for ongoing work and long-term contract extensions.
• Become part of a growing community of expert contributors dedicated to making AI smarter.

Application Process

• Submit your resume.
• Complete a short screening.
• Project matching and onboarding.

Our team reviews applications daily. Please complete your application steps to be considered for this opportunity.

Key skills/competency

• Data Scientist
• AI Training
• AI Evaluation
• Machine Learning
• Statistics
• Data Analysis
• Python
• SQL
• Model Evaluation
• LLMs