Machine Learning Evaluation Specialist
Alignerr · Bengaluru, Karnataka, India
This listing has closed — view similar roles below.
- Hybrid
- Contract
- $60,000 / year
- Bengaluru, Karnataka, India
Job highlights
- Design AI evaluation problems using specialized research.
- Test state-of-the-art AI systems rigorously.
- Influence AI model development directly.
- Work remotely with global researchers.
- Requires advanced ML and domain expertise.
About the role
Machine Learning Evaluation Specialist (AI Training)
About The Role
What if your years of research expertise could directly shape the trajectory of AI development? We're looking for elite machine learning specialists to design the kinds of problems that push state-of-the-art AI systems to their absolute limits — and help the field understand precisely where they break down.
This is a rare, high-impact opportunity to put your hard-earned domain knowledge to work at the frontier of AI evaluation and safety. Your contributions won't collect dust in a journal — they'll actively influence how the next generation of AI models is built, tested, and improved.
Organization: Alignerr
Type: Hourly Contract
Location: Remote
Commitment: 10–40 hours/week
What You'll Do
- Design complex, original machine learning problems rooted in your specialized domain of expertise
- Craft evaluation tasks that demand advanced knowledge far beyond standard ML pipelines
- Draw from your own research to create challenges that would stump even the most capable LLMs
- Define precise problem statements, gold-standard solutions, and robust evaluation criteria
- Assess AI-generated ML solutions for correctness, creativity, and methodological rigor
- Document expected failure modes, difficulty levels, and the domain knowledge required to solve each problem
- Collaborate asynchronously with a global network of top researchers and engineers
Who You Are
- You hold a graduate degree (MS or PhD) in a scientific or technical field that intersects with machine learning
- You have strong working knowledge of ML fundamentals — model selection, feature engineering, evaluation metrics, and pipeline design
- You're deeply embedded in active research problems in your field and know where the hard edges are
- You can identify precisely where general ML knowledge falls short and specialized domain expertise becomes essential
- You have experience publishing or conducting original research (highly valued)
- You communicate complex ideas with clarity and precision in writing
- You're self-directed, intellectually curious, and thrive on genuinely challenging work
Example Domains (Not Exhaustive)
- Computational biology, genomics, or bioinformatics
- Climate science and environmental modeling
- Medical imaging and healthcare ML
- Materials science and computational chemistry
- Astrophysics and signal processing
- NLP for low-resource or specialized corpora
- Robotics, control theory, or reinforcement learning
- Financial modeling and quantitative analysis
Why Join Us
- Work at the frontier — contribute directly to AI evaluation and safety research alongside top research labs
- Make your expertise count — put your specialized knowledge to use in a way that creates real, measurable impact on how AI develops
- Full autonomy — set your own schedule and work entirely remotely on your terms
- Intellectually stimulating work — no routine tasks; every challenge requires genuine deep thinking
- Global collaboration — connect and work asynchronously with a world-class community of researchers and engineers
- Ongoing opportunity — strong performers gain access to extended contracts and deeper research involvement
- Build your profile — become a recognized contributor to cutting-edge AI development
Key skills/competency
- Machine Learning
- AI Evaluation
- Research Expertise
- Domain Knowledge
- Problem Design
- LLM Challenges
- ML Fundamentals
- Original Research
- Critical Thinking
- Scientific Writing
Skills & topics
- Machine Learning
- AI
- Evaluation
- Specialist
- Research
- Remote
- Contract
- AI Training
- LLM
- Domain Expertise
How to get hired
- Tailor your resume: Highlight your graduate degree, ML fundamentals, and research publications.
- Showcase domain expertise: Emphasize specific scientific or technical fields and their intersection with ML.
- Demonstrate research impact: Detail original research, problem design, and evaluation contributions.
- Articulate complex ideas: Use your cover letter to clearly explain your unique qualifications.
- Highlight self-direction: Mention experience working independently and asynchronously.
Technical preparation
Master ML fundamentals: selection, feature engineering, metrics.,Deepen specialized domain knowledge for complex problems.,Practice designing novel ML challenges.,Refine evaluation criteria and assessment methods.
Behavioral questions
Describe a complex research problem you solved.,How do you identify AI system limitations?,Explain a technical concept to a non-expert.,How do you manage autonomous, challenging work?
Frequently asked questions
- What is the primary goal of a Machine Learning Evaluation Specialist at Alignerr?
- The primary goal is to design challenging, original machine learning problems that push the limits of AI systems, helping to identify their weaknesses and improve their development. This role is crucial for advancing AI evaluation and safety.
- What academic qualifications are essential for this Machine Learning Evaluation Specialist role?
- A graduate degree (MS or PhD) in a scientific or technical field that intersects with machine learning is essential. Experience in publishing or conducting original research is highly valued.
- What kind of machine learning knowledge is required for the Machine Learning Evaluation Specialist position?
- You need strong working knowledge of ML fundamentals, including model selection, feature engineering, evaluation metrics, and pipeline design. Deep understanding of your specialized research domain is also critical.
- Can I work remotely as a Machine Learning Evaluation Specialist for Alignerr?
- Yes, this is a fully remote position, offering the flexibility to set your own schedule and work from anywhere, with a commitment of 10-40 hours per week.
- What distinguishes Alignerr's Machine Learning Evaluation Specialist role from other ML positions?
- This role focuses on the frontier of AI evaluation and safety, directly influencing AI development through high-impact research-based problem design, rather than standard ML pipeline work.
- How does Alignerr support professional growth for its Machine Learning Evaluation Specialists?
- Alignerr offers ongoing opportunities for strong performers, including extended contracts and deeper research involvement, allowing you to build your profile as a recognized contributor in AI development.
- What are some example domains where a Machine Learning Evaluation Specialist might apply their expertise?
- Example domains include computational biology, genomics, bioinformatics, climate science, environmental modeling, medical imaging, healthcare ML, materials science, computational chemistry, astrophysics, signal processing, NLP for specialized corpora, robotics, control theory, reinforcement learning, and financial modeling.