AI Safety Evaluation Specialist

AI Safety Evaluation Specialist

Keystone Recruitment is seeking experienced safety specialists on behalf of a client to evaluate and strengthen advanced AI systems. This fully remote, project-based contract role focuses on identifying vulnerabilities, assessing systemic risks, and contributing to the development of more robust and reliable AI models. This opportunity is suited for professionals with a strong background in AI safety, model evaluation, security testing, or related analytical fields, offering competitive hourly compensation.

Key Responsibilities

• Identify and document model limitations, vulnerabilities, and failure patterns in AI systems.
• Annotate outputs and classify risks using structured taxonomies and established evaluation frameworks.
• Develop reproducible testing cases and structured reports to articulate findings clearly.
• Conduct adversarial testing across various language models and socio-technical risk scenarios.
• Evaluate AI outputs meticulously against defined safety benchmarks and comprehensive guidelines.
• Produce clear, actionable documentation suitable for both technical and non-technical stakeholders.
• Collaborate effectively with project teams to continuously improve model robustness and refine evaluation methodologies.

Required Qualifications

• Demonstrated experience in AI safety, security research, red teaming, or model evaluation.
• Strong analytical capabilities and a structured approach to problem-solving.
• Ability to document findings clearly, concisely, and reproducibly.
• Familiarity with risk assessment frameworks or established benchmarking methodologies.
• Excellent written communication skills for diverse audiences.

Preferred Qualifications

• Prior experience with adversarial testing or dedicated model vulnerability research.
• A background in cybersecurity, machine learning, or socio-technical risk analysis.
• Experience working with evaluation datasets or specialized safety benchmarking tools.
• Proven ability to translate complex technical findings into clear, structured reports.

Engagement Details

• This is an independent contractor engagement.
• Fully remote position with flexible scheduling options globally.
• Project-based work, with potential for extensions based on performance and project requirements.
• Hourly compensation ranges from $85 to $185, depending on expertise and project scope.
• Weekly payments are processed via secure payment platforms.

Applicants must be legally authorized to provide independent contractor services in their country of residence.

Key skills/competency

• AI Safety
• Model Evaluation
• Risk Assessment
• Vulnerability Identification
• Adversarial Testing
• Security Research
• Machine Learning
• Language Models
• Structured Problem-Solving
• Technical Documentation