4 days ago
AI Safety Evaluation Specialist
Keystone Recruitment
Hybrid
Contractor
$330,000
Hybrid
Job Overview
Job TitleAI Safety Evaluation Specialist
Job TypeContractor
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$330,000
LocationHybrid
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
Role Overview
We are hiring on behalf of one of our clients for experienced safety specialists to evaluate and strengthen advanced AI systems. This AI Safety Evaluation Specialist role focuses on identifying vulnerabilities, assessing systemic risks, and contributing to the development of more robust and reliable AI models. This is a project-based, remote contract opportunity suited for professionals with experience in AI safety, model evaluation, security testing, or related analytical fields.
Key Responsibilities
- Identify and document model limitations, vulnerabilities, and failure patterns
- Annotate outputs and classify risks using structured taxonomies and evaluation frameworks
- Develop reproducible testing cases and structured reports
- Conduct adversarial testing across language models and socio-technical risk scenarios
- Evaluate AI outputs against defined safety benchmarks and guidelines
- Produce clear, actionable documentation for technical and non-technical stakeholders
- Collaborate with project teams to improve model robustness and evaluation methodologies
Required Qualifications
- Experience in AI safety, security research, red teaming, model evaluation, or related technical domains
- Strong analytical and structured problem-solving skills
- Ability to document findings clearly and reproducibly
- Familiarity with risk assessment frameworks or benchmarking methodologies
- Strong written communication skills
Preferred Qualifications
- Experience with adversarial testing or model vulnerability research
- Background in cybersecurity, machine learning, or socio-technical risk analysis
- Experience working with evaluation datasets or safety benchmarking tools
- Ability to translate complex technical findings into clear, structured reports
Engagement Details
- Independent contractor engagement
- Fully remote with flexible scheduling
- Project-based work; extensions may occur based on project needs and performance
- Hourly compensation range: $85–$185 depending on expertise and scope
- Weekly payments via secure payment platforms
Applicants must be legally authorized to provide independent contractor services in their country of residence.
Key skills/competency
- AI Safety
- Model Evaluation
- Risk Assessment
- Adversarial Testing
- Vulnerability Analysis
- Machine Learning
- Cybersecurity
- Structured Problem-Solving
- Technical Documentation
- Language Models
How to Get Hired at Keystone Recruitment
- Research Keystone Recruitment's culture: Study their mission, values, recent news, and employee testimonials on LinkedIn and Glassdoor.
- Tailor your resume for AI Safety roles: Highlight experience in AI safety, model evaluation, and security testing.
- Showcase relevant project experience: Detail your contributions to identifying AI vulnerabilities or assessing risks.
- Prepare for technical safety evaluations: Be ready to discuss adversarial testing, risk assessment, and documentation practices.
- Demonstrate strong communication skills: Emphasize your ability to produce clear reports for diverse stakeholders.
Frequently Asked Questions
Find answers to common questions about this job opportunity
01What is the primary focus of the AI Safety Evaluation Specialist role at Keystone Recruitment's client?
02Is the AI Safety Evaluation Specialist position at Keystone Recruitment fully remote?
03What kind of experience is Keystone Recruitment seeking for this AI Safety Evaluation Specialist role?
04How are AI Safety Evaluation Specialists compensated by Keystone Recruitment for this project?
05What specific technical skills are crucial for this AI Safety Evaluation Specialist role with Keystone Recruitment?
06What types of AI systems will I be evaluating in this AI Safety Evaluation Specialist role at Keystone Recruitment?
07How important is documentation in the AI Safety Evaluation Specialist role for Keystone Recruitment?
08What is the typical duration for project engagements as an AI Safety Evaluation Specialist with Keystone Recruitment?
Explore similar opportunities that match your background