4 days ago
AI Safety Evaluation Specialist
Keystone Recruitment
Remote
Contractor
$280,000
Remote
Job Overview
Job TitleAI Safety Evaluation Specialist
Job TypeContractor
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$280,000
LocationRemote
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
AI Safety Evaluation Specialist
Keystone Recruitment is seeking experienced safety specialists on behalf of one of our clients to evaluate and strengthen advanced AI systems. This project-based, remote contract opportunity focuses on identifying vulnerabilities, assessing systemic risks, and contributing to the development of more robust and reliable AI models. It is ideal for professionals with a strong background in AI safety, model evaluation, security testing, or related analytical fields.
Key Responsibilities
- Identify and document model limitations, vulnerabilities, and failure patterns
- Annotate outputs and classify risks using structured taxonomies and evaluation frameworks
- Develop reproducible testing cases and structured reports
- Conduct adversarial testing across language models and socio-technical risk scenarios
- Evaluate AI outputs against defined safety benchmarks and guidelines
- Produce clear, actionable documentation for technical and non-technical stakeholders
- Collaborate with project teams to improve model robustness and evaluation methodologies
Required Qualifications
- Experience in AI safety, security research, red teaming, model evaluation, or related technical domains
- Strong analytical and structured problem-solving skills
- Ability to document findings clearly and reproducibly
- Familiarity with risk assessment frameworks or benchmarking methodologies
- Strong written communication skills
Preferred Qualifications
- Experience with adversarial testing or model vulnerability research
- Background in cybersecurity, machine learning, or socio-technical risk analysis
- Experience working with evaluation datasets or safety benchmarking tools
- Ability to translate complex technical findings into clear, structured reports
Engagement Details
- Independent contractor engagement
- Fully remote with flexible scheduling
- Project-based work; extensions may occur based on project needs and performance
- Hourly compensation range: $85–$185 depending on expertise and scope
- Weekly payments via secure payment platforms
Applicants must be legally authorized to provide independent contractor services in their country of residence.
Key skills/competency
- AI Safety
- Model Evaluation
- Vulnerability Assessment
- Risk Analysis
- Adversarial Testing
- Security Research
- Language Models
- Structured Problem-Solving
- Documentation
- Cybersecurity
How to Get Hired at Keystone Recruitment
- Research Keystone Recruitment's client focus: Understand the types of advanced AI clients Keystone Recruitment partners with and their specific safety needs.
- Tailor your resume for AI safety: Highlight experience in AI safety, security research, and model evaluation using keywords like 'adversarial testing' and 'risk assessment'.
- Showcase analytical problem-solving: Prepare to discuss how you've identified vulnerabilities and developed structured solutions in past roles.
- Demonstrate strong communication: Be ready to provide examples of clear, actionable documentation for both technical and non-technical stakeholders.
- Highlight remote collaboration skills: Emphasize your ability to work effectively and independently in a global, flexible, project-based environment.
Frequently Asked Questions
Find answers to common questions about this job opportunity
01What kind of AI systems will I be evaluating as an AI Safety Evaluation Specialist at Keystone Recruitment's client?
02What does a typical project involve for an AI Safety Evaluation Specialist at Keystone Recruitment?
03Is previous experience with adversarial testing mandatory for the AI Safety Evaluation Specialist role?
04How flexible is the remote work schedule for the AI Safety Evaluation Specialist position?
05What payment structure can I expect as an AI Safety Evaluation Specialist with Keystone Recruitment?
06What types of documentation will I be creating in this AI Safety Evaluation Specialist role?
Explore similar opportunities that match your background