3 days ago
AI Safety Evaluation Specialist
Keystone Recruitment
Hybrid
Contractor
$384,800
Hybrid
Job Overview
Job TitleAI Safety Evaluation Specialist
Job TypeContractor
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$384,800
LocationHybrid
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
AI Safety Evaluation Specialist at Keystone Recruitment
We are hiring on behalf of one of our clients for experienced safety specialists to evaluate and strengthen advanced AI systems. This role focuses on identifying vulnerabilities, assessing systemic risks, and contributing to the development of more robust and reliable AI models.
This is a project-based, remote contract opportunity suited for professionals with experience in AI safety, model evaluation, security testing, or related analytical fields.
Key Responsibilities
- Identify and document model limitations, vulnerabilities, and failure patterns
- Annotate outputs and classify risks using structured taxonomies and evaluation frameworks
- Develop reproducible testing cases and structured reports
- Conduct adversarial testing across language models and socio-technical risk scenarios
- Evaluate AI outputs against defined safety benchmarks and guidelines
- Produce clear, actionable documentation for technical and non-technical stakeholders
- Collaborate with project teams to improve model robustness and evaluation methodologies
Required Qualifications
- Experience in AI safety, security research, red teaming, model evaluation, or related technical domains
- Strong analytical and structured problem-solving skills
- Ability to document findings clearly and reproducibly
- Familiarity with risk assessment frameworks or benchmarking methodologies
- Strong written communication skills
Preferred Qualifications
- Experience with adversarial testing or model vulnerability research
- Background in cybersecurity, machine learning, or socio-technical risk analysis
- Experience working with evaluation datasets or safety benchmarking tools
- Ability to translate complex technical findings into clear, structured reports
Engagement Details
- Independent contractor engagement
- Fully remote with flexible scheduling
- Project-based work; extensions may occur based on project needs and performance
- Hourly compensation range: $85–$185 depending on expertise and scope
- Weekly payments via secure payment platforms
Applicants must be legally authorized to provide independent contractor services in their country of residence.
Key skills/competency
- AI Safety
- Model Evaluation
- Vulnerability Identification
- Adversarial Testing
- Risk Assessment
- Security Testing
- Machine Learning
- Technical Documentation
- Analytical Thinking
- Socio-technical Risk Analysis
How to Get Hired at Keystone Recruitment
- Research Keystone Recruitment's culture: Study their mission, values, recent news, and employee testimonials on LinkedIn and Glassdoor to understand their client focus.
- Tailor your resume: Customize your application to highlight specific experience in AI safety, model evaluation, adversarial testing, and security research relevant to the AI Safety Evaluation Specialist role.
- Showcase relevant projects: Detail your contributions to AI risk assessment, vulnerability identification, or robust AI development in your portfolio or resume.
- Prepare for technical deep dives: Be ready to discuss specific methodologies for identifying model limitations, classifying risks, and developing reproducible testing cases.
- Demonstrate strong communication: Practice articulating complex technical findings clearly and concisely, both for technical and non-technical audiences.
Frequently Asked Questions
Find answers to common questions about this job opportunity
01What does the AI Safety Evaluation Specialist role at Keystone Recruitment primarily involve?
02Is this AI Safety Evaluation Specialist position a full-time role or a contract?
03What kind of experience is Keystone Recruitment looking for in an AI Safety Evaluation Specialist?
04How will I be compensated for the AI Safety Evaluation Specialist role with Keystone Recruitment?
05What are the remote work expectations for the AI Safety Evaluation Specialist at Keystone Recruitment?
Explore similar opportunities that match your background