3 days ago
AI Safety Evaluation Specialist
Keystone Recruitment
Remote
Contractor
$300,000
Remote
Job Overview
Job TitleAI Safety Evaluation Specialist
Job TypeContractor
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$300,000
LocationRemote
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
AI Safety Evaluation Specialist (Remote)
We are hiring on behalf of one of our clients for experienced safety specialists to evaluate and strengthen advanced AI systems. This AI Safety Evaluation Specialist role focuses on identifying vulnerabilities, assessing systemic risks, and contributing to the development of more robust and reliable AI models.
This is a project-based, remote contract opportunity suited for professionals with experience in AI safety, model evaluation, security testing, or related analytical fields.
Key Responsibilities
- Identify and document model limitations, vulnerabilities, and failure patterns
- Annotate outputs and classify risks using structured taxonomies and evaluation frameworks
- Develop reproducible testing cases and structured reports
- Conduct adversarial testing across language models and socio-technical risk scenarios
- Evaluate AI outputs against defined safety benchmarks and guidelines
- Produce clear, actionable documentation for technical and non-technical stakeholders
- Collaborate with project teams to improve model robustness and evaluation methodologies
Required Qualifications
- Experience in AI safety, security research, red teaming, model evaluation, or related technical domains
- Strong analytical and structured problem-solving skills
- Ability to document findings clearly and reproducibly
- Familiarity with risk assessment frameworks or benchmarking methodologies
- Strong written communication skills
Preferred Qualifications
- Experience with adversarial testing or model vulnerability research
- Background in cybersecurity, machine learning, or socio-technical risk analysis
- Experience working with evaluation datasets or safety benchmarking tools
- Ability to translate complex technical findings into clear, structured reports
Engagement Details
- Independent contractor engagement
- Fully remote with flexible scheduling
- Project-based work; extensions may occur based on project needs and performance
- Hourly compensation range: $85–$185 depending on expertise and scope
- Weekly payments via secure payment platforms
Applicants must be legally authorized to provide independent contractor services in their country of residence.
Key skills/competency
- AI Safety
- Model Evaluation
- Risk Assessment
- Vulnerability Identification
- Adversarial Testing
- Structured Problem Solving
- Technical Documentation
- Machine Learning
- Cybersecurity
- Language Models
How to Get Hired at Keystone Recruitment
- Research Keystone Recruitment's culture: Study their mission, values, recent news, and employee testimonials on LinkedIn and Glassdoor to understand their client-focused approach in AI safety.
- Tailor your resume: Customize your resume to highlight experience in AI safety, security testing, model evaluation, and risk assessment, aligning with the AI Safety Evaluation Specialist requirements.
- Showcase relevant projects: Detail specific projects demonstrating your ability to identify AI vulnerabilities, conduct adversarial testing, and produce structured reports for technical and non-technical audiences.
- Prepare for technical questions: Anticipate in-depth discussions on AI safety frameworks, model limitations, security research methodologies, and your approach to structured problem-solving in evaluations.
- Emphasize remote work readiness: Highlight your experience and proficiency in independent contractor work, flexible scheduling, and effective remote collaboration, crucial for a global remote role.
Frequently Asked Questions
Find answers to common questions about this job opportunity
01What does the AI Safety Evaluation Specialist role at Keystone Recruitment entail?
02What kind of AI models will I be evaluating in this specialist position?
03Is prior experience in AI safety crucial for this Keystone Recruitment role?
04How does Keystone Recruitment handle compensation for this remote contract?
05What are the key technical skills needed for an AI Safety Evaluation Specialist at Keystone Recruitment?
06How long are the project engagements typically for an AI Safety Evaluation Specialist?
07What does 'adversarial testing' mean in the context of this AI Safety Evaluation Specialist role?
08What kind of documentation is expected from the AI Safety Evaluation Specialist?
09What sets a successful AI Safety Evaluation Specialist apart at Keystone Recruitment?
10Can I work from any country as an AI Safety Evaluation Specialist for Keystone Recruitment?
Explore similar opportunities that match your background