Software Engineer, AI Research Annotation
Keystone Recruitment
Job Overview
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
About the Role: Software Engineer, AI Research Annotation
Join leading AI labs as a Software Engineer, AI Research Annotation, applying your software engineering expertise to significantly improve model reasoning. This global remote position involves reviewing real-world coding tasks and meticulously documenting problem-solving processes to support the development of advanced AI coding capabilities.
You will contribute to structured dataset creation, working with authentic software engineering scenarios from open-source repositories to capture realistic developer thought processes. This role is offered as an hourly independent contractor position, with flexible hours ranging from 15-25 per week, with the potential to scale up to 40 hours.
Detailed Responsibilities
- Review AI-generated code trajectories on SWE-bench style tasks.
- Reconstruct step-by-step reasoning, explaining solution approaches.
- Annotate decision points, debugging strategies, and technical choices.
- Utilize full codebase context (issues, tests, PRs) for accurate documentation.
- Ensure annotations reflect authentic software engineering practices.
Technical Requirements
- 2+ years hands-on software development experience.
- Strong proficiency in Python, JavaScript/TypeScript, or similar languages.
- Experience debugging complex codebases and resolving issues.
- Familiarity with Git workflows, pull requests, and issue tracking.
- Ability to articulate technical reasoning clearly in writing.
Project Structure
- Duration: 1-2 months with opportunities for extension.
- Start: Immediate onboarding.
- Onboarding: Resume review + 15-minute AI technical interview.
- Pace: Self-scheduled within weekly commitment minimum.
- Payment: Weekly via Stripe/Wise.
Ideal Background
- Computer Science/Software Engineering degree (Bachelor's or higher).
- Experience with open-source contributions or repository analysis.
- Understanding of software development lifecycles and testing.
- Comfort working with technical documentation and specifications.
Success Profile
Top performers will receive rate increases and priority consideration for future AI research projects, becoming part of a cohort of experienced engineers shaping the next-generation of coding AI.
Key skills/competency
- Software Development
- AI Research
- Code Annotation
- Debugging
- Python
- JavaScript/TypeScript
- Git Workflows
- Technical Documentation
- Problem-Solving
- Software Engineering Practices
How to Get Hired at Keystone Recruitment
- Research Keystone Recruitment's culture: Study their mission, values, recent news, and employee testimonials on LinkedIn and Glassdoor to understand their focus on AI research and contractor roles.
- Tailor your resume for AI Research Annotation: Highlight your 2+ years of software development experience, proficiency in Python or JavaScript/TypeScript, and any experience with debugging complex codebases and open-source contributions.
- Prepare for the AI technical interview: Brush up on coding challenges, debugging strategies, and articulate your technical reasoning clearly, as a 15-minute AI technical interview is part of the onboarding.
- Showcase your documentation skills: Emphasize your ability to explain complex technical concepts and document decision points, debugging, and technical choices, as this is central to the Software Engineer, AI Research Annotation role.
- Demonstrate commitment to remote work: Highlight your self-management and time organization skills for success in a flexible, global remote independent contractor role.
Frequently Asked Questions
Find answers to common questions about this job opportunity
Explore similar opportunities that match your background