Senior Software Engineer
Referral By NK · United States
- Hybrid
- Contract
- $156,000 / year
- United States
Tailored resume — keyword-matched to this role.
Hiring manager — we find who's hiring.
Intro email — drafted to reach them directly.
Job highlights
- Evaluate AI-generated code for advanced models.
- Leverage 3+ years of software engineering experience.
- Utilize expertise in JavaScript, Go, or Python.
- Collaborate remotely with a dedicated project team.
- Contribute to the future of AI-assisted development.
About the role
Senior Software Engineer AI Evaluation
Join us in helping advance the next generation of AI models! We are looking for experienced Software Engineers to contribute to a cutting-edge AI evaluation project. This project plays a key role in improving how frontier language models understand, reason about, and generate production-quality software. You'll evaluate AI-generated solutions through the lens of an experienced software engineer, helping shape the future of AI-assisted software development.
Project Timeline
Project Duration: Approximately 1 week
Expected Timeline: July 8–14
Expected commitment: at least 10 hours
Candidates must be located in US/ Canada due to the need for real-time collaboration with the project team.
Background & Qualifications
- YOE: 3+ years of professional software engineering experience (or equivalent open-source experience)
Technical Expertise
- JavaScript / TypeScript: Node.js, React, TypeScript, modern web application development
- Go: Backend services, distributed systems, REST/gRPC APIs, cloud-native development (Kubernetes experience is a plus)
- Python: Backend development, open-source libraries, DevOps/infrastructure automation, testing frameworks
Software Engineering
- Experience working with large production codebases
- Ability to quickly understand unfamiliar repositories and software architectures
- Experience reviewing code for readability, maintainability, and software design
- Strong engineering judgment and a solid understanding of software engineering best practices
GitHub & Open Source
- Strong understanding of Git and GitHub workflows (e.g., branching, Issues, Pull Requests, and Code Reviews)
- Experience navigating open-source repositories and collaborating through GitHub
- Familiarity with open-source development workflows is highly preferred
- Familiarity with SWE-Bench or SWE-Bench Pro is a plus
Communication
- Strong English communication skills
Responsibilities
- Become familiar with assigned open-source codebases and their engineering practices
- Review code changes, technical discussions, and related software artifacts
- Evaluate AI-generated code for correctness, quality, maintainability, and engineering best practices
- Provide structured evaluations with clear technical reasoning
- Collaborate with fellow technical reviewers to ensure high-quality and consistent evaluations
Note: This is not a software development role. Rather than writing code, you'll leverage your software engineering expertise to evaluate AI-generated solutions and help improve the capabilities of state-of-the-art coding models.
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Contract and Payment Terms
- You will be engaged as an independent contractor.
- This is a fully remote role that can be completed on your own schedule.
- Projects can be extended, shortened, or concluded early depending on needs and performance.
- Your work at Mercor will not involve access to confidential or proprietary information from any employer, client, or institution.
- Payments are weekly on Stripe or Wise based on services rendered.
- Please note: We are unable to support H1-B or STEM OPT candidates at this time.
About Mercor
Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will work on projects that focus on training and enhancing AI systems. You will be paid competitively, collaborate with leading researchers, and help shape the next generation of AI systems in your area of expertise.
Key skills/competency
- Senior Software Engineer
- AI Evaluation
- JavaScript
- TypeScript
- Node.js
- React
- Go
- Python
- Code Review
- Open Source
Skills & topics
- Senior Software Engineer
- AI Evaluation
- JavaScript
- TypeScript
- Node.js
- React
- Go
- Python
- Code Review
- Open Source
- Remote
- Contract
- US
- Canada
How to get hired
- Tailor your resume: Highlight your 3+ years of software engineering experience, focusing on JavaScript/TypeScript, Go, or Python, and your experience with open-source projects and code reviews.
- Showcase technical skills: Emphasize your proficiency in Git, GitHub workflows, and any experience with SWE-Bench or similar AI evaluation tools.
- Address collaboration: Explicitly mention your strong English communication skills and ability to collaborate effectively in a remote, real-time setting.
- Prepare for evaluation: Be ready to discuss your approach to reviewing code for quality, maintainability, and adherence to best practices, as this is central to the role.
- Highlight remote readiness: Confirm your location in the US/Canada and your comfort with an independent contractor, remote, hourly engagement.
Technical preparation
Master Git and GitHub workflows.,Review common JavaScript, Go, Python patterns.,Practice evaluating code for quality.,Familiarize with SWE-Bench if possible.
Behavioral questions
Describe a complex codebase you learned quickly.,How do you provide constructive code feedback?,How do you ensure consistency in evaluations?,How do you handle real-time remote collaboration?
Frequently asked questions
- What is the expected duration for the Senior Software Engineer AI Evaluation role at Mercor?
- The Senior Software Engineer AI Evaluation project is an hourly contract role with an approximate duration of 1 week, with an expected timeline of July 8-14. While the initial project is short, there is potential for extensions based on needs and performance.
- What technical skills are most important for this Senior Software Engineer AI Evaluation position?
- Mercor is looking for strong experience in JavaScript/TypeScript (Node.js, React), Go (backend services, distributed systems), or Python (backend development, automation). Expertise in Git, GitHub workflows, code review, and understanding large codebases is also crucial. Familiarity with SWE-Bench is a plus.
- Is this a hands-on coding role for a Senior Software Engineer?
- No, this Senior Software Engineer role is not for writing code. Instead, you will use your software engineering expertise to evaluate AI-generated code and solutions, providing feedback to improve AI models.
- What are the location requirements for the Senior Software Engineer AI Evaluation role?
- Candidates for this role must be located in the US or Canada to facilitate real-time collaboration with the project team. The role is fully remote.
- How are Senior Software Engineers paid for this project?
- You will be engaged as an independent contractor and paid an hourly rate between $50-$75. Payments are processed weekly via Stripe or Wise based on services rendered.
- What is Mercor's role in the AI industry?
- Mercor partners with leading AI labs and enterprises to train frontier AI models using human expertise. They focus on enhancing AI systems through projects that leverage specialized knowledge, like this Senior Software Engineer AI Evaluation role.
- Can candidates on H1-B or STEM OPT apply for this role?
- Unfortunately, Mercor is unable to support H1-B or STEM OPT candidates for this particular project. Please check Mercor's career page for other opportunities that may be suitable.