Software Engineer, Code QA

Software Engineer, Code QA at Taskify AI

One of Taskify AI's clients is actively seeking experienced Software Engineers to support high-impact research collaborations with leading AI labs. As a Software Engineer, Code QA, you will play a crucial role in building evaluation datasets designed to assess AI reasoning, explanation quality, and technical judgment specifically in coding-related interactions. This is a unique opportunity for engineers who enjoy critical code analysis, clearly articulating technical reasoning, and actively shaping how AI systems comprehend software development workflows.

About the Project

This project centers on creating code-based question-and-answer datasets that focus on evaluating natural-language reasoning and the quality of technical explanations, rather than just executable correctness. Tasks are structured to simulate realistic, chat-style developer prompts, incorporating all necessary context directly within the prompt. This includes elements such as:

• Code snippets
• Error messages
• Logs
• Configuration files
• Feature requirements

The overarching goal is to mimic authentic developer interactions to effectively evaluate an AI system's ability to reason through complex scenarios.

Key Responsibilities

• Craft realistic developer prompts across various categories, including code review, debugging, error diagnosis, configuration issues, and refactoring/design trade-offs.
• Source and adapt content from actual pull requests to develop authentic technical scenarios.
• Write clear, technically accurate model responses that showcase strong reasoning capabilities and high-quality explanations.
• Ensure that all responses consistently reflect sound engineering judgment and adhere to best practices.

Ideal Qualifications

• 2+ years of experience in software engineering, technical research, or technical education/content development.
• Bachelor’s degree in Computer Science, Software Engineering, or a related field (an advanced degree is preferred).
• Strong proficiency in one or more programming languages, such as Python, JavaScript, Java, or C++.
• Hands-on experience with debugging, testing, and code validation processes.
• Strong technical writing skills and exceptional attention to detail.
• Ability to clearly articulate trade-offs, assumptions, and underlying reasoning.

Project Details

• Start Date: Immediate
• Duration: 1–2 months
• Commitment: Part-time (15–25 hours per week, with flexibility up to 40 hours per week)
• Work Arrangement: Fully remote and asynchronous

Application & Onboarding Process

• Submit your resume.
• Complete a short AI-led interview (approximately 15 minutes).
• Receive follow-up communication within a few days regarding next steps and onboarding.

Compensation & Contract Terms

• Compensation: $70–$120 per hour, depending on experience and expertise.
• Payments: Weekly payments via supported payment platforms.
• Classification: Independent contractor.
• Project Scope: May be extended, shortened, or concluded based on performance and business needs.
• Confidentiality: No access to confidential or proprietary information from any employer, client, or institution.

Key skills/competency

• Software Engineering
• Code Quality Assurance
• Technical Writing
• AI Reasoning Evaluation
• Debugging
• Code Review
• Python/JavaScript/Java/C++
• Dataset Creation
• Software Development Workflows
• Problem Solving