Software Development Engineer in Test III - AI
HighLevel · India
- Hybrid
- Full-time
- $140,000 / year
- India
Job highlights
- Test AI-driven products using advanced AI tools.
- Design and implement automation frameworks for AI systems.
- Evaluate and improve RAG pipelines for accuracy.
- Collaborate with cross-functional teams on AI initiatives.
- Mentor junior engineers and lead quality efforts.
About the role
About HighLevel:
HighLevel is an AI-powered business operating system that gives agencies, entrepreneurs and SMBs the infrastructure to build, automate and scale. Today, HighLevel supports SMBs across 150+ countries, fueling community-driven growth rooted in real customer outcomes.
To date, businesses operating on HighLevel have generated over $7 billion in ecosystem value, demonstrating the impact of shared infrastructure at scale. By centralizing conversations, automation and intelligence into one system, we help businesses move faster, reduce complexity and execute efficiently.
Behind the platform, HighLevel powers more than 4 billion API hits and 2.5 billion message events daily. With 250 terabytes of distributed data, 250+ microservices and over 1 million domain names supported, our architecture is built for performance, resilience and long-term scalability.
Our people
With over 2,000 team members across 10+ countries, HighLevel operates as a global, remote-first organization built for speed and ownership. We value initiative, clarity and execution, creating space for ambitious people to build systems that support millions of businesses worldwide. Here, innovation thrives, ideas are celebrated and people come first, no matter where they call home.
Our impact
Every month, HighLevel enables more than 1.5 billion messages, 200 million leads and 20 million conversations for the more than 1 million businesses we support. Behind those numbers are real people building independence, expanding opportunity and creating measurable impact. We’re proud to be a part of that.
Learn more about us on our YouTube Channel or Blog Posts.
About the Role:
We are looking for a Senior Software Development Engineer in Test (SDET 3) with strong expertise in AI tools and testing AI-driven products. This role requires a hands-on individual who can design and execute test strategies for AI systems, automate intelligently, and mentor junior team members. You will work independently, drive tasks to closure, and play a key role in ensuring the quality of our AI initiatives.
Requirements:
- 5+ years of overall experience in QA/SDET roles
- Proven experience working with AI tools (e.g., OpenAI, Claude, or similar LLM platforms)
- Strong understanding of testing AI/ML products and their unique challenges
- Hands-on knowledge of RAG pipelines and their testing methodologies
- Proficiency in automation frameworks (e.g., Playwright, Cypress, Selenium, or similar)
- Experience with API testing (Postman, REST, GraphQL, gRPC)
- Strong problem-solving skills with the ability to work independently
- Mentorship and leadership skills in guiding peers or junior engineers
Responsibilities:
- Design, develop, and maintain automation frameworks and test strategies for AI products
- Test and validate AI-powered features (including OpenAI, Claude, and other AI integrations)
- Evaluate and improve RAG (Retrieval-Augmented Generation) flows, ensuring accuracy and performance
- Leverage AI automation tools to optimize test coverage and efficiency
- Collaborate closely with developers, product managers, and AI researchers to ensure high-quality releases
- Own tasks end-to-end, working independently to deliver on commitments.
- Mentor and guide junior SDETs, fostering a culture of quality and continuous learning
Nice to have:
- Experience with CI/CD pipelines and cloud platforms (AWS/GCP/Azure)
- Familiarity with MLOps pipelines and monitoring AI systems
- Exposure to performance and scalability testing of AI workloads
Key skills/competency:
- AI Testing
- SDET
- Automation Frameworks
- RAG Pipelines
- API Testing
- Problem-Solving
- Mentorship
- LLM Platforms
- Software Development
- Quality Assurance
Skills & topics
- Software Development Engineer in Test
- AI Testing
- Automation
- RAG Pipelines
- LLM
- SDET
- QA
- API Testing
- Playwright
- Cypress
- Selenium
- HighLevel
- AI
- Machine Learning
- Cloud
How to get hired
- Tailor your resume: Highlight your AI testing experience, automation skills, and RAG pipeline knowledge. Quantify achievements where possible.
- Showcase AI expertise: Emphasize your experience with LLM platforms like OpenAI or Claude and your understanding of AI/ML testing challenges.
- Demonstrate leadership: Provide examples of your ability to mentor junior engineers and lead testing initiatives independently.
- Prepare for technical interviews: Be ready to discuss your experience with automation frameworks, API testing, and AI-specific testing methodologies.
- Understand HighLevel's mission: Research their AI-powered business operating system and how your role contributes to their impact.
Technical preparation
Behavioral questions
Frequently asked questions
- What specific AI tools or LLM platforms are most relevant for the Senior Software Development Engineer in Test III role at HighLevel?
- HighLevel is looking for candidates with proven experience in AI tools such as OpenAI, Claude, or similar LLM platforms. Your understanding and ability to test these systems, especially within RAG pipelines, will be critical for success in this Senior SDET III position.
- How does HighLevel leverage AI in its business operating system, and what does that mean for testing?
- HighLevel uses AI to provide a business operating system for agencies, entrepreneurs, and SMBs, focusing on automation and scalability. For testing, this means a strong emphasis on validating AI-powered features, ensuring the accuracy and performance of RAG flows, and optimizing test coverage using AI automation tools.
- What is HighLevel's stance on using AI tools in the hiring process for the Senior Software Development Engineer in Test III role?
- HighLevel may use AI tools to assist in reviewing applications and analyzing resumes for the Senior Software Development Engineer in Test III position. However, final hiring decisions are always made by humans, ensuring that human judgment remains central to the process.
- What kind of mentorship responsibilities are expected of a Senior Software Development Engineer in Test III at HighLevel?
- As a Senior SDET III, you are expected to mentor and guide junior SDETs. This involves fostering a culture of quality and continuous learning, sharing your expertise in AI testing, automation, and RAG pipelines to help your peers grow.
- Are there opportunities to work with cloud platforms and CI/CD in this Senior SDET III role at HighLevel?
- Yes, experience with CI/CD pipelines and cloud platforms like AWS, GCP, or Azure is considered a plus for the Senior Software Development Engineer in Test III role. Familiarity with MLOps and monitoring AI systems is also beneficial.
- What is RAG, and why is testing it important for the Senior Software Development Engineer in Test III role at HighLevel?
- RAG stands for Retrieval-Augmented Generation. Testing RAG pipelines is crucial for AI products as it ensures the accuracy, relevance, and performance of the AI's ability to generate responses based on retrieved information. Hands-on knowledge of testing these flows is a requirement for this role.
- How does HighLevel operate as a company, and what is the work environment like for a Senior Software Development Engineer in Test III?
Similar roles
Open positions we recommend based on this role.