Testing Engineer – AI & LLM
Indium · India
- Hybrid
- Contract
- $100,000 / year
- India
Job highlights
- Test cutting-edge LLM and Generative AI applications.
- Design and execute diverse test plans.
- Validate AI model outputs and behavior.
- Develop and enhance automated testing.
- Collaborate with AI/ML development teams.
About the role
Testing Engineer – AI & LLM
We’re looking for a Testing Engineer to join our AI/ML team and work on cutting-edge LLM and Generative AI applications.
What you’ll be doing:
- Design and execute test plans for AI/LLM-based applications
- Perform functional, regression, and performance testing
- Validate prompt engineering techniques and model outputs
- Identify and track issues in AI behaviour and responses
- Collaborate with AI/ML teams to improve model performance
- Build and enhance automated testing frameworks
What we’re looking for:
- Experience in software testing with exposure to AI/ML or LLMs
- Hands-on experience with LLM testing & prompt engineering
- Strong understanding of model evaluation (accuracy, bias, relevance)
- Proficiency in Python/Java and automation tools (Selenium, PyTest)
- Strong analytical and problem-solving skills
About Indium:
Indium Tech is a next-gen digital engineering company driving innovation across Application Engineering, Generative AI, Data & AI, Cloud & Product Engineering, Intelligent Automation, Low-Code Development, Gaming, and Quality Engineering. With over 25 years in the industry and a vibrant team of 5,000+ professionals across India, Indium is trusted by global enterprises to deliver cutting-edge digital solutions. Our expertise has earned us recognition from Forbes as one of the Top AI Products and Clutch a Top Data Analytics Company, and ISG as Strong Contender in Data Science Services. We're proud to be Great Place to Work certified and ranked on the Inc. 5000 list of America's fastest-growing companies.
Key skills/competency:
- Testing Engineer
- AI & LLM
- Generative AI
- Prompt Engineering
- Model Evaluation
- Python
- Java
- Selenium
- PyTest
- Automated Testing
Skills & topics
- Testing Engineer
- AI
- LLM
- Generative AI
- Prompt Engineering
- Automation Testing
- Python
- Selenium
- PyTest
- Remote
How to get hired
- Tailor your resume: Highlight AI, LLM, prompt engineering, and specific testing tools like Selenium or PyTest.
- Showcase your experience: Detail your work with AI/ML models, model evaluation, and Python/Java proficiency.
- Prepare for technical questions: Be ready to discuss test plan design for AI applications and automation frameworks.
- Demonstrate problem-solving: Emphasize your analytical skills and experience identifying AI behavior issues.
- Understand Indium's mission: Research their digital engineering focus and AI/ML innovations.
Technical preparation
Practice AI/LLM testing scenarios.,Build Python/Java test automation scripts.,Familiarize with Selenium and PyTest.,Study model evaluation metrics.
Behavioral questions
Describe a complex AI behavior issue you found.,How do you collaborate with development teams?,Share an example of improving test frameworks.,How do you handle validating subjective AI outputs?
Frequently asked questions
- What are the key responsibilities for a Testing Engineer at Indium focusing on AI & LLM?
- As a Testing Engineer at Indium, you will be responsible for designing and executing test plans for AI/LLM applications, performing functional, regression, and performance testing, validating prompt engineering techniques, and identifying/tracking issues in AI behavior. You will also collaborate with AI/ML teams and build automated testing frameworks.
- What specific experience does Indium look for in an AI & LLM Testing Engineer candidate?
- Indium seeks candidates with experience in software testing, particularly with AI/ML or LLMs. Key requirements include hands-on experience with LLM testing and prompt engineering, a strong understanding of model evaluation (accuracy, bias, relevance), and proficiency in Python/Java and automation tools like Selenium and PyTest.
- Is this a remote position for the Testing Engineer role at Indium?
- Yes, the Testing Engineer position at Indium, focusing on AI & LLM, is advertised as a remote role, allowing you to work from your preferred location.
- What technologies are crucial for the Testing Engineer – AI & LLM role at Indium?
- Proficiency in Python or Java, along with experience in automation tools such as Selenium and PyTest, is crucial. Familiarity with LLM testing, prompt engineering, and model evaluation concepts is also highly valued for this role at Indium.
- How does Indium evaluate AI model performance?
- Indium looks for candidates with a strong understanding of model evaluation metrics such as accuracy, bias, and relevance when assessing AI model performance. This is a critical aspect of the Testing Engineer role.
- What is Indium's background in AI and digital engineering?
- Indium Tech is a digital engineering company with over 25 years of experience, specializing in areas like Generative AI, Data & AI, and Quality Engineering. They are recognized by Forbes and Clutch and are Great Place to Work certified.