12 days ago
AI Application Test Engineer
Halian | Managed Services, Recruitment Agency & Contract Staffing
On Site
Full Time
$120,000
Abu Dhabi Emirate, United Arab Emirates
Job Overview
Job TitleAI Application Test Engineer
Job TypeFull Time
CategoryCommerce
Experience5 Years
DegreeMaster
Offered Salary$120,000
LocationAbu Dhabi Emirate, United Arab Emirates
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
Job Overview
The AI Application Test Engineer at Halian is responsible for testing AI-powered applications, including conversational AI and RAG-based systems, ensuring they meet quality standards and perform reliably at scale.
Key Responsibilities
- Design and execute test cases for AI/ML applications
- Perform end-to-end testing across development, staging, and production
- Test LLM-based applications for accuracy and consistency
- Validate retrieval systems, vector search functionality, and document ingestion pipelines
- Conduct functional, regression, and integration testing for AI components
- Identify and document bugs, edge cases, and model behavior issues
- Collaborate with cross-functional teams for quality across the AI lifecycle
- Contribute to developing AI testing best practices and standards
Required Technical Skills
- 3+ years of AI/ML testing experience (chatbots, RAG-based solutions)
- Strong experience with Azure AI services including Azure OpenAI, Azure ML, Cognitive Services, and Azure AI Search
- Proficiency in test automation frameworks and tools (Selenium, Playwright, PyTest)
- Good coding skills in Python for test automation
- Understanding of LLM behavior, prompt testing, and conversational AI patterns
- Experience testing REST APIs and cloud-native applications on Azure
- Familiarity with version control using Git
Preferred Qualifications
- Experience with Azure DevOps and CI/CD pipelines
- Knowledge of data quality testing and validation techniques
- Familiarity with vector databases (e.g., Pinecone, Weaviate, Azure AI Search)
- Understanding of AI model evaluation metrics
- Experience in Agile environments using Jira or similar tools
- Exposure to MLOps practices
About Halian
With over 28 years of experience, Halian provides agile, practical solutions that transform businesses and careers. The company values diversity, equity, and inclusion, and is committed to connecting organizations with top talent from all backgrounds.
Key skills/competency
- AI Testing
- Conversational AI
- RAG Systems
- Azure AI
- Test Automation
- Python
- REST APIs
- Cloud Testing
- Agile
- MLOps
How to Get Hired at Halian | Managed Services, Recruitment Agency & Contract Staffing
- Research Halian's culture: Study the mission, values, and recent news.
- Customize your resume: Highlight AI testing and Azure skills.
- Tailor your application: Emphasize experience in conversational AI.
- Prepare for technical interviews: Review Python, automation and API testing.
Frequently Asked Questions
Find answers to common questions about this job opportunity
01What qualifications does Halian require for the AI Application Test Engineer role?
02How important is experience with Azure AI services at Halian?
03Can candidates with MLOps experience apply for the AI Application Test Engineer position at Halian?
04What is the focus of the testing work at Halian for this role?
05How does Halian value diversity for the AI Application Test Engineer role?
Explore similar opportunities that match your background