AI Test Specialist

About LTIMindtree

LTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies.

We are seeking an AI Test Specialist to join our team across various locations.

The Role: AI Test Specialist

As an AI Test Specialist, you will play a crucial role in ensuring the consistency, factual accuracy, and ethical compliance of AI systems, especially within insurance risk management use cases. You will design and implement robust testing frameworks for large language models (LLMs) and autonomous agents, integrate testing pipelines with advanced AI solutions, and build comprehensive validation environments to ensure the reliability and integrity of client-tailored AI applications.

Key Responsibilities

• Design and automate test frameworks to verify consistency, factual accuracy, tone alignment, and contextual precision for insurance risk management use cases.
• Build versioned golden data sets and automated benchmarking suites for regression fine-tuning validation aligned to clients' operations.
• Integrate AI testing pipelines with OpenAI for Clients tailored solutions and model evaluations.
• Develop and validate autonomous agents for multi-step reasoning, tool use, and goal-oriented decision making.
• Create test harnesses for LangChain, LangGraph, or custom orchestration layers mapped to clients' operational processes.
• Build environments to test reasoning consistency, error handling, task prioritization, and outcome reliability in client scenarios.
• Implement detection scoring for hallucinations, looping, goal misalignment, and ethical constraint violations per client standards.
• Construct testbeds to assess collaboration, task distribution, and negotiation among agents in claims/risk management contexts.
• Develop strategies to test MCP-based systems, ensuring robust context passing among models, agents, and tools.
• Validate schema alignment, context synchronization, and response correctness across chained client AI systems.
• Implement versioned tests to prevent data loss or intent drift between model interactions critical to clients' operations.
• Build end-to-end automation pipelines integrating data, model, and agent validation within CI/CD for clients' AI systems.
• Create modular frameworks supporting traditional API tests and LLM/agent tests in a single scalable stack for Clients globally.
• Leverage MLOps tools for traceable experiment validation tailored to clients' needs.
• Implement feedback loops tracking model drift, hallucination rates, and performance degradation in clients' applications.
• Automate validation across ingestion, feature transformation, and prompt context building for clients' specific datasets.
• Generate synthetic test data to ensure broad model validation coverage.
• Validate fairness, bias, and ethical compliance of outputs aligned with clients' commitment to responsible AI.
• Define and track test KPIs: factuality, coherence, reasoning depth, task completion success rate.

Equal Opportunity Employer

LTIMindtree is proud to be an equal opportunity employer. We are committed to equal employment opportunity regardless of race, ethnicity, nationality, gender, gender-identity, gender expression, language, age, sexual orientation, religion, marital status, veteran status, socio-economic status, disability or any other characteristic protected by applicable law.

Key skills/competency

• AI testing
• LLM validation
• Test automation
• Ethical AI
• Data quality assurance
• Prompt engineering
• Autonomous agents
• CI/CD pipelines
• MLOps
• Risk management