Test Automation Specialist

About Morgan Stanley

Since 1935, Morgan Stanley has been known as a global leader in financial services, always evolving and innovating to better serve clients and communities in more than 40 countries around the world. In the Human Capital Management division, we provide advice and insight to attract, retain, reward, and develop a talented global workforce, giving employees the support and tools they need to succeed. Join our Budapest Analytics Center of Excellence, a cornerstone of Morgan Stanley’s global Data and Intelligence organization. Our team plays a pivotal role in advancing the firm’s Artificial Intelligence initiatives by designing and delivering innovative validation solutions for AI-driven products. With AI recognized as one of the top strategic priorities within Human Capital Management, this is an exciting opportunity to contribute to a high-visibility program that shapes how the firm adopts and governs emerging technologies.

The Opportunity: Test Automation Specialist

We're seeking someone to join our team as a Test Automation Specialist in our Human Capital Management - AI Model Validation Team. This role is pivotal in ensuring the reliability, quality, and compliance of Generative AI-powered (GenAI) products through systematic validation, testing, and evaluation. You will work closely with product managers, data scientists, and engineering teams to understand product requirements, assess AI model behavior, and drive the development of automated and AI-assisted validation frameworks.

What You'll Do In The Role

• Drive the validation of GenAI-based products, models and AI-driven product features, ensuring outputs meet business requirements and ethical standards.
• Develop, maintain, and improve automated or semi-automated validation workflows, leveraging scripting and AI-assisted testing tools.
• Collaborate with product owners, developers, and other stakeholders to translate business requirements into testable validation criteria.
• Analyse and interpret validation results to identify inconsistencies, biases, and performance issues in AI model outputs.
• Create detailed documentation on validation methodology, results, and recommendations.
• Support continuous improvement by proposing process enhancements and contributing to validation best practices across the team.
• Communicate validation findings effectively to both technical and non-technical stakeholders, ensuring shared understanding and actionable outcomes.
• Participate in cross-functional reviews to help shape product design and feature rollouts from a validation perspective.

What You'll Bring To The Role

• Bachelor’s or Master's degree in Mathematics, Computer Science, Physics, Economics, or a related quantitative field.
• At least 5+ years of professional experience in software testing, test automation, business analysis, or a related technical validation role would generally be expected to find the skills required for this role.
• Experience with natural language processing, large language models (LLM) and/or generative AI is a plus.
• Genuine interest in AI and other emerging technologies.
• Knowledge in Python and understanding of various data sources and technologies such as relational databases (SQL), data lakes and APIs.
• Strong analytical and problem-solving skills, with attention to detail and rigor in testing methodologies.
• Familiarity with common statistical methods is a plus.
• Ability to drive projects independently, work with diverse stakeholders, and deliver results from conception to completion.

Key skills/competency

• AI Model Validation
• Generative AI (GenAI)
• Test Automation
• Python
• SQL
• Natural Language Processing (NLP)
• Large Language Models (LLM)
• Data Analysis
• Problem-Solving
• Ethical AI