Senior Prompt Engineer

Senior Prompt Engineer

As a Senior Prompt Engineer specializing in SaaS application testing at Oracle, you will be instrumental in designing and validating prompts for large language models. Your role involves assessing AI system performance and integrating generative AI directly into QA workflows. You will collaborate across various teams to embed advanced AI tools into the software development lifecycle, significantly enhancing testing efficiency, proactively identifying issues, and driving innovation through strategic AI adoption.

Key Responsibilities

• Prompt Design & Evaluation: Create, test, and refine prompts for AI-powered features within the SaaS platform. Evaluate AI outputs for accuracy, consistency, bias, and alignment with business requirements. Design edge case scenarios and stress tests to identify prompt/model failure modes.
• AI-Driven QA Integration: Incorporate AI-based tools and techniques into both automated and manual QA processes. Utilize LLMs to generate test cases, validate UI/UX behavior, and pinpoint potential bugs or inconsistencies. Collaborate closely with QA and DevOps teams to embed AI models into CI/CD pipelines.
• Strategic AI Adoption: Champion and lead the adoption of AI tools and workflows across engineering, QA, and product teams. Evaluate AI tools, libraries, and APIs for seamless integration into the development and testing toolchain. Develop internal best practices, frameworks, and comprehensive documentation for AI-assisted development and testing.
• Cross-Functional Collaboration: Partner with Product Managers, QA Analysts, Data Scientists, and Engineers to continuously improve model performance and user experience. Communicate critical AI performance insights to stakeholders and actively contribute to roadmap discussions.

Qualifications

• 2+ years of experience working with large language models or AI systems in production environments.
• Strong understanding of prompt engineering concepts, generative AI behavior, and prompt evaluation techniques.
• Hands-on experience integrating AI tools into QA workflows or software development processes.
• Proficient in scripting or automation (e.g., Python, JavaScript, or Bash).
• Familiarity with SaaS platforms, agile methodologies, and modern software delivery pipelines.
• Ability to define and track performance metrics for AI-enhanced features and QA tools.
• Experience driving cross-team initiatives or change adoption within a technical organization.

Desired Skills and Experience

• Experience with AI development platforms (OpenAI, Anthropic, Hugging Face, etc.).
• Familiarity with AI safety, model robustness, prompt injection, and testing adversarial behavior.
• Exposure to MLOps practices, LangChain, vector databases, or RAG systems.
• Background in quality engineering, automation frameworks (e.g., Selenium, Cypress), or model evaluation pipelines.

Key skills/competency

• Prompt Engineering
• Large Language Models (LLMs)
• Generative AI
• SaaS Application Testing
• QA Automation
• CI/CD Pipelines
• Python/JavaScript
• Agile Methodologies
• AI Performance Metrics
• Cross-functional Collaboration