AI Architect

Introduction

A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. You’ll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM technology, and Red Hat, you’ll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success. You’ll be encouraged to challenge the norm, explore new ideas, and create innovative solutions that deliver real results. Our culture of growth and empathy focuses on your long-term career development while valuing your unique skills and experiences.

Your Role And Responsibilities

We are seeking a visionary AI Architect to lead the design and implementation of autonomous AI systems. In this role, you will move beyond standard RAG (Retrieval-Augmented Generation) pipelines to architect "thinking" systems—AI agents capable of reasoning, tool use, long-term memory, and multi-agent collaboration.

You will define the reference architecture for our Agentic AI platforms on hyperscalers like Azure, AWS or IBM Cloud or on-premise, and leveraging state-of-the-art open-source and proprietary solutions to ensure our agents are observable, reliable, and scalable in production.

Business & Client Engagement Responsibilities

• Serve as the primary client-facing technical lead for Agentic AI engagements
• Support use‑case discovery and value definition
• Manage stakeholder expectations regarding autonomy levels, guardrails, risks, and operational impacts
• Produce client-friendly architectural narratives, diagrams, and options that simplify technical trade-offs and design decisions
• Collaborate with product owners, compliance teams, operations, and engineering to ensure holistic delivery of Agentic AI systems from concept to production

Required Technical And Professional Expertise

Agentic System Architecture

• Design Autonomous Loops rather than simple linear chains.
• Multi-Agent Orchestration: Design patterns for multi-agent collaboration
• State & Memory Management: Define strategies for managing agent
• Cloud Native Integration: Architect scalable agent hosting solutions on Hyperscalers and On premise

Production Engineering & Observability

• Agent Tracing: Implement end-to-end observability to trace complex, non-deterministic agent execution paths
• Evaluation Frameworks: to grade agent performance on reasoning accuracy, tool selection capabilities, and goal completion rates.
• Tool Interface Design: Define and leverage standard protocols for agents to securely interact

Governance & Guardrails

• Human-in-the-Loop (HITL): Design breakpoint architectures.
• Safety & Compliance: Implement guardrails to prevent hallucination, prompt injection, and unauthorized tool access.

Core Technical Stack (Must Haves)

• Language: Expert proficiency in Python with patterns for agent concurrency.
• Agent Frameworks: Deep hands-on experience with Agentic frameworks like LangChain, AutoGen, or Semantic Kernel.
• ML Ops & Observability: Advanced knowledge of MLflow for agent tracking, model registry, and specifically LLM Tracing.
• Cloud Platforms: Azure: Azure OpenAI Service, AI Search, CosmosDB, Azure Functions. AWS: Amazon Bedrock (Agents), SageMaker, DynamoDB, AWS Lambda. IBM Cloud: WatsonX.AI

Architectural Concepts

• Agentic Patterns: Deep understanding of agentic architectural patterns.
• Vector & Graph Data: Experience implementing Vector Databases and Knowledge Graphs as long-term memory for agents.
• Containerization: Proficiency with patterns for deploying agent runtimes.

Soft Skills & Leadership

• Ability to explain non-deterministic AI behavior to stakeholders (i.e., why the agent might take different paths to solve the same problem).
• Experience mentoring Senior Engineers in the shift from "Predictive ML" to "Generative/Agentic AI."

Preferred Technical And Professional Experience

Nice to have

• Experience with Fine-tuning small language models (SLMs) like Llama 3 or Phi-3 for specific tool-calling tasks to reduce latency/cost.
• Familiarity with the Model Context Protocol (MCP) and A2A standard.

Interview Challenge Scope

Be prepared to discuss how you would design a "Customer Support Agent" systems

Key skills/competency

• Agentic AI Architecture
• Multi-Agent Orchestration
• Cloud Native Integration
• Python Programming
• LangChain/AutoGen/Semantic Kernel
• MLflow
• Azure/AWS/IBM Cloud
• Vector Databases
• Knowledge Graphs
• AI Governance