Lead AI Engineer FM Hosting LLM Inference

@ Capital One

Overview

At Capital One, we are creating responsible and reliable AI systems that change banking for good. Our longstanding leadership in machine learning drives real-time, personalized customer experiences and breakthrough product innovations. As part of this effort, you will help bring emerging AI capabilities to reimagine customer and business interactions.

Team Description

The Intelligent Foundations and Experiences (IFX) team works with cross-functional partners across Capital One to advance AI science and engineering. They build proprietary solutions central to business operations that deliver value to millions of customers.

Role Responsibilities

• Partner with engineers, research scientists, and program/product managers to deliver AI-powered products.
• Design, develop, test, deploy, and support AI software components including foundation model training, LLM inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Leverage a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, and PyTorch.
• Invent and introduce state-of-the-art optimization techniques for large-scale production AI systems.
• Contribute to the technical vision and long-term roadmap of foundational AI systems at Capital One.

Ideal Candidate

If you are someone who loves building robust systems, maintains high quality work, stays on top of the latest AI research, and has a passion for solving undefined challenges, you will be a great fit for the Lead AI Engineer FM Hosting LLM Inference role.

Basic Qualifications

• Bachelor's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 4 years of relevant experience OR Master's degree plus 2 years of experience.
• At least 4 years of experience programming with Python, Go, Scala, or Java.

Preferred Qualifications

• 6 years of experience deploying scalable and responsible AI solutions on cloud platforms (AWS, Google Cloud, Azure, etc.).
• Experience in designing, developing, delivering, and supporting AI services.
• Proficient in AI/ML algorithms and technologies including LLM Inference, Similarity Search, VectorDBs, and Guardrails using languages such as Python, C++, C#, Java, or Golang.
• Expertise in optimizing training and inference software for improved hardware utilization, latency, throughput, and cost.

Additional Information

Compensation varies by location: Cambridge, MA and McLean, VA offer $193,400 - $220,700, while New York, NY, San Francisco, CA, and San Jose, CA offer $211,000 - $240,800 per year. This role is eligible for performance-based incentive compensation. Capital One offers a comprehensive benefits package and considers sponsoring qualified applicants for employment authorization.

Key skills/competency

• AI Systems
• Machine Learning
• LLM Inference
• Cloud Platforms
• Python
• Optimization
• PyTorch
• Data Science
• Software Engineering
• Scalability