Senior Deep Learning Engineer, LLM Accuracy Evaluation

About the Role: Senior Deep Learning Engineer, LLM Accuracy Evaluation

NVIDIA is looking for senior engineers to pioneer new methodologies for accurately assessing the performance of ground-breaking deep learning models, including Large Language Models (LLMs), RAG, agents, and vision models. In this role, you will collaborate across the organization to bring the latest flagship models from our community and partners—such as Gemma and Llama-3—to life as optimized NVIDIA Inference Microservices (NIM).

This position offers an outstanding opportunity to craft the future of AI at a fast-growing company at the forefront of the AI revolution. Join our team of world-class software engineers and partners to deliver the most advanced models with lightning-fast inference. You'll work on the most powerful, enterprise-grade GPU clusters capable of hundreds of PetaFLOPS and gain early access to unreleased hardware, making a direct impact on NVIDIA's roadmap and the broader AI landscape!

What You'll Be Doing

• Collaborate closely with our partners and the open-source community to deliver their flagship models as highly optimized NVIDIA Inference Microservices (NIM).
• Research and develop innovative deep learning methodologies to accurately evaluate new model families across diverse domains.
• Analyze, influence, and enhance AI/DL libraries, frameworks, and APIs, ensuring consistency with the best engineering practices.
• Research, prototype, and build robust tools and infrastructure pipelines to support our groundbreaking AI initiatives.

What We Need To See

• BS, MS, or PhD in Computer Science, AI, Applied Math, or a related field, or equivalent experience.
• 10+ years of hands-on experience in AI for natural language processing (NLP) and large language models (LLMs).
• Strong problem-solving, debugging, performance analysis, test design, and documentation skills.
• Solid mathematical foundations and expertise in AI/DL algorithms.
• Excellent written and verbal communication skills, with the ability to work both independently and collaboratively in a fast-paced environment.

Ways To Stand Out From The Crowd

• Experience in accuracy evaluation of LLMs (OpenLLM Leaderboard or HELM).
• Hands-on experience with inference and deployment environments like TensorRT, ONNX, or Triton.
• Passion for DevOps/MLOps practices in deep learning product development.
• Experience running large-scale workloads in high-performance computing (HPC) clusters.
• Strong understanding of Linux environments and containerization technologies like Docker.

Key skills/competency

• Deep Learning
• LLM Accuracy Evaluation
• Natural Language Processing (NLP)
• AI/DL Algorithms
• Inference Optimization
• TensorRT
• ONNX
• Triton
• DevOps/MLOps
• High-Performance Computing (HPC)