Deep Learning Engineer, LLM Accuracy Evaluation

@ NVIDIA

Overview

The Deep Learning Engineer, LLM Accuracy Evaluation at NVIDIA pioneers new methodologies for assessing advanced deep learning models, including LLMs, RAG, agents, and vision models.

What You’ll Be Doing

You will collaborate closely with partners and the open-source community to deliver flagship models as highly optimized NVIDIA Inference Microservices (NIM). Key responsibilities include researching and developing innovative methodologies to evaluate model families, analyzing and enhancing AI/DL libraries, and building robust tools and infrastructure pipelines.

What We Need To See

• BS, MS, or PhD in Computer Science, AI, Applied Math, or related fields (or equivalent experience).
• 10+ years hands-on AI experience in NLP and large language models.
• Strong problem-solving, debugging, performance analysis, and documentation skills.
• Solid mathematical foundations and expertise in AI/DL algorithms.
• Excellent communication skills and ability to work independently and collaboratively.

Ways To Stand Out From The Crowd

• Experience in accuracy evaluation of LLMs (e.g., OpenLLM Leaderboard, HELM).
• Hands-on experience with inference and deployment tools like TensorRT, ONNX, or Triton.
• DevOps/MLOps passion in deep learning product development.
• Experience running large-scale workloads in HPC clusters.
• Strong understanding of Linux environments and containerization technologies like Docker.

Key Skills/Competency

• Deep Learning
• LLM
• AI Evaluation
• NLP
• GPU Clusters
• Inference
• DevOps
• MLOps
• HPC
• Docker