Research Engineer II
Microsoft
Job Overview
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
Overview
AI agents are rapidly becoming foundational across industries, transforming how organizations automate workflows, make decisions, and scale productivity. Microsoft Agent 365 is the unified control plane for AI agents, providing a comprehensive platform for registry, lifecycle management, access governance, observability, interoperability, and security.
The Research Engineer II will develop core tooling, frameworks, and infrastructure to enable agents to use tools effectively, assess tool performance, and drive continuous improvement across the agent ecosystem.
Responsibilities
- Design and develop evaluation pipelines for offline and online experimentation.
- Automate performance metrics including success rates, latency, and reliability.
- Build and operationalize tool comparison frameworks with scorecards and dashboards.
- Implement telemetry, logging, and realtime monitoring systems.
- Strengthen backend infrastructure with caching, rate-limiting, and security hardening.
- Develop internal tools and packages to streamline tool onboarding and integration.
- Collaborate with program managers, engineering teams, and Responsible AI partners on end-to-end solutions.
- Contribute to long-term architecture and strategy for the Agent 365 tooling ecosystem.
Qualifications
Required: Bachelor's Degree in Computer Science or related field and 2+ years of technical engineering experience with languages such as C, C++, C#, Java, JavaScript, or Python or equivalent experience.
Other Requirements: Must pass Microsoft Cloud Background Check upon hire and every two years thereafter.
Preferred: Advanced degree or equivalent experience, backend services/API development, experience with large language models, and cloud-based distributed systems (Azure, AWS, or GCP).
Key skills/competency
- Evaluation pipelines
- Backend development
- Telemetry systems
- Tool automation
- Security hardening
- Cloud services
- Distributed systems
- Programming languages
- Data-driven decisions
- Monitoring and logging
How to Get Hired at Microsoft
- Customize Resume: Highlight AI and backend skills.
- Research Microsoft: Understand Agent 365 and culture.
- Prepare Projects: Emphasize tool development experience.
- Practice Interviews: Focus on coding and system design.
Frequently Asked Questions
Find answers to common questions about this job opportunity
Explore similar opportunities that match your background