Principal Data Engineer

Overview

Core AI is at the forefront of Microsoft’s mission to redefine how software is built and experienced. We are responsible for building the foundational platforms, services, programming models, and developer experiences that power the next generation of applications using Generative AI. Our work enables developers and enterprises to harness the full potential of AI to create intelligent, adaptive, and transformative software.

The EngThrive Insights team is responsible for understanding the developer experience. Our mission is to deliver insights that enable Microsoft to be the best place to work as an engineer, where it is fast and easy to do great work.

In this role, you will serve as one of the leads for the data platform that underpins EngThrive insights. Your responsibilities will include designing and maintaining data pipelines, curating high-quality datasets, performing in-depth data analysis, developing dashboards, and building the data and AI infrastructure that supports these capabilities and our missions.

Success in this position requires versatility across multiple domains, including software engineering and data analytics, as well as a strong passion for, and understanding of, the tools and processes that shape the engineering workflow.

At Microsoft, our mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Responsibilities

• Shape the Product Vision: Define and influence the product roadmap by aligning technical strategy with business goals and customer needs.
• Architect for Scale and Sustainability: Design and evolve durable, scalable system architectures that balance long-term maintainability with short-term delivery needs, making thoughtful engineering trade-offs.
• Direct Impact: Leads by example and mentor others to produce extensible and maintainable code used across the platform. Creates and applies metrics to drive the quality and stability of code, as well as appropriate coding patterns and best practices.
• Foster Engineering Alignment: Build consensus across engineering teams and partner organizations by driving clarity, alignment, and shared ownership of technical direction.
• Build Foundational Capabilities: Contribute to and lead the development of core platform components and reusable building blocks that accelerate team velocity and product innovation.
• Mentor and Grow Talent: Support the growth of engineers through mentorship, knowledge sharing, and by enabling long-term ownership of key systems.
• Champion Customer-Centric Development: Engage directly with customers and product teams to capture feedback, understand demand signals, and refine product messaging—ensuring the voice of the customer shapes product evolution.
• Operational Excellence: Drive operational excellence in managing and operating large-scale data systems with a high bar for data quality, stability, security and privacy. Set high standards for reliability and performance.

Qualifications

Required Qualifications:

• Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.

Other Requirements:

• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred Qualifications:

• Demonstrated expertise in building, evolving and leading complex data platforms.
• Experience with data visualization technologies such as PowerBI, Tableau, Grafana, or similar.
• 4+ years of experience working with complex data sets, data pipelines and solving challenging data problems at scale.
• 4+ years of experience and expertise working with data products such as Microsoft Fabric, Databricks, or similar.

Key skills/competency

• Data Platform Engineering
• Data Pipeline Design
• Data Analysis
• AI Infrastructure
• Generative AI
• Scalable System Architecture
• Product Vision
• Mentorship
• Operational Excellence
• Microsoft Fabric / Databricks