Data Engineer 2

Overview

The Cloud & AI organization at Microsoft is dedicated to accelerating our mission and bold ambitions, ensuring the security of digital technology platforms, devices, and clouds in our customers’ diverse environments, as well as securing our own internal estate. Our culture thrives on a growth mindset, inspiring excellence, and empowering teams and leaders to bring their best daily, creating life-changing innovations globally. Microsoft stands as one of the largest enterprise service companies worldwide.

Do you possess a passion for studying, analyzing, and confronting the most intricate digital security threats? Are you a perpetual learner, eager to adapt and acquire new skills regularly? Do you find fulfillment in resolving challenges faced by security response teams? If so, this Data Engineer 2 opportunity is for you.

About the Team

The Telemetry Enforcement Engineering team, a crucial part of Cyber Defense Operations (CDO), is actively seeking a Data Engineer 2. This role involves driving the centralization of access and curation of critical security data, while applying rigorous operational standards to ensure real-time availability and accuracy across thousands of mission-critical data sources. CDO, led by Microsoft’s Chief Information Security Officer, ensures that all information and services are protected, secured, and available through innovation and a robust risk & management framework.

Microsoft's mission is to empower every person and organization to achieve more. We foster a culture of inclusion grounded in respect, integrity, and accountability, enabling all employees to thrive.

Responsibilities

• Data-Driven Analysis: Analyze key metrics, KPIs, and other data sources (e.g., bugs, unhealthy data pipelines) to identify security trends and escalate appropriately. Recommend improvements and metrics to address measurement gaps, leveraging multiple data sources for analysis and anomaly detection.
• Technical Insight: Provide expert technical insight on incident analysis and management, threat mitigation, and translate these into platform requirements, collaborating closely with software development teams.
• Operational Rigor: Demonstrate exceptional operational rigor, supported by real-world experience in cybersecurity operations, threat mitigation, and incident response.
• Stakeholder Management: Manage critical stakeholder calls and meetings, including during non-business hours, while addressing critical security incidents.
• Security Knowledge: Possess extensive hands-on knowledge of security concepts, including cyber-attacks, techniques, threat vectors, risk management, and incident management.
• Data Engineering & Management: Understand and administer data source access and availability, proactively defining controls and policies to enhance both.
• On-call Duties: Participate in on-call rotations and duties as required.

Qualifications

Required Minimum Qualifications:

• Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or a related field AND 1+ year(s) experience in business analytics, data science, software development, data modeling, or data engineering OR
• Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or a related field AND 2+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.
• 1+ years of experience in a Data Engineer or Software Engineer role with a focus on cloud-based big data processing.

Other Requirements:

Ability to meet Microsoft, customer, and/or government security screening requirements are required for this role. These include, but are not limited to, the Microsoft Cloud Background Check and Citizenship & Citizenship Verification for export-controlled information access.

Additional Or Preferred Qualifications:

• 3+ years of experience in data engineering.
• 3+ years of hands-on experience with Microsoft Kusto clusters (KQL), Azure Cosmos DB, Azure Access Control methods.
• 3+ years assisting in tuning and optimizing data source and data pipeline availability rules and alerts.
• 3+ years collaborating with IT, DevOps, and other teams to remediate vulnerabilities and improve security controls/policies.
• 3+ years of pipeline experience, including Synapse/Python.
• 3+ years of experience of DRI/Live site Engagement/Oncall experience.
• 8+ years of security operations work experience, with 5 years of experience collaborating with IR/SOC.

Key skills/competency

• Data Engineering
• Cloud Security
• Data Analysis
• Kusto KQL
• Azure Cosmos DB
• Data Pipelines
• Incident Response
• Threat Mitigation
• Python
• Operational Rigor