Data Engineer

Summary

Apple's Media, Graphics, and Compute Technologies Group (MGC) is seeking a talented and dedicated Data Engineer to join their Data Engineering team. This team is crucial for supporting data-driven analytics through big data collection, warehousing, and analytics at scale. They provide the infrastructure for numerous trend and operational dashboards, as well as ad-hoc use cases for services like Apple TV, Apple Music, and FaceTime. The team leverages Generative AI and Machine Learning to deliver best-in-class data analytics and monitoring.

This role offers the opportunity to help design, enhance, and develop a very-high-volume processing pipeline. You will collaborate with talented engineers within the team and cross-functional teams in an agile and dynamic environment that values engineering excellence, creativity, and innovation, contributing significantly to the next generation of processing pipelines and data analytics platforms.

Description

The team utilizes modern Data Engineering, Generative AI, and Machine Learning technologies to provide actionable insights. In this role, you will:

• Collaborate with data scientists across functional teams to define and enhance performance metrics that provide valuable insights for stakeholders.
• Build and maintain ingestion pipelines for real-time data processing.
• Develop real-time applications driving operational monitoring.
• Create batch ETL/ELT applications populating the data warehouse.
• Apply Generative AI and Retrieval Augmented Generation (RAG) techniques to enhance data analytics capabilities.
• Implement Machine Learning technologies for anomaly detection.

Responsibilities

• Tune and scale Apache Kafka producer/consumer, Spark Structured Streaming, and Flink applications in a cloud environment.
• Manage and monitor large-scale data collection and analytics pipelines at the application level.
• Perform capacity planning to scale infrastructure and applications running on Kubernetes.
• Troubleshoot production issues and conduct performance analysis of distributed systems.
• Collaborate with cross-functional teams to ensure high availability and reliability of data pipelines.
• Stay current with the latest data engineering trends and apply corresponding technologies.

Minimum Qualifications

• Bachelor's degree in Computer Science or equivalent professional experience.
• Experience building large scale distributed systems in Java/Python or similar languages.
• Proficient in SQL.
• Experience with data warehouse architectures and dimensional modeling.
• Demonstrated ability to conduct performance analysis and troubleshoot large scale distributed systems.
• Strong collaboration skills with the ability to understand complex architectures and work effectively across teams.
• Hands-on experience with Docker and Kubernetes.

Preferred Qualifications

• Production experience with Apache Kafka, Spark, or Flink.
• Working knowledge of Trino or similar distributed query engines.
• Experience building multi-agent AI systems or agentic workflows.
• Familiarity with Retrieval Augmented Generation (RAG) techniques working in conjunction with LLMs.
• Experience with creating and consuming Model Context Protocol (MCP) services.

Key skills/competency

• Data Engineering
• Generative AI
• Machine Learning
• Apache Kafka
• Spark
• Flink
• Kubernetes
• SQL
• Python
• Java
• Distributed Systems
• Data Warehousing