Data Engineer Data Platforms Google

Introduction

A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. You’ll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from strategic partners, robust IBM technology, and Red Hat, you’ll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success and your long-term career development is paramount.

Your Role And Responsibilities

As a Data Engineer specializing in Google's data platforms at IBM, you will design, build, and maintain data engineering solutions on Google's Cloud ecosystem. The responsibilities include:

• Design and develop batch and real-time data pipelines using Google DataProc, DataFlow, PubSub, BigQuery, and Big Table.
• Utilize Google Cloud Storage, BigTable, and other services with Spark, Hadoop, Apache Beam, or Python to build engineering solutions.
• Manage data platforms with Google Cloud Scheduler and Cloud Composer (Airflow), ensuring smooth operations.
• Implement data migration solutions for seamless data transfer between systems.
• Optimize the data layer using Google services such as BigQuery, Big Table, and Cloud Spanner.

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• Expertise in designing and maintaining solutions on Google Cloud ecosystem.
• Experience in developing batch and real-time data pipelines.
• Proficiency with Google Cloud Storage, BigTable, BigQuery, DataProc, and DataFlow.
• Knowledge in scheduling and managing data platforms using Cloud Scheduler and Cloud Composer.
• Understanding of data layer design using Google services.

Preferred Technical And Professional Experience

• Exposure to open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.
• Experience in developing data migration solutions using Google services.
• Proficiency in using Cloud Composer for managing data pipelines.

Key skills/competency

Google Cloud, Data Pipelines, BigQuery, Data Migration, Cloud Composer, Apache Beam, Spark, Hadoop, Python, Data Engineering