Data Engineer Data Platforms Google

Introduction

A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. Youll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys using robust IBM technology and strategic partners like Red Hat. Curiosity and innovation are at the core of IBM Consulting, where your unique skills and experiences are valued for long-term career growth.

Your Role And Responsibilities

As a Data Engineer specializing in Google's data platforms, you will:

• Design Data Pipelines: Develop batch and real-time data pipelines for Data Warehouse and Datalake using Google DataProc, DataFlow, PubSub, BigQuery, and Big Table.
• Develop Data Engineering Solutions: Utilize Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and DataFlow with Apache Beam or Python.
• Manage Data Platforms: Schedule and manage systems using Google Cloud Scheduler and Cloud Composer (Airflow).
• Implement Data Migration: Create data migration solutions that ensure seamless data transfers.
• Optimize Data Layer: Enhance design for efficient data storage and retrieval using BigQuery, Big Table, and Cloud Spanner.

Preferred Education & Expertise

Master's Degree preferred with substantial expertise in the Google Cloud ecosystem and open-source technologies like Apache Airflow, dbt, and Spark (Python/Scala).

Key skills/competency

• Google Cloud
• Data Pipelines
• Batch Processing
• Real-time Processing
• Data Migration
• Data Layer Design
• Apache Beam
• Spark
• Hadoop
• Cloud Composer