Data Engineer - Data Platforms - Google
IBM
Job Overview
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.
Job Description
Introduction
A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. You’ll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. At IBM Consulting, curiosity fuels success by encouraging innovative, real results-driven ideas and a culture that values individual growth and diverse skills.
Your Role And Responsibilities
As a Data Engineer - Data Platforms - Google at IBM, you will design, build, and maintain data engineering solutions on Google Cloud. Your role involves developing batch and real-time data pipelines, data migration solutions and optimizing data layer designs.
- Design Data Pipelines: Develop batch and real-time pipelines using Google DataProc, DataFlow, PubSub, BigQuery, and Big Table.
- Develop Engineering Solutions: Leverage Google Cloud Storage, Spark/Hadoop on DataProc, and Apache Beam or Python on DataFlow.
- Manage Data Platforms: Utilize Cloud Scheduler and Cloud Composer (Airflow) to ensure pipeline efficiency.
- Implement Data Migration: Create and deploy solutions for seamless data transfer.
- Optimize Data Layer: Enhance data storage and retrieval using BigQuery, Big Table, and Cloud Spanner.
Preferred Education
Master's Degree
Required Technical And Professional Expertise
- Expertise in Google Cloud ecosystem including DataProc, DataFlow, PubSub, BigQuery, etc.
- Experience in developing batch and real-time data pipelines.
- Proficiency in using open-source tools and Google Cloud services.
- Knowledge in managing data platforms and scheduling operations.
- Understand data layer design for efficient storage and retrieval.
Preferred Technical And Professional Experience
- Exposure to open-source technologies: Apache Airflow, dbt, Spark/Python, or Spark/Scala.
- Experience in developing and implementing data migration solutions.
- Expertise in Cloud Composer (Airflow) for managing data platforms.
Key skills/competency
- Google Cloud
- Data Pipelines
- BigQuery
- Data Migration
- Cloud Composer
- Apache Beam
- DataFlow
- Airflow
- Data Engineering
- Python
How to Get Hired at IBM
- Research IBM Consulting: Understand their culture, mission, and values.
- Customize your resume: Highlight Google Cloud and pipeline expertise.
- Showcase technical skills: Demonstrate experience with Apache Airflow and Python.
- Prepare for interviews: Practice answers on data migration and pipeline management.
Frequently Asked Questions
Find answers to common questions about this job opportunity
Explore similar opportunities that match your background