Senior Software Engineer
Northwestern Mutual · Milwaukee, WI
- On site
- Full-time
- $178,440 / year
- Milwaukee, WI
Job highlights
- Lead AWS data platform and pipeline design.
- Architect and implement data lake/warehouse solutions.
- Build and optimize ETL/ELT workflows.
- Ensure data quality, governance, and security.
- Mentor engineers and influence technical roadmap.
About the role
About The Team
The Field domain data team builds and manages data pipelines, creating pipelines to collect data from multiple sources (databases, APIs, files), transform, and move data into storage systems (Data Mesh/data lakes / warehouses). We also automate ETL (Extract, Transform, Load) workflows. We design scalable data architecture and systems that can handle large-scale data reliably and efficiently. We ensure data quality, governance & security by validating and cleaning incoming data, applying governance policies (who can access what), and maintaining compliance and security controls. We work with other teams across the organization, collaborating with Product/business teams (to understand data needs), Data scientists (to support ML/AI use cases), and DevOps / platform teams.
What You'll Do
- Lead the design and delivery of end-to-end data platforms and pipelines (batch and streaming) using AWS services to support analytics, reporting, ML, and operational use cases.
- Define and own data architecture decisions, standards, and best practices. Produce and maintain Architecture Decision Records (ADRs), runbooks, and high-quality diagrams.
- Architect and implement data lake and/or data warehouse solutions (ingestion, storage, transformation, serving) using S3, Glue, Redshift, Athena, and related services.
- Build and optimize ETL/ELT workflows with Spark, Glue, Lambda, Step Functions, and orchestration frameworks (Airflow, Dagster).
- Design and operate streaming ingestion and processing using Kinesis, MSK, and Kafka ecosystems; collaborate on re-architecture efforts to reduce cost and improve resilience.
- Drive data modeling, schema design, partitioning, and query optimization for large data sets to meet performance and cost targets.
- Establish and operationalize data quality, validation, observability, lineage, and metadata practices (Glue Data Catalog, Data Hub/Amundsen or equivalent).
- Implement robust security, governance, and compliance controls (IAM, Lake Formation, KMS, VPC, data masking/security scanning) and collaborate with security and privacy teams.
- Champion cost optimization across the data stack (serverless architectures, right-sizing, Graviton/spot/commitment strategies) and provide continuous cost governance.
- Own production reliability: monitoring, SLIs/SLOs, incident response, postmortems, and continuous improvements.
- Lead and mentor engineers, run design reviews, and promote cross-team collaboration with product, analytics, ML, and platform teams.
- Influence roadmap, capacity planning, and hiring for the data engineering function; represent the team in senior technical forums.
What You'll Bring to the Role
- 7+ years of hands-on experience in data engineering, architecting and operating data platforms at scale.
- Demonstrated leadership on large, complex data projects and strong ability to influence technical and product stakeholders.
- Extensive hands-on experience with AWS data services: S3, Glue, Redshift (Spectrum/RA3), Athena, EMR, Lambda, Kinesis/MSK, DMS.
- Strong programming skills in Python and/or Scala (or Java); proven track record writing production-quality, testable code.
- Deep expertise in SQL with experience designing and optimizing queries for high-volume data.
- Experience with distributed processing frameworks (Spark) and modern ETL/ELT patterns.
- Proficiency with orchestration and workflow tools (Airflow, Step Functions, Dagster) and CI/CD for data pipelines.
- Strong skills in infrastructure-as-code (Terraform, CloudFormation, or CDK) and GitOps practices.
- Experience implementing data security, governance, and access controls at scale.
- Expertise in observability, monitoring, and incident management (CloudWatch, Prometheus, Grafana, structured logging, tracing).
- Excellent communication skills and proven ability to mentor and lead engineers.
Preferred Qualifications
- AWS certifications (e.g., AWS Certified Data Analytics – Specialty, AWS Certified Solutions Architect) or equivalent.
- Experience with data lakehouse technologies (Delta Lake, Iceberg, Databricks Unity Catalog), and large-scale performance troubleshooting.
- Familiarity with metadata and lineage tooling (Amundsen, DataHub, OpenLineage).
- Prior experience with cost-optimization programs (Graviton, serverless, spot instances) and demonstrable results.
- Contributions to open-source projects, publications, or conference presentations on data engineering topics.
- Experience working in regulated industries (finance, healthcare) and implementing compliance controls.
- AI usage for building Data Pipelines.
Skills You'll Have
- Strategic thinker with strong problem-solving and architectural judgment.
- Ability to manage multiple complex priorities and deliver high-quality work under deadlines.
- Strong mentoring and team leadership skills; fosters a culture of knowledge sharing and continuous improvement.
- Customer-focused: translates business needs into pragmatic, maintainable technical solutions.
- Strong attention to detail, excellent documentation habits, and commitment to operational excellence.
Key skills/competency
- Data Engineering
- AWS Data Services
- Data Architecture
- ETL/ELT
- Spark
- Python
- SQL
- Data Modeling
- Cloud Computing
- DevOps
Skills & topics
- Senior Software Engineer
- Data Engineer
- AWS
- Data Platform
- Data Pipeline
- ETL
- Spark
- Python
- Scala
- SQL
- Data Architecture
- Cloud Computing
- DevOps
- Machine Learning
- Analytics
How to get hired
- Tailor your resume: Highlight AWS data services, Python/Scala, Spark, and leadership experience.
- Showcase your projects: Detail experience with data lakes, ETL pipelines, and AWS tools.
- Prepare for technical questions: Review SQL optimization, data modeling, and distributed systems concepts.
- Demonstrate leadership: Be ready to discuss mentoring and influencing technical decisions.
- Research Northwestern Mutual: Understand their mission, values, and commitment to client interests.
Technical preparation
Master AWS data services (S3, Glue, Redshift, etc.).,Practice Python/Scala for data manipulation.,Review Spark, SQL, and ETL/ELT concepts.,Familiarize with IaC tools like Terraform.
Behavioral questions
Describe a complex data project you led.,How do you mentor junior engineers?,How do you ensure data quality and security?,How do you optimize costs in cloud environments?
Frequently asked questions
- What are the key AWS data services used in this Senior Software Engineer role at Northwestern Mutual?
- This Senior Software Engineer role at Northwestern Mutual heavily utilizes AWS data services including S3, Glue, Redshift, Athena, EMR, Lambda, Kinesis/MSK, and DMS for building and managing data platforms and pipelines.
- What programming languages are essential for the Senior Software Engineer position at Northwestern Mutual?
- Strong programming skills in Python and/or Scala (or Java) are essential for this Senior Software Engineer role at Northwestern Mutual, focusing on writing production-quality, testable code for data engineering tasks.
- How does Northwestern Mutual approach salary for a Senior Software Engineer?
- Northwestern Mutual offers competitive salaries for Senior Software Engineers, with a pay range typically between $118,960.00 and $205,200.00 USD, depending on experience, location, and specific pay structure. They prioritize fairness and transparency in their compensation.
- What level of experience is required for the Senior Software Engineer role at Northwestern Mutual?
- The Senior Software Engineer role at Northwestern Mutual requires a minimum of 7+ years of hands-on experience in data engineering, with a proven track record of architecting and operating data platforms at scale.
- Can you elaborate on the 'Data Mesh' concept mentioned in the Senior Software Engineer job description?
- The 'Data Mesh' concept in this Senior Software Engineer role refers to a decentralized data architecture approach where data is treated as a product, owned by domain teams, and discoverable, addressable, trustworthy, and secure for consumers. Northwestern Mutual aims to build data pipelines supporting this paradigm.
- What are the opportunities for career growth as a Senior Software Engineer at Northwestern Mutual?
- Northwestern Mutual emphasizes career growth for its Senior Software Engineers, offering opportunities to lead complex projects, mentor junior engineers, influence technical roadmaps, and develop expertise within a best-in-class financial services company.
- Does Northwestern Mutual offer remote or hybrid work for this Senior Software Engineer position?
- While the job description does not explicitly state the work arrangement, roles at Northwestern Mutual often offer flexible work schedules. It is recommended to clarify the specific work arrangement (on-site, hybrid, remote) during the application process.
Similar roles
Open positions we recommend based on this role.