Principal Data Development and Engineering Expert

Principal Data Development and Engineering Expert

Co-operators is a leading Canadian financial services cooperative committed to building a resilient and sustainable society, with an internal team crucial to this strategy. Our people are our priority, offering a vibrant company culture and development opportunities that empower our team to realize their full potential and fulfill our purpose. You'll work with individuals who genuinely care about you, our clients, and our communities.

Our Information Technology team aims to be number one in leveraging technology for business strategies, connecting concepts and solutions to create value and efficiency for clients, staff, and communities. Our success stems from a skilled and diverse team passionate about excellence, innovation, and agility.

As a Principal Data Development and Engineering Expert, you play a vital role in data organization. You provide hands-on engineering leadership, overseeing the design and development of data products that collect and store large datasets, and directing the creation and implementation of data platforms that support and accelerate data product development.

You ensure data products are resilient, easily and securely accessible to teams. You verify data product compliance with data access policies and rules set by the governance team, and you drive the design and implementation of robust security measures to protect sensitive information. You are also tasked with showing staff how to use data tools and technologies and overseeing the launch of new tools. You assume end-to-end responsibility for the development, deployment, and maintenance of data products. You possess extensive experience and in-depth knowledge of Azure cloud infrastructure and automation principles, always prioritizing security.

Furthermore, you partner with business stakeholders, reporting specialists, and advanced analytics specialists to develop products that meet diverse stakeholder requirements. For data platform implementation, you also collaborate with network, security, data governance, and data engineering managers, as well as third parties like Microsoft and Databricks.

Finally, you contribute to the Co-operators data and cloud community by contributing to the cloud catalog.

How you create an impact:

• Design and implement cloud solutions for Master Data Management: Lead the technical strategy and development of the Master Data Management (MDM) platform, prioritizing cloud-native MDM. Plan capacity; create infrastructure, including data flows, tools, and network aspects like private endpoints and subnets.
• Develop data integration models: Create event-driven integration using Azure Event Grid/Service Bus (topics/queues) and implement idempotent consumers for Change Data Capture (CDC) and publish/subscribe patterns. Integrate MDM with Azure Data Lake to expose MDM data for downstream analytics, leveraging Data Lake as a source.
• Define entity/domain models and canonical schemas: Establish models for customer/party, product, vendor, location, etc., supporting reference records and hierarchies.
• Implement duplicate merging and survivorship strategies: Include deterministic and probabilistic match/merge, confidence scoring, rule creation, and auditability.
• Define and implement data models, survivorship rules, workflows, and integration patterns across all master data domains (e.g., customer, product, advisor, account data).
• Oversee migration of legacy on-premise MDM platforms to the modern cloud MDM environment, ensuring high data quality and minimal business disruption. Lead technical aspects of MDM migration in partnership with architects, cloud operations, and vendor teams.
• Continuous Integration/Continuous Deployment (CI/CD): Build and maintain robust CI/CD pipelines, including code quality and security analysis, and vulnerability testing, for automated deployment of data engineering pipelines across various environments.
• Security and compliance: Integrate security principles throughout development and deployment cycles, using vulnerability scanning, threat modeling, and compliance monitoring tools.
• Performance optimization: Define and implement an automated process to generate alerts on failures or performance degradation of data products, taking proactive measures to resolve issues before they become evident.
• Documentation and maintenance: Define, lead, and implement a documentation system to record the state of existing implementation and ensure continuous maintenance and updates.
• Process improvement: Identify and implement data engineering best practices and process improvements to enhance the efficiency and effectiveness of the data engineering team.
• Stakeholder communication: Collaborate with business and technology stakeholders to identify and prioritize customer-centric data initiatives that address business challenges and support value creation; ensure the data engineering team is aware of priorities, and communicate project objectives, progress, and results; work with security, governance, and technology stakeholders to identify and apply infrastructure platform code changes and ensure the data engineering team is aware of changes. Oversee and coordinate security and platform updates for underlying data infrastructure by collaborating with various data delivery teams and fostering a working group to share best practices within the Co-operators data engineering community.

How you shine:

You bring innovation to improve operational efficiency and possess the ability to influence change, prioritizing client needs. You demonstrate critical thinking to understand assumptions, evaluate arguments, draw conclusions, and proactively propose solutions. You have strong communication skills to convey messages clearly and consider diverse perspectives. You build trusting relationships and mentor colleagues to foster their development.

To join our team:

• At least 10 years of experience implementing multi-domain Master Data Management products using cloud MDM tools.
• Proven experience as a Principal Technical Lead driving complex cross-enterprise data initiatives.
• Hands-on experience with the following Azure Platform-as-a-Service (PaaS) platforms: Azure Data Lake Storage (ADLS), Azure Databricks, Azure Data Factory (ADF), Azure Synapse, Event Hub, API Management (APIM), Azure Key Vault, Azure SQL, and Purview.
• Good understanding of backup, disaster recovery, and data recovery strategies for these PaaS platforms.
• Excellent knowledge of Azure infrastructure, including cloud security and networking, and experience implementing best practices.
• Excellent project management skills with the ability to prioritize and manage multiple projects simultaneously, demonstrated verbal and written communication skills, and experience presenting to small and large audiences.

Key skills/competency:

• Azure Data Lake Storage (ADLS)
• Azure Databricks
• Azure Data Factory (ADF)
• Azure Synapse
• Azure Event Hub
• API Management (APIM)
• Azure Key Vault
• Azure SQL
• Azure Purview
• Master Data Management (MDM)
• CI/CD
• Cloud Security
• Data Governance
• Data Integration
• Stakeholder Management