Senior Data Engineer

Senior Data Engineer at OpenSky Data Systems

This is a full-time hybrid role for a Senior Data Engineer in our Data & AI Practice at OpenSky Data Systems in Naas, Co. Kildare. You will be at the forefront of technological development, working with the latest Microsoft Data & AI technologies, including Fabric, Azure OpenAI, AI Studio, and Cognitive Services. OpenSky Data Systems collaborates closely with Microsoft to ensure staff and clients utilize the most modern technology stacks.

We are seeking a proactive individual who enjoys evaluating new technologies, innovating, educating, and collaborating to deliver critical services that empower our clients. The ideal candidate will possess practical experience across various Data Warehouse technologies.

About OpenSky Data Systems' Data & AI Practice

OpenSky Data Systems boasts a 20-year legacy of delivering technology innovation to customers across Ireland, the UK, and Europe. Our multi-disciplinary teams operate within three Centres of Excellence: Modern Applications (.NET), Microsoft Cloud (Power Platform), and Data & AI. We undertake projects with significant impact, from reducing HPV cancer incidence in Europe to addressing homelessness in Ireland.

Our Data & AI Practice is a leader in Microsoft solutions, partnered with prestigious AI institutes like CeADAR (Ireland’s Centre for AI) and the Alan Turing Institute (UK). We commit substantial annual investment to research and training, ensuring our team's skills remain cutting-edge for delivering consistent excellence to our customers.

Key Responsibilities

• Client Consultation: Provide expert consultation on data engineering strategies, offering creative and innovative solutions.
• Data Pipeline Development: Design, develop, and maintain scalable data pipelines using Microsoft Azure Synapse Analytics and Microsoft Fabric.
• Database Systems & ETL: Build, maintain, and optimize database systems and ETL pipelines with tools like Azure Data Factory (ADF) and SQL Server Integration Services (SSIS).
• Data Warehousing: Design and maintain star schema data models, reusable pipelines, and warehouse objects.
• Data Quality & MDM: Implement master data management solutions, applying best practices for data profiling, cleansing, and standardization.
• DevOps & CI/CD: Utilize Azure DevOps, GitHub, and CI/CD workflows for managing deployments.
• Data Governance: Implement data governance and cataloguing solutions using Microsoft Purview, adhering to Microsoft best practices and methodologies, ensuring compliance with GDPR, NIS2, and ISO27001 standards.
• BI Reporting & Data Visualization: Create, update, and maintain interactive dashboards and reports using Power BI, SSRS, and Excel to provide actionable business insights.
• Machine Learning: Apply machine learning techniques for data preparation and analysis.
• Team Leading: Lead and mentor team members, supporting their development and ensuring successful project delivery.

Requirements

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience in a Senior Data Engineering or similar role, with practical experience in Azure Synapse Analytics for data ingestion, ETL/ELT, and data integration.
• Strong experience in client engagement, acting as a technical advisor and providing recommendations.
• Proven ability to manage multiple projects, meet deadlines, and work effectively within a team.
• Strong problem-solving skills, attention to detail, and excellent communication abilities for both technical and non-technical audiences.
• Solid understanding of AI and machine learning concepts, with exposure to applying AI techniques or understanding their impact on data projects.
• Expertise in data wrangling and data preparation tools and techniques.
• Proficiency in Microsoft SQL Server and T-SQL scripting.
• Creative thinker with an ability to innovate and provide strategic insights.

Desirable Skills

• Experience in team leading and mentorship.
• Microsoft Certifications: Azure AI Engineer Associate, Power BI Data Analyst Associate.
• Expertise in Microsoft Purview & Microsoft Cloud Adoption Framework.
• Knowledge of Azure AI services.
• Expertise in Microsoft Azure Synapse Analytics and Power BI.
• Proficient in Azure DevOps, GitHub, and CI/CD processes.
• Advanced knowledge of Azure Data Factory, SSIS, and Azure Data Storage (Blobs, Files, Queues, Tables, Disks, Data Lake Storage).

What OpenSky Data Systems Offers

You will join a high-performance team dedicated to delivering projects of national and international significance, impacting how large businesses and governments operate.

• Competitive salary
• Incremental increases in annual leave
• Continuous Professional Development
• Education Assistance Programme
• Digital doctor service
• Hybrid working
• Employee Assistance Programme (EAP)

Key skills/competency

• Data Engineering
• Azure Synapse Analytics
• Microsoft Fabric
• ETL/ELT
• SQL Server
• Power BI
• Data Governance
• Azure DevOps
• Machine Learning
• Client Consultation