GenAI Data Automation Engineer

GenAI Data Automation Engineer at Protos IT

Protos IT is seeking a GenAI Data Automation Engineer to spearhead the design and implementation of innovative, AI-driven automation solutions across hybrid AWS and Azure environments. This role is crucial for building intelligent, scalable data pipelines and automations that seamlessly integrate cloud services, enterprise tools, and Generative AI to bolster mission-critical analytics, reporting, and customer engagement platforms. The ideal candidate is highly mission-focused, delivery-oriented, and applies critical thinking to create innovative functions and resolve complex technical challenges.

Key Responsibilities

• Design and maintain robust data pipelines in AWS, utilizing services such as S3, RDS/SQL Server, Glue, Lambda, EMR, DynamoDB, and Step Functions.
• Develop efficient ETL/ELT processes for data movement across various systems, including DynamoDB, SQL Server (AWS), and between AWS and Azure SQL environments.
• Integrate data from AWS Connect and Nice inContact CRM into the enterprise data pipeline to support comprehensive analytics and operational reporting.
• Engineer and enhance data ingestion pipelines with technologies like Apache Spark, Flume, and Kafka for real-time and batch processing into Apache Solr and AWS Open Search platforms.
• Leverage Generative AI services and frameworks (AWS Bedrock, Amazon Q, Azure OpenAI, Hugging Face, LangChain) to:
  • Automate vector generation and embedding from unstructured data for Generative AI models.
  • Implement automated data quality checks, metadata tagging, and lineage tracking.
  • Enhance ingestion/ETL processes with LLM-assisted transformation and anomaly detection.
  • Construct conversational BI interfaces for natural language access to Solr and SQL data.
  • Develop AI-powered copilots to monitor pipelines and automate troubleshooting.
• Implement SQL Server stored procedures, indexing, query optimization, profiling, and execution plan tuning to maximize performance.
• Apply CI/CD best practices using GitHub, Jenkins, or Azure DevOps for both data pipelines and GenAI model integration.
• Ensure stringent security and compliance measures through IAM, KMS encryption, VPC isolation, RBAC, and firewalls.
• Support Agile DevOps processes, delivering pipeline and AI-enabled features in sprint-based cycles.

Required Qualifications

• Bachelor's degree in Computer Science or a related field, coupled with 2+ years of hands-on data engineering and automation experience.
• Demonstrated experience with LLM and Generative AI frameworks, specifically AWS Bedrock, Azure OpenAI, or other open-source platforms.
• Proficiency in SQL, SSIS, Python, Spark, Bash, PowerShell, and AWS/Azure CLIs.
• Practical experience with AWS services including S3, RDS/SQL Server, Glue, Lambda, EMR, and DynamoDB.
• Familiarity with Apache Flume, Kafka, and Solr for large-scale data ingestion and search capabilities.
• Experience integrating REST API calls into data pipelines and workflows.
• Familiarity with JIRA, GitHub / Azure DevOps / Jenkins for SDLC and CI/CD automation.
• Strong troubleshooting and performance optimization skills across SQL, Spark, and other data engineering solutions.
• Experience operationalizing Generative AI (GenAI Ops) pipelines, including model deployment, monitoring, retraining, and lifecycle management for LLMs and AI-enabled data workflows.
• Excellent communication and presentation skills.

Key skills/competency

• Generative AI
• Data Automation
• AWS
• Azure
• Data Pipelines
• ETL/ELT
• LLM Frameworks
• SQL Server Optimization
• CI/CD
• Apache Spark