Big Data Platform Engineer

About the Role: Big Data Platform Engineer at Veridic Solutions

Veridic Solutions, a client of Dice, is seeking a Big Data Platform Engineer to join their team. This role involves working closely with application and development teams to understand project requirements and provide accurate estimates for Big Data initiatives.

Key Responsibilities

• Work directly with application and development teams to completely understand all requirements for project and ad hoc requests and provide accurate estimates.
• Participate in a team of technical subject matter experts that provides 24x7 operational support of mission-critical Big Data environments for highly-available and contingent platforms.
• Create and maintain documentation and define department best practices in support of proper change management, security and operational requirements.
• Investigate and understand new technologies and additional capabilities of current platforms that support MasterCard's vision of Big Data, leading their adoption when appropriate.
• Educate peers and cross-train teammates with Big Data concepts and application of technology to serve business needs.
• Interact with managed services resources and direct maintenance and operational support activities per MasterCard standards and industry best-practices.

About You

The ideal candidate will possess a strong background in Big Data platforms and systems administration, coupled with excellent troubleshooting and programming skills. Communication is key to success in this role.

• Strong knowledge of Hadoop platforms: Expertise in Hadoop and other distributed data processing platforms is essential.
• Linux/Unix and system administration: Advanced Linux knowledge is a must, including shell scripting and debugging capabilities.
• Mission-critical Big Data project experience: Demonstrated experience delivering projects in a highly available, contingent environment.
• Troubleshooting: Proven ability to solve complex programming and system problems.
• Programming: Proficiency in at least one programming language (Java preferred), with the ability to learn new languages. Experience with infrastructure management or automation tools is a significant advantage.
• Communication skills: Excellent written and verbal communication, capable of articulating thoughts clearly and concisely up to executive level.

Desirable or Additional Capabilities

Candidates with the following skills and experiences will be highly valued:

• Knowledge and hands-on experience with Kubernetes, as well as automation using tools such as Ansible and Chef.
• Understanding of and experience with network engineering in support of large IT organizations.
• Knowledge in statistics and machine learning.
• Background in ETL and practical experience dealing with large ETL pipelines.
• Basic SQL knowledge is a must; advanced data warehousing and MPP knowledge is a plus.

Key skills/competency

• Big Data Platforms
• Hadoop
• Linux/Unix System Administration
• Troubleshooting
• Java Programming
• Distributed Data Processing
• ETL Pipelines
• Kubernetes
• Ansible/Chef Automation
• SQL Databases