Job Overview
Job TitleSenior Data Engineer
Job TypeFull Time
Offered Salary₹0
LocationKurla, Maharashtra, India
Who's the hiring manager?
Sign up to PitchMeAI to discover the hiring manager's details for this job. We will also write them an intro email for you.

Job Description
Senior Data Engineer
Crisil is seeking a highly experienced Senior Data Engineer with 7 to 10 years of experience and a strong understanding of PySpark, SQL, Oracle, and PostgreSQL databases, along with proven data management expertise. This role is critical in implementing robust data architectures and driving data-driven decision-making.
Key Responsibilities
- Implement Data Architecture: Design and implement scalable, secure, and efficient data architectures on-prem and cloud platforms (Azure/GCP/AWS). Collaborate with cross-functional teams to define data requirements and develop data models.
- Data Ingestion and Integration: Develop and maintain data ingestion pipelines using PySpark, Kafka, and Flume. Integrate data from diverse sources including relational databases, NoSQL databases, APIs, and files.
- Batch and Stream Processing: Build and manage batch and stream processing pipelines using Apache Spark, Flink, and Beam. Integrate with messaging systems like Kafka, Kinesis, and Google Cloud Pub/Sub.
- SQL Proficiency: Utilize very strong SQL knowledge for query optimization, indexing, and database design.
- Delta Lake and Data Warehouse: Design and implement Delta Lake and data warehouse/mart solutions for business intelligence and analytics. Develop pipelines for data ingestion, processing, and storage.
- Distributed Databases and Data Warehousing: Implement and maintain data warehouses such as Amazon Redshift, Google BigQuery, and Azure Synapse Analytics.
- Database Design and Development: Design, develop, and maintain scalable relational database systems (Oracle, MySQL, PostgreSQL, SQL Server). Translate business requirements into effective database solutions and optimize queries.
- Performance Tuning and Optimization: Analyze and monitor database performance, identify and resolve bottlenecks. Optimize data processing workflows and queries for improved performance and reduced latency.
- Data Management: Implement data archival and retention policies. Ensure data security and integrity through access controls, encryption, and backup strategies.
- Automation and Integration: Identify and implement automation solutions for data workflows. Integrate database solutions into software applications.
- Data Mart and Data Lake: Design and implement data marts and data lakes for BI, reporting, and analytics. Develop pipelines for data lakes like Hadoop, Amazon S3, and Azure Data Lake Storage.
- CI/CD and Automation: Develop and maintain automated testing, deployment, and monitoring scripts using Jenkins, GitLab CI/CD, or similar tools. Ensure continuous integration and delivery.
- Data Analysis and Modeling: Perform data analysis, including data modeling, mining, and visualization. Collaborate to develop models driving business insights.
- Exploration of New Tools: Explore new tools and technologies, and develop proofs-of-concept for open-source data engineering tools.
- Documentation: Document database designs, configurations, and technical specifications.
Requirements
- Bachelor's degree in Computer Science, Information Technology, or a related field. Relevant certifications are a plus.
- 7 to 10 years of experience in data engineering, with a focus on cloud-based data architectures (Azure/GCP/AWS).
- Strong proficiency in SQL and experience with relational and NoSQL databases.
- In-depth knowledge of database performance optimization techniques.
- Familiarity with data archival mechanisms and retention strategies.
- Solid understanding of database security principles.
- Experience with database administration and monitoring tools.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration skills.
- Detail-oriented mindset.
- Ability to adapt to changing priorities and manage multiple projects.
Key skills/competency
- Data Engineering
- PySpark
- SQL
- Oracle
- PostgreSQL
- Data Management
- Cloud Platforms (Azure/GCP/AWS)
- Data Warehousing
- Data Modeling
- Performance Tuning
How to Get Hired at Crisil
- Tailor your resume: Highlight your 7-10 years of experience in data engineering, focusing on PySpark, SQL, Oracle, PostgreSQL, and cloud platforms like Azure/GCP/AWS.
- Showcase cloud architecture expertise: Emphasize your experience in implementing scalable and secure data architectures on Azure, GCP, or AWS.
- Demonstrate SQL and database skills: Provide specific examples of your strong SQL knowledge, including query optimization and experience with relational and NoSQL databases.
- Quantify achievements: Use data and metrics to illustrate the impact of your work in areas like performance tuning, data management, and CI/CD implementation.
Frequently Asked Questions
Find answers to common questions about this job opportunity
01What are the primary cloud platforms mentioned for the Senior Data Engineer role at Crisil?
02What specific database technologies are crucial for this Senior Data Engineer position?
03What level of experience is required for the Senior Data Engineer at Crisil?
04Does Crisil require specific certifications for the Senior Data Engineer role?
05What are the expectations for data processing methods in this Senior Data Engineer role?
Explore similar opportunities that match your background