Senior Data Engineer - ETL (3-5 yrs)

Job Title : Senior Data Engineer

Department : Data Engineering / IT

Reports To : Data Engineering Manager / Director of Data Engineering

Type : Full-time

Job Summary :

We are seeking a highly skilled Senior Data Engineer to design, build, and maintain scalable data pipelines and architectures. The ideal candidate will have extensive experience in data warehousing, ETL processes, and big data technologies. You will collaborate with data scientists, analysts, and other stakeholders to ensure data availability, reliability, and efficiency.

Key Responsibilities :

1. Data Pipeline Development :

- Design, build, and manage scalable ETL pipelines to process large datasets efficiently.

- Automate and optimize data ingestion from various sources.

2. Database and Data Warehouse Management :

- Develop and maintain data warehouses (e.g., Snowflake, Redshift, BigQuery).

- Optimize database performance and storage management.

3. Data Modeling :

- Design and implement data models for analytics and reporting purposes.

- Ensure data quality and consistency across different platforms.

4. Big Data Technologies :

- Work with big data tools such as Apache Spark, Hadoop, Kafka, etc.

- Manage and process data using distributed computing frameworks.

5. Collaboration and Support :

- Collaborate with data scientists, analysts, and software engineers to meet data needs.

- Provide technical guidance and mentorship to junior data engineers.

6. Security and Compliance :

- Implement data security practices to ensure compliance with industry standards.

- Manage data access policies and permissions.

7. Monitoring and Troubleshooting :

- Develop monitoring systems to track data pipeline performance.

- Troubleshoot and resolve data-related issues promptly.

Required Qualifications :

Education : Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

Experience : 5+ years of experience as a Data Engineer or similar role.

Technical Skills :

- Proficiency in SQL and at least one programming language (Python, Java, or Scala).

- Experience with cloud platforms (AWS, Azure, or GCP).

- Hands-on experience with ETL tools (Informatica, Talend, or Apache NiFi).

- Familiarity with containerization and orchestration tools (Docker, Kubernetes).

- Strong understanding of distributed systems and big data processing.

Soft Skills :

- Excellent problem-solving and analytical skills.

- Strong communication and teamwork abilities.

Preferred Qualifications :

- Experience with real-time data processing tools (Apache Kafka, Apache Flink).

- Knowledge of data governance and MDM (Master Data Management).

- Certification in cloud technologies (AWS Certified Big Data, Google Cloud Data Engineer).