Data Engineer - ETL (2-4 yrs)

Client : A fast growing data-science company, serving clients across industries and geographies.

Position : Data Engineer (Streaming Data Ingestion)

Experience : 2 to 4 years of data engineering experience

Location : Noida

Work Mode : In Office only

Interview Process : 3 Tech rounds, Fitment round, HR round.

Joining : Immediate to 15 days

Job Description :

Responsibilities :

a) Design, develop, and maintain scalable data pipelines for streaming data ingestion.

b) Implement and manage data processing frameworks using Apache Flink and Kafka.

c) Develop and maintain ETL processes using Python and Apache Airflow.

d) Optimize data storage and retrieval using file formats such as Parquet.

e) Implement data partitioning and bucketing strategies using Hive.

f) Collaborate with cross-functional teams to understand data requirements and deliver solutions.

g) Ensure data quality and integrity through CI/CD processes.

h) Manage cloud infrastructure using AWS and Infrastructure as Code (IaC) tools like Terraform.

Requirements :

a) 2 to 4 years of hands-on experience in data engineering.

b) Proficiency in AWS services and cloud computing.

c) Strong programming skills in Python.

d) Experience with streaming technologies such as Apache Flink and Kafka.

e) Knowledge of workflow orchestration tools like Apache Airflow.

f) Familiarity with file formats, especially Parquet.

g) Basic understanding of Infrastructure as Code (IaC)

h) Experience with data partitioning and bucketing in Hive.

i) Understanding of CI/CD processes and tools.

j) Excellent problem-solving skills and attention to detail.

k) Strong communication and collaboration skills.

Preferred :

a) Experience with other cloud platforms (e.g., GCP, Azure).

b) Knowledge of additional data processing frameworks.

c) Familiarity with data governance and security best practices.

Educational Qualifications :

- BE/BTech from a Tier1/Tier2 college (Preferred Computer Science major or related field)