Big Data Engineer - Hadoop/Spark (8-10 yrs)

Primary Skill : Hadoop, Spark, Hive, ETL, Pyspark, Python.

Secondary Skill : Integration and continuous deployment (CI/CD) pipelines.

Experience : 8+ Years.

Location : Bangalore / Chennai (Second Round Face to Face).

Notice Period Immediate Joiner.

We are currently looking for skilled Data Engineer to join our team.

Below are the details of our requirements : .

Job Title : Big Data Engineer.

- Experience with big data technologies Hadoop, Spark, Hive, ETL, Pyspark.

- Proven experience as a development data engineer or similar role, with ETL background.

- Experience with data integration / ETL best practices and data quality principles.

- Play a crucial role in ensuring the quality and reliability of the data by designing, implementing, and executing comprehensive testing.

- By going over the User Stories build the comprehensive code base and business rules for testing and validation of the data.

- Knowledge of continuous integration and continuous deployment (CI/CD) pipelines.

- Familiarity with Agile/Scrum development methodologies.

- Excellent analytical and problem-solving skills.

- Strong communication and collaboration skills.

Job Responsibilities :

- Develop and maintain data pipelines using ETL processes.

- Take responsibility for Apache Hadoop development and implementation.

- Work closely with data science team to implement data analytics pipelines.

- Help define data governance policies and support data-versioning processes.

- Maintain security and data privacy, working closely with data protection officer.

- Analyze vast number of data stores to uncover insights.