Senior Data Engineer - Python/PySpark (6-12 yrs)

Sr. Python Data Engineer

Experience : 6 to 12 Years

Skills : Python, Pyspark, Data warehousing, Data pipeline design

Shift hours : 2.00 - 11.00pm IST

Work location : Bangalore or Chennai

Responsibilities :

- Design and develop data pipelines and ETL processes.

- Collaborate with data scientists and analysts to understand data needs.

- Maintain and optimize data warehousing solutions.

- Ensure data quality and integrity throughout the data lifecycle.

- Develop and implement data validation and cleansing routines.

- Work with large datasets from various sources.

- Automate repetitive data tasks and processes.

- Monitor data systems and troubleshoot issues as they arise.

Qualifications :

- Bachelors degree in Computer Science, Information Technology, or a related field.

- Proven experience as a Data Engineer or similar role (Minimum 6+ years experience as Data Engineer).

- Strong proficiency in Python and PySpark.

- Excellent problem-solving abilities.

- Strong communication skills to collaborate with team members and stakeholders.

- Individual Contributor.

Technical Skills Required :

- Python, PySpark and SQL/Snowflake.

- Data warehousing, Data pipeline design.

- Data Quality, Data validation, Data cleansing.

- Microsoft Fabric, ADF, Databricks, Master Data management/Data Governance, Data Mesh, Data Lake/Lakehouse Architecture.