Data Quality Lead - Numpy/Pandas (6-8 yrs)

Required Skills :

- 7-8 years of experience in Data Engineering, Data Quality and Data Validation.

- Hands-on experience in Python and libraries like Pandas, Spark, NumPy.

- Working experience with relational and NOSQL databases.

- Strong understanding of Data Warehouse/Data Lake Architecture and Development and data pipelines.

- Experience with data processing concepts like mapping documents and complex data relationships.

- Experience with requirement analysis, defect tracking, coordinating with team members in different locations, and test reporting and signoff.

- Strong analytical and technical skills to address issues.

- Detailed and organized approach to solving problems.

Responsibilities :

- Collaborate with cross-functional teams to ensure quality and timely delivery of products.

- Identify areas of improvement in data quality processes and propose solutions to enhance data accuracy and reliability and create test strategies.

- Collaborate with the Data Delivery technology teams and architects to define and develop solutions.

- Collaborate with data engineering and development teams to implement data quality best practices and optimize data workflows.

- Develop and execute Data Quality frameworks, solutions and automated test cases for software applications to ensure the reliability of data pipelines and data transformations.

- End to end integration testing between multiple independent systems and interfaces (flat files, APIs (ETL)) etc.

- Utilize AWS services (EC2, Glue, Athena etc. to test data integration and migration processes.

- Conduct functional, integration, regression, and performance testing of database systems using industry standard technologies (SQL, NOSQL, Python, etc.)

- Create and maintain documentation of test plans, test cases and testing results, data quality issues, testing procedures, and resolutions.