Data Catalog Engineer - AWS/DataLake/Data Warehousing (7-8 yrs)

Job Title : Data Catalog Engineer AWS Data Lake/Data Warehouse

Location : HDB Financial Services Limited. MBP Office Navi Mumbai.

Experience : 78 Years or More

Employment Type : Full-Time

Job Summary :

HDB Financial Services Limited is seeking an experienced and motivated Data Catalog Engineer to join our Data Engineering team. The ideal candidate will have extensive experience designing and implementing metadata and data catalog solutions in AWS-centric data lake/data warehouse environments. This role is pivotal in enhancing data discoverability, governance, and lineage across our enterprise data assets.

Key Responsibilities :

- Lead the end-to-end implementation of a data cataloging solution within AWS (preferably AWS Glue Data Catalog or third-party tools like Apache Atlas, Alation, Collibra, etc.).

- Establish and manage metadata frameworks for structured and unstructured data assets in the data lake and data warehouse

environments.

- Integrate the data catalog with AWS-based storage solutions such as S3, Redshift, Athena, Glue, and EMR.

- Collaborate with data Governance/BPRG/IT projects teams to define metadata standards, data classifications, and stewardship processes.

- Develop automation scripts for catalog ingestion, lineage tracking, and metadata updates using Python, Lambda, Pyspark or Glue/EMR customs jobs.

- Work closely with data engineers, data architects, and analysts to ensure metadata is accurate, relevant, and up to date.

- Implement role-based access controls and ensure compliance with data privacy and regulatory standards.

- Create detailed documentation and deliver training/workshops for internal stakeholders on using the data catalog.

Required Skills and Qualifications :

- 78+ years of experience in data engineering or metadata management roles.

- Proven expertise in implementing and managing data catalog solutions within AWS environments.

- Strong knowledge of AWS Glue, S3, Athena, Redshift, EMR, Data Catalog and Lake Formation.

- Hands-on experience with metadata ingestion, data lineage, and classification processes.

- Proficiency in Python, SQL, and automation scripting for metadata pipelines.

- Familiarity with data governance and compliance standards (e.g., GDPR, RBI guidelines).

- Experience integrating with BI tools (e.g., Tableau, Power BI) and third-party catalog tools is a plus.

- Strong communication,

- Problem-solving, and stakeholder management skills.

Preferred Qualifications :

- AWS Certifications (e.g., AWS Certified Data Analytics, AWS Solutions Architect).

- Experience with data catalog tools like Alation, Collibra, or Informatica EDC. Or open sources tools hand-on experience.

- Exposure to data quality frameworks and stewardship practices.

- Knowledge of data migration with data catalog and data-mart is plus

- Strong knowledge of AWS Glue, S3, Athena, Redshift, EMR, Data Catalog and Lake Formation.

- Hands-on experience with metadata ingestion, data lineage, and classification processes.

- Proficiency in Python, SQL, and automation scripting for metadata pipelines.

- Familiarity with data governance and compliance standards (e.g., GDPR, RBI guidelines).

- Experience integrating with BI tools (e.g., Tableau, Power BI) and third-party catalog tools is a plus.

- Strong communication,

- Problem-solving, and stakeholder management skills.

- AWS Certifications (e.g., AWS Certified Data Analytics, AWS Solutions Architect).

- Experience with data catalog tools like Alation, Collibra, or Informatica EDC. Or open sources tools hand-on experience.

- Exposure to data quality frameworks and stewardship practices.

- Knowledge of data migration with data catalog and data-mart is plus