Senior Data Engineer - Azure Data Factory (8-10 yrs)

Senior Data Engineer :

- Senior Data Engineer will be responsible for designing, implementing, and maintaining data solutions on SQL Server (SSIS, SSAS, UC4 Atomic)

- Collaborate with various stakeholders, and ensuring the efficient processing, storage, and retrieval of large volumes of data

Technical Expertise and Responsibilities :

- Should be able to design and build solutions in SSIS to extract, transform and load data into different source and target systems.

- Should be able to analyze and understand the existing data landscape and provide recommendations/innovative ideas for rearchitecting / optimizing / streamlining to bring efficiency and scalability.

- Must be able to collaborate and effectively communicate with onshore counterparts to address technical gaps, requirement challenges, and other complex scenarios.

- Monitor and troubleshoot data systems to ensure high performance and reliability.

- Should be highly analytical and detail-oriented with extensive familiarity with database management principles.

- Optimize data processes for speed and efficiency.

- Ensure the data architecture supports business requirements and data governance policies.

- Define and execute the data engineering strategy in alignment with the companys goals.

- Integrate data from various sources, ensuring data quality and consistency.

- Stay updated with emerging technologies and industry trends.

- Understand the big picture business process utilizing deep knowledge in banking industry and translate them to data requirements.

- Enabling and running data migrations across different databases and different servers

- Perform thorough testing and validation to support the accuracy of data transformations and data verification used in machine learning models.

- Analyze data and different systems to define data requirements.

- Define data mapping working along with business and digital team and data team.

- Data pipeline maintenance/testing/performance validation

- Assemble large, complex data sets that meet functional / non-functional business requirements.

- Analyze and identify gaps on data needs and work with business and IT to bring in alignment on data needs.

- Troubleshoot and resolve technical issues as they arise.

- Optimize data flow and collection for cross-functional teams.

- Work closely with Data counterparts at onshore, product owners, and business stakeholders to understand data needs and strategies.

- Collaborate with IT and DevOps teams to ensure data infrastructure aligns with overall IT architecture.

- Implement best practices for data security and privacy.

- Drive continuous improvement initiatives within the data engineering function

- Optimize data flow and collection for cross-functional teams.

- Understand impact of data conversions as they pertain to servicing operations.

- Manage higher volume and more complex cases with accuracy and efficiency.

What is the expectation from the candidates current role/profile?

- Should be proficient in SSIS, SQL and Query optimization.

- Expertise in SQL knowledge and experience working with relational databases.

- Should have worked in onshore offshore model managing challenging scenarios.

- Ability to work with ambiguity and vague requirements and transform them into deliverables.

- Good combination of technical and interpersonal skills with strong written and verbal communication; detail-oriented with the ability to work independently.

- Drive automation efforts across the data analytics team utilizing Infrastructure as Code (IaC) using Terraform, Configuration Management, and Continuous Integration (CI) / Continuous Delivery (CD) tools such as Jenkins.

- Help build define architecture frameworks, best practices & processes. Collaborate on Data warehouse architecture and technical design discussions.

- Expertise in Python will be of added advantage

- Experience in data bricks will be of added advantage.

- Should have expertise in data life cycle, data ingestion, transformation, data loading, validation, and performance tuning.