Data Engineer - IICS

Job Description:

As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and ETL processes using Informatica Intelligent Cloud Services (IICS), Informatica PowerCenter, and Azure Data Factory (ADF). You will also leverage Databricks to enhance our data processing capabilities and perform advanced analytics. Additionally, you will be involved in production support activities to ensure the smooth operation of data systems. Your expertise will be crucial in ensuring the integrity, availability, and performance of our data systems.

Key Responsibilities:

• Design, develop, and implement ETL solutions using IICS-CDI to extract, transform, and load data from various sources into the data warehouse.
• Utilize PowerCenter Designer and IICS Cloud Console for ETL development.
• Develop and maintain ETL mappings, workflows, and schedules using IICS-CDI.
• Collaborate with data architects and analysts to understand data requirements and translate them into technical specifications for ETL development.
• Ensure data quality, integrity, and consistency by performing data validation, cleansing, and enrichment activities.
• Utilize Azure Data Factory (ADF) for orchestrating data workflows and integrating data from cloud and on-premises sources.
• Work with Databricks to process large datasets, build data pipelines, and perform advanced analytics using Apache Spark.
• Monitor ETL jobs to ensure successful completion, identify and resolve errors and performance issues.
• Provide production support for data pipelines and ETL processes, troubleshooting issues and ensuring timely resolution.
• Collaborate with data architects and analysts to understand data requirements and translate them into technical specifications for ETL development.
• Ensure data quality, integrity, and consistency by performing data validation, cleansing, and enrichment activities.
• Conduct performance tuning and optimization of ETL processes and data pipelines to enhance efficiency and reduce processing time.
• Optimize and maintain existing data pipelines for performance and reliability.
• Stay updated with the latest trends and technologies in data engineering and cloud computing.

Qualifications:

• Minimum of 4+ years of experience in ETL development using IICS-CDI, including experience with IICS Cloud Console and PowerCenter Designer.
• Solid experience with SQL and database technologies such as SQL Server, Oracle, and preferably Azure Synapse.
• Experience with scripting languages such as Python or Shell scripting is a plus.
• Strong understanding of data warehousing concepts, ETL frameworks, and best practices.
• Familiarity with data modeling and data integration techniques.
• Familiarity with cloud technologies such as Azure, AWS, or Google Cloud Platform.
• Excellent analytical and problem-solving skills, with a keen attention to detail.
• Strong communication and interpersonal skills.
• Familiarity with Agile development methodologies is a plus.