Data Engineer

Role & responsibilities

Description:

As a Senior Data Engineer, you manage and develop the solutions in close alignment with various business and Spoke stakeholders. You are responsible for the implementation of the IT governance guidelines.

Collaborate with the Spokes Data Scientists, Data Analysts, and Business Analysts, when relevant.

Tasks

• Create and manage data pipeline architecture for data ingestion, pipeline setup and data curation
• Experience working with and creating cloud data solutions
• Assemble large, complex data sets that meet functional/non-functional business requirements
• Implement the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Pyspark, SQL and AWS big data-technologies
• Build analytics tools that use the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics
• Manipulate data at scale: getting data in a ready-to-use state in close alignment with various business and Spoke stakeholders

Preferred candidate profile

Hard skills

advanced knowledge:

• ETL
• Data Lake, Data Warehouse, RDS architectures knowledge
• Python, SQL (Any other OOP language is also valuable)
• Pyspark (preferably) or Spark Knowledge
• Object-oriented programming, Clean Code and good documentation skills
• AWS: S3, Athena, Lambda, Glue, IAM, SQS, EC2, Quicksight, and etc.
• Git
• Data Analysis & Visualization

Optional:

• AWS CDK Cloud Development Kit
• CI/CD knowledge