Aws Data Engineer

Need Immediate Joiners

Responsibilities:

• Defines, designs, develops and test software components/applications using AWS - (Databricks on AWS, AWS Glue, Amazon S3, AWS Lambda, Amazon Redshift, AWS Secrets Manager)
• Strong SQL skills with experience.
• Experience handling Structured and unstructured datasets.
• Experience in Data Modeling and Advanced SQL techniques.
• Experience implementing AWS Glue, Airflow, or any other data orchestration tool using latest technologies and techniques.

• Good exposure in Application Development.

• The candidate should work independently with minimal supervision.

Must Have:

• Hands-on experience with distributed computing framework like Databricks, Spark-Ecosystem (Spark Core, PySpark, Spark Streaming, SparkSQL)
• Willing to work with product teams to best optimize product features/functions.
• Experience on Batch workloads and real-time streaming with high volume data frequency
• Performance optimization on Spark workloads

• Environment setup, user management, Authentication and cluster management on Databricks

• Professional curiosity and the ability to enable yourself in new technologies and tasks.
• Good understanding of SQL and a good grasp of relational and analytical database management theory and practice.

Key Skills:

• Python, SQL and PySpark

• Big Data Ecosystem (Hadoop, Hive, Sqoop, HDFS, HBase)

• Spark Ecosystem (Spark Core, Spark Streaming, Spark SQL) / Databricks

• AWS (AWS Glue, Databricks on AWS, Lambda, Amazon Redshift, Amazon S3, AWS Secrets Manager)

• Data Modelling, ETL Methodology