Big Data Engineer

Qualification

• 4-5 years of good hands-on exposure with Big Data technologies Pyspark (Data frame and Spark SQL), Hadoop, and Hive
• Good hands-on experience of python and Bash Scripts
• Hands-on experience with using Cloud Platform provided Big Data technologies
• Good understanding of SQL and data warehouse tools like (Redshift)
• Strong analytical, problem-solving, data analysis and research skills
• Demonstrable ability to think outside of the box and not be dependent on readily available tools
• Excellent communication, presentation and interpersonal skills are a must

Good to have:

• Orchestration with Airflow and Any job scheduler experience
• Experience in migrating workload from on-premises to cloud and cloud to cloud migrations

Roles & Responsibilities

• Develop efficient ETL pipelines as per business requirements, following the development standards and best practices.
• Perform integration testing of different created pipeline in AWS/Azure env.
• Provide estimates for development, testing & deployments on different env.
• Participate in code peer reviews to ensure our applications comply with best practices.