Idexcel Technologies - Data Engineer - Python/PySpark (5-8 yrs)

Data Engineer

Job Description :

- Candidate will join our project and Advance Analytics project.

- He will be working on Multiple Data Integration projects.

- He will be working on creating Data Lake in open table format.

- He will involve in creating Realtime advance reporting platform and its Datawarehouse.

- He will be involved in building Data pipeline, Lambdas, Data Solution Architecting,

- Deployment of code, Automation, Code review.

- He should be self-motivated person to led technical upgradation, schema redesigning.

- Keep himself updated on Data engineering developments both opensource and AWS.

Required Skills :

- BE or BTech with good Communication and presentation skill.

- AWS full DE stack,

- Experience in building Realtime Data analytic solution on AWS is must.

- Experience in core Banking Industry is a Plus,

- Experience in developing Analytical solutions for Finance function.

- Knowledge of Big Data Solutions such as Hadoop, NoSQL, MapReduce etc

- Good Knowledge of agile Development like Scrum and Sprint planning. CI/CD

- Working Knowledge of API consumption.

- Data Integration with external tools like Saleforce, SAP, Codata is a plus.

- Candidates with AWS Certification will be preferred.

Tools : Aurora Postgres DB, SQL, Spark, Kafka, Python, Redshift, Snowflake, Airflow, Glue (ETL, Catalog, Crawler), Lambda, SQS, SNS, Data Lake, Cloud watch, Cloud Trail, DBT, AWS SDK, Boto3, Kinesis, kinesis Firehose, Everbridge, Code pipeline. MongoDB, Lake formation, Cloud Formation, Git Branching, code reviews,

Optional Skills : Terraform, Docker, Kubernetes