SRE - Hadoop

JOB DESCRIPTION

• Minimum of 1 year of experience in Linux/Unix Administration.
• Minimum of 2 years of hands on experience with managing infra on public cloud i.e Azure/AWS/GCP
• Over 4+ years of experience in Hadoop administration.
• Strong understanding of networking, open-source technologies, and tools.
• Familiar with best practices and IT operations for maintaining always-up, always-available services.
• Experience and participation during on-call rotation.
• Excellent communication skills.
• Solid expertise in Linux networking, including IP, iptables, and IPsec.Proficient in scripting and coding with languages such as Perl, Golang, or Python.
• Strong Knowledge of databases like Mysql,Nosql,Sql serverHand on experience with setting up , configuring and Managing Nginx as reverse proxy and load balancing in high traffic environments.
• Hands-on experience with both private and public cloud environments. Strong troubleshooting skills and operational expertise in areas such as system capacity, bottlenecks, memory, CPU, OS, storage, and networking.
• Practical experience with the Hadoop stack, including Hdfs,HBase,Hive, Pig, Airflow, YARN, HDFS, Ranger, Kafka, and Druid.
• Good to have experience with Design,develop and maintain Airflow DAGs and tasks to automate BAU processes,ensuring they are robust,scalable and efficient.
• Good to have experience with ELK stack administration.
• Experience in administering Kerberos and LDAP. Familiarity with open-source configuration management and deployment tools like Puppet, Salt, or Ansible.Responsible for the implementation and ongoing administration of Hadoop infrastructure.
• Experience in capacity planning and performance tuning of Hadoop clusters. Collaborate effectively with infrastructure, network, database, application, and business intelligence teams to ensure high data quality and availability.
• Develop tools and services to enhance debuggability and supportability.Work closely with security teams to apply Hadoop updates, OS patches, and version upgrades as needed.
• Troubleshoot complex production issues, identify root causes, and provide mitigation strategies.
• Work closely with teams to optimize the overall performance of the PhonePe Hadoop ecosystem.
• Experience with setting up & managing monitoring stack like OpenTsdb,Prometheus,ELK,Grafana,Loki