Senior Data Engineer - Azure Data Services (6-8 yrs)

Job Summary :

We are seeking an experienced and results-driven Data Engineer to design, develop, and maintain robust, scalable, and efficient data pipelines and architectures using Azure Data Services, Databricks, and PySpark. You will collaborate with cross-functional teams to support data-driven decision-making across the organization while ensuring data quality, governance, and optimal performance.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines and ETL processes using Databricks, PySpark, and Azure Data Factory.

- Utilize Databricks for big data analytics and real-time data processing with a focus on performance and efficiency.

- Collaborate with data scientists, analysts, architects, and business stakeholders to understand data requirements and translate them into effective data solutions.

- Work closely with cloud platforms (Azure or AWS) to deploy and manage data solutions in a scalable and reliable environment.

- Implement and maintain data quality checks, data governance practices, and data security standards to ensure accuracy and compliance.

- Monitor, troubleshoot, and optimize existing pipelines for performance, scalability, and minimal downtime.

- Design and implement effective data models and data warehouse architectures to support business analytics and reporting needs.

- Conduct code reviews, enforce best practices in data engineering, and maintain comprehensive documentation of processes and configurations.

- Mentor junior data engineers, fostering knowledge sharing and skills development within the team.

- Stay current with the latest trends and advancements in data engineering, Databricks, cloud technologies, and big data frameworks

Required Qualifications :

Education : Bachelors or Masters degree in Computer Science, Information Systems, or a related technical field.

Experience : Minimum of 6 years of hands-on experience in Data Engineering, with a strong focus on Azure-based data solutions and Databricks.

Technical Skills :

- Proficiency in Azure Data Factory, Azure Databricks, SQL, Python, and ETL development.

- Solid understanding of data modelling, data warehousing, and big data concepts.

- Experience integrating data from multiple sources with a focus on accuracy and governance.

- Hands-on experience with real-time data processing and performance tuning of data pipelines.

Preferred Skills :

- Experience working with AWS cloud services (optional but beneficial).

- Knowledge of CI/CD pipelines for data deployments.

- Familiarity with Delta Lake, Apache Spark, and data orchestration frameworks.

Why Join Us?

- Work on cutting-edge big data solutions with modern cloud-native technologies.

- Collaborate in a high-performing, agile, and innovative team.

- Competitive salary, benefits, and opportunities for professional growth.