Data Engineer

Job Summary

We are seeking a highly skilled and motivated Data Engineer with strong expertise in Python and PySpark to join our growing data team. You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure that power our analytics and machine learning initiatives.

Years of experience needed –

5+ years of experience in data engineering or a related field.

Technical Skills:

• Design and implement robust, scalable, and high-performance data pipelines using PySpark and Python.
• Develop ETL/ELT workflows to ingest, transform, and load data from various sources (e.g., APIs, databases, cloud storage).
• Optimize data processing jobs for performance and cost-efficiency in distributed environments.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Ensure data quality, integrity, and governance across all pipelines.
• Monitor and troubleshoot production data workflows and resolve issues proactively.
• Leverage Azure for data storage, processing, and orchestration
• Strong proficiency in Python for data manipulation and scripting.
• Hands-on experience with PySpark and distributed data processing.
• Solid understanding of SQL and relational databases.
• Familiarity with data modeling, data warehousing, and performance tuning.
• Experience with version control (e.g., Git) and CI/CD pipeline