Lead Data Engineer - Python Programming (6-15 yrs)

Job Summary:.

We are seeking a highly skilled Associate Architect / Lead Data Engineer / Sr. Data Engineer to lead the design, implementation, and optimization of scalable data architectures.

The ideal candidate will have a deep understanding of data modeling, ETL processes, cloud data solutions, and big data technologies.

You will work closely with cross-functional teams to build robust, high-performance data pipelines and infrastructure to enable data-driven decision-making of Positions: Associate Architect 1; Lead Data Engineer 2; Sr.

Mandatory Skills : AWS, Python, SQL, Airflow, DBT.

Responsibilities :

- Design and Develop scalable and resilient data architectures that support business needs, analytics, and AI/ML workloads.

- Data Pipeline Development : Design and implement robust ETL/ELT processes to ensure efficient data ingestion, transformation, and storage.

- Big Data & Cloud Solutions : Architect data solutions using cloud platforms like AWS, Azure, or GCP, leveraging services such as Snowflake, Redshift, BigQuery, and Databricks.

- Database Optimization : Ensure performance tuning, indexing strategies, and query optimization for relational and NoSQL databases.

- Data Governance & Security : Implement best practices for data quality, metadata management, compliance (GDPR, CCPA), and security.

- Collaboration & Leadership : Work closely with data engineers, analysts, and business stakeholders to translate business requirements into scalable solutions.

- Technology Evaluation : Stay updated with emerging trends, assess new tools and frameworks, and drive innovation in data engineering.

Required Skills :

- Education : Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

- Experience : 6 15 years of experience in data engineering.

- Cloud Platforms : Strong expertise in AWS data services.

- Big Data Technologies : Experience with Hadoop, Spark, Kafka, and related frameworks.

- Databases : Hands-on experience with SQL, NoSQL, and columnar databases such as PostgreSQL, MongoDB, Cassandra, and Snowflake.

- Programming : Proficiency in Python.

- ETL Tools : Experience with tools like Apache Airflow, Talend, DBT, or Informatica.

- Machine Learning & AI Integration (Preferred) : Understanding of how to architect data solutions for AI/ML applications