Data Engineer - ETL/Data Warehousing (3-5 yrs)

Job Description :

(Only Immediate Joiners) / (Remote -Hybrid Opportunity)

We are looking for a skilled Data Engineer to design, develop, and optimize scalable data pipelines and architectures. The ideal candidate will have expertise in ETL processes, data modelling, and cloud-based data platforms. You will work closely with data analysts, software engineers, and business teams to ensure high-quality data delivery and efficient data processing solutions.

Requirements :

Key Responsibilities :

- Develop, and maintain ETL pipelines to move, transform, and load data from multiple sources into data warehouses or data lakes.

- Work with large datasets and ensure data quality and integrity throughout the ETL process.

- Integrate various data sources (on-premises and cloud) using Cloud databases, ensuring scalability and performance.

- Develop complex SQL queries to extract, transform, and load data efficiently.

- Optimize SQL queries for performance and ensure data consistency across systems.

- Manage and maintain relational databases to ensure reliable data storage and retrieval.

- Integrate data from different sources (including external APIs, third-party applications, etc.) into centralized data storage systems.

- Continuously monitor, optimize, and troubleshoot data pipelines to ensure high performance and reliability.

- Implement error handling, logging, and monitoring mechanisms to ensure smooth operations.

- Documentation and Best Practices : Create and maintain clear documentation for ETL processes, data pipelines, and data models.

Requirements :

Education :

- Bachelor's degree in computer science, Information Technology, or related field. B.Tech / B.E

Skills (Mandatory) :

- Strong proficiency in SQL for querying and managing data from relational databases.

- Excellent problem-solving, analytical, and leadership skills.

- Familiarity with DevOps for CI/CD in data integration pipelines.

- Experience with Data bricks is a plus.

- Built and optimized ETL pipelines using [e.g., Python, ADF, etc.]

- Worked with large datasets and structured/unstructured data

- Designed and maintained SQL databases & NoSQL databases

- Implemented data warehouse solutions using [e.g., AWS Redshift, Azure Synapse]

- Developed real-time & batch data processing pipelines

- Improved query performance and database optimization

- Automated data pipeline monitoring and troubleshooting issues

Projects (Optional if work experience is strong) :

- Built an end-to-end data pipeline using [tools like Apache Airflow, Python, and SQL]

- Created a real-time data streaming solution using Kafka and Spark

- Developed an ETL pipeline for structured & semi-structured data

Skills (Good to Have) :

- Experience with additional cloud platforms (e.g., Azure, AWS, Google Cloud).

- Advanced skills in data visualization tools (e.g., Power BI, SSRS, Tableau).

- Proficiency in Python or other scripting languages.

Certifications (Optional but beneficial) :

- AWS Certified Data Analytics - Specialty

- Microsoft Azure Data Engineer Associate (DP-203)

- Google Cloud Professional Data Engineer

Soft Skills :

- Strong communication and interpersonal abilities, with the ability to work effectively with cross-functional teams.

- Strong attention to detail with a focus on delivering high-quality solutions.