Upload Button Icon Add office photos
Engaged Employer

i

This company page is being actively managed by Hoonartek Team. If you also belong to the team, you can get access from here

Hoonartek Verified Tick

Compare button icon Compare button icon Compare
3.7

based on 134 Reviews

filter salaries All Filters

21 Hoonartek Jobs

Lead Databricks Engineer

7-10 years

Pune

1 vacancy

Lead Databricks Engineer

Hoonartek

posted 3mon ago

Job Role Insights

Flexible timing

Job Description

Job Title: Azure Databricks Lead
Job Overview:
As an Azure Databricks Lead/Specialist , you will play a critical role in designing, implementing, and optimizing data solutions using Azure Databricks. Your expertise will contribute to building robust data pipelines, ensuring data quality, and enhancing overall performance. You'll collaborate with cross-functional teams to deliver high-quality solutions aligned with business requirements.
Responsibilities:
1. Design and Develop Data Pipelines :
o Create scalable data processing pipelines using Azure Databricks and PySpark/Scala .
o Implement ETL (Extract, Transform, Load) processes to ingest, transform, and load data from various sources.
o Collaborate with data engineers and architects to ensure efficient data movement and transformation.
2. Data Quality Implementation :
o Establish data quality checks and validation rules within Azure Databricks.
o Monitor data quality metrics and address anomalies promptly.
o Work closely with data governance teams to maintain data accuracy and consistency.
3. Unity Catalog Integration :
o Leverage Azure Databricks Unity Catalog to manage metadata, tables, and views.
o Integrate Databricks assets seamlessly with other Azure services.
o Ensure proper documentation and organization of data assets.
4. Delta Lake Expertise :
o Understand and utilize Delta Lake , which provides ACID transactions and time travel capabilities on top of data lakes.
o Implement Delta Lake tables for reliable data storage and versioning.
o Optimize performance by leveraging Delta Lake features.
5. Performance Tuning and Query Optimization :
o Profile and analyze query performance.
o Optimize SQL queries, Spark jobs, and transformations for efficiency.
o Tune resource allocation to achieve optimal execution times.
6. Resource Optimization :
o Manage compute resources effectively within Azure Databricks clusters.
o Scale clusters dynamically based on workload requirements.
o Monitor resource utilization and cost efficiency.
7. Source System Integration :
o Integrate Azure Databricks with various source systems (e. g. , databases, data lakes, APIs).
o Ensure seamless data ingestion and synchronization.
o Handle schema evolution and changes in source data.
8 Stored Procedure Conversion in Databricks:
Convert existing stored procedures (e. g. , from SQL Server) into Databricks-compatible code.
Optimize and enhance stored procedures for better performance within Databricks
SSRS conversion experience
Qualifications and Skills:
Education : Bachelor's degree in Computer Science, Information Technology, or a related field.
Experience :
o Minimum 5 years of experience architecting and building data platforms on Azure .
o Proficiency in Azure Databricks , PySpark , and SQL .
o Familiarity with Delta Lake concepts.
Certifications (preferred):
o Microsoft Certified: Azure Data Engineer Associate or similar.
o Databricks Certified Associate Developer for Apache Spark .
Soft Skills :
o Strong problem-solving abilities.
o Excellent communication and collaboration skills.
o Ability to work in a fast-paced, agile environment.
o
Additional Considerations:
Forward-Thinking Approach :
o As an Azure Databricks Specialist, you'll contribute to innovative solutions that drive business insights and enhance data capabilities.
o Stay updated with industry trends and emerging technologies.


Employment Type: Full Time, Permanent

Functional Areas: Other

Read full job description

Prepare for Engineer roles with real interview advice

People are getting interviews at Hoonartek through

(based on 9 Hoonartek interviews)
Job Portal
Campus Placement
Company Website
Recruitment Consultant
45%
22%
11%
11%
11% candidates got the interview through other sources.
Moderate Confidence
?
Moderate Confidence means the data is based on a sufficient number of responses received from the candidates

What people at Hoonartek are saying

What Hoonartek employees are saying about work life

based on 134 employees
74%
83%
51%
94%
Flexible timing
Monday to Friday
No travel
Day Shift
View more insights

Hoonartek Benefits

Submitted by Company
Work From Home
Health Insurance
Job Training
Soft Skill Training
Team Outings
Submitted by Employees
Work From Home
Job Training
Soft Skill Training
Health Insurance
Team Outings
Cafeteria +6 more
View more benefits

Compare Hoonartek with

TCS

3.7
Compare

Infosys

3.7
Compare

Wipro

3.7
Compare

Tech Mahindra

3.6
Compare

HCLTech

3.6
Compare

Accenture

3.9
Compare

Capgemini

3.8
Compare

Cognizant

3.8
Compare

IBM

4.1
Compare

LTIMindtree

3.5
Compare

Collabera Technologies

3.5
Compare

Softenger

4.3
Compare

XcelServ Solutions

4.6
Compare

Capital Numbers Infotech

4.6
Compare

Creative Synergies Group

4.4
Compare

Akal Information Systems

4.0
Compare

Foray Software

3.5
Compare

Tekwissen

4.8
Compare

Damco Solutions

4.1
Compare

Nelito System

3.6
Compare

Similar Jobs for you

Technical Consultant at Birlasoft (India) Limited

Pune

4-8 Yrs

Not Disclosed

Senior Technical Specialist at Datavail Infotech Pvt. Ltd.

Mumbai

5-9 Yrs

Not Disclosed

Senior Software Engineer at Valgenesis

Chennai

6-9 Yrs

Not Disclosed

Lead at IRIS SOLUTIONS

Noida

6-10 Yrs

Not Disclosed

Lead at Hexaware Technologies Ltd.

Chennai, Pune + 5

5-10 Yrs

Not Disclosed

Lead at Hexaware Technologies Ltd.

Chennai, Pune + 5

5-10 Yrs

Not Disclosed

Lead at Hexaware Technologies Ltd.

Chennai, Pune + 5

7-10 Yrs

Not Disclosed

Technical Project Lead at Eurofins It Solutions India Pvt Ltd

New Delhi

8-10 Yrs

Not Disclosed

System Analyst at Prudential

Chennai, Pune + 5

2-6 Yrs

Not Disclosed

Lead Software Engineer at Notified

Hyderabad / Secunderabad

5-7 Yrs

Not Disclosed

Lead Databricks Engineer

7-10 Yrs

Pune

Computer Science, SQL, Information Technology +5 more

3mon ago·via naukri.com

Pyspark / Databricks Lead

6-11 Yrs

₹ 20 - 35L/yr

Pune

AWS, Azure DevOps, Data Bricks +1 more

12d ago·via naukri.com

Databricks Architect

10-17 Yrs

Pune, Mumbai

Azure DevOps, Data Bricks, Pyspark

15d ago·via naukri.com

Lead Generation Executive

3-7 Yrs

Pune

Telecalling, Email Marketing, Inside Sales +5 more

2mon ago·via naukri.com

Data Migration Tech Lead

8-9 Yrs

Pune

Automation Testing, Project Management, Business Operations +7 more

2mon ago·via naukri.com

PCSM Developer

5-10 Yrs

Mumbai

Medical Coding, Linux Administration, Risk Management +6 more

3mon ago·via naukri.com

EP Developer

3-5 Yrs

Remote

ESIC, Medical Coding, Software Configuration Management +18 more

3mon ago·via naukri.com

Senior Data Engineer - Apache Nifi

6-10 Yrs

Remote

Medical Coding, Computer Science, Production Support +5 more

3mon ago·via naukri.com

Senior Full Stack Developer

3-8 Yrs

Remote

Medical Coding, Manual Testing, Computer Science +7 more

3mon ago·via naukri.com

Senior Full Stack Dev ( SQL & NoSQL)

10-12 Yrs

Remote

Medical Coding, Manual Testing, SQL +7 more

3mon ago·via naukri.com