Capgemini Pyspark Developer Interview Questions and Answers

Q1. Get the employee with salary higher than average salary of their department
Ans. 
Retrieve employees earning above their department's average salary using PySpark.
• Use DataFrame API to group by department and calculate average salary.

• Join the original DataFrame with the average salary DataFrame.

• Filter employees where their salary is greater than the average salary of their department.

• Example: If department A has an average salary of 5000, select employees with salary > 5000.
Answered by AI
Add your answer
Q2. Given a dataset with columns name, subject and marks. Get the name of students with marks greater than 40. Data: A maths 56 A science 34 B maths 45 B science 98 Expected output: B
Ans. 
Filter students with marks greater than 40 from a dataset of names, subjects, and marks.
• Use PySpark DataFrame to load the dataset.

• Apply a filter condition to select rows where marks > 40.

• Extract the unique names of students who meet the criteria.

• Example: For the given data, only student 'B' has marks greater than 40.
Answered by AI
Add your answer
Q3. Can we change Hive managed table to external table
Ans. 
Yes, you can change a Hive managed table to an external table using SQL commands.
• Use the command: ALTER TABLE table_name SET TBLPROPERTIES ('EXTERNAL'='TRUE');

• You must ensure that the data is moved to a location accessible by Hive.

• Example: ALTER TABLE my_table SET TBLPROPERTIES ('EXTERNAL'='TRUE');

• After conversion, Hive will not manage the data, allowing for external access.
Answered by AI
Add your answer

1. Find duplicate
2. 2,3 highest salary

Basic to moderate sql questions