Pyspark GCP Engineer

Job Description Data Engineer/Lead

Required Minimum Qualifications

• Bachelors degree in computer science, CIS, or related field
• 5-10 years of IT experience in software engineering or related field
• Experience on project(s) involving the implementation of software development life cycles (SDLC) 

Primary Skills : PySpark, SQL, GCP EcoSystem(Biq Query, Cloud Composer, DataProc)

• Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks leveraging open source tools and data processing frameworks.
• Hands-on on technologies such as Kafka, Apache Spark (SQL, Scala, Java), Python, Hadoop Platform, Hive, airflow
• Experience in GCP Cloud Composer, Big Query, DataProc
• Offer system support as part of a support rotation with other team members.
• Operationalize open-source data-analytic tools for enterprise use.
• Ensure data governance policies are followed by implementing or validating data lineage, quality checks, and data classification.
• Understand and follow the company development lifecycle to develop, deploy and deliver.