Director- Data & Analytics Engineer

Job Title: Director- Data & Analytics Engineer

GCL - F

Introduction to role:

As the Director of Data & Analytics Engineering, youll be at the forefront of revolutionizing how competitive intelligence is created, shared, and consumed within AstraZeneca. Through Connected Insights, we are making our data Findable, Accessible, Interoperable, and Re-usable (FAIR). Youll architect solutions that ensure unstructured data is readily available for AI applications, using innovative technologies like vector databases and knowledge graphs. Your leadership will guide a team of dedicated engineers in developing scalable data solutions and database enhancements for large-scale products. Are you prepared to drive innovation and excellence in data engineering?

Accountabilities:

1. Data Platform Design and Implementation
- Design and implement advanced data capabilities, including auto ingestion, data cataloguing, automated access control, lifecycle management, backup & restore, and AI-ready data structures.
- Implement vector databases and knowledge graphs to support AI and machine learning initiatives.

2. AI-Focused Solution Architecture
- Collaborate with AI Engineering leads and Architects to design AI-ready data architectures.
- Analyze data requirements for AI applications, modeling both structured and unstructured data sources.

3. ETL and Data Processing
- Implement optimal ETL workflows using SQL, APIs, ETL tools, AWS big data technologies, and AI-specific techniques.
- Develop processes to prepare data for AI model training and inference.

4. AI Integration and Technical Leadership
- Lead technical deliveries across multiple initiatives, focusing on integrating AI capabilities into existing data solutions.
- Provide technical feedback on design, architecture, and integration of AI-enhanced data sourcing platforms.

5. Collaborator, Teamwork and Problem Solving
- Liaise with technical infrastructure teams to resolve issues impacting AI application performance.
- Engage with architects, product owners, and business stakeholders to ensure efficient engineering of AI-driven data solutions.

6. Agile Project Management
- Lead a dedicated Data pod including managing backlogs, sprints, and planning.
- Collaborate with product pods to help them meet their deliveries.

7. Standards and Best Practices
- Define data engineering and AI integration standards in collaboration with architects and AI Engineering leads.
- Establish standard processes for managing AI model versioning and data lineage.

8. Quality Assurance and Documentation
- Test, document, and quality assess new data and AI solutions.
- Implement robust testing frameworks for AI models and data pipelines.

9. Research and Development
- Explore emerging AI technologies and drive their integration into existing data infrastructure.

10. Technical Problem Solving and Innovation
- Adopt a "can-do" approach to technical challenges related to AI integration.
- Coach team members on solving complex AI and data engineering problems.

11. Team Leadership and Development
- Build and support your team through hiring, coaching, and mentoring.
- Foster a culture of continuous learning in AI and data technologies.

12. Code and Design Quality
- Perform regular quality checks of both data engineering and AI-related code.
- Guide engineers on design patterns emphasizing AI-specific considerations.

13. Data Interoperability and FAIR Principles
- Lead initiatives to enhance data interoperability through rich metadata.
- Ensure all data solutions align with FAIR principles.

14. Knowledge Graph Development
- Be responsible for the design, implementation, and maintenance of knowledge graphs using Neo4j.
- Integrate knowledge graphs with AI applications to enhance data context.

Essential Skills/Experience:

Must have a B.Tech/M.Tech/MSc in Computer Science, Engineering, or related field.
Experience in leading data engineering teams to deliver robust and scalable data products, with a focus on preparing datasets for AI/ML use cases.
Deep expertise in the AWS data engineering ecosystem (SNS, SQS, Lambda, Glue, S3, EMR, log management, AWS containers, EC2, EBS, access control, data streaming, AWS CLI & SDK, backup & restore, etc).
Excellent programming skills in Python or Java, including Object-Oriented Programming, and proficiency with Airflow, Apache Spark, source control (GIT), and versioning.
Extensive experience in designing, building, and optimizing large-scale data pipelines, including ingestion, transformation, and orchestration using tools such as Airflow.
Familiarity with Snowflake tools and services.
Hands-on experience with metadata management and the application of controlled vocabularies and ontologies to ensure data interoperability and discoverability.
Working knowledge of vector databases and implementing semantic search capabilities for unstructured and semi-structured datasets.
Strong understanding of data modelling concepts, SQL (including advanced SQL), and database design especially for unstructured and semi-structured data (XML, JSON).
Experience designing data cataloguing, auto-ingestion, automated access control, lifecycle management, backup & restore, and other self-service data management features.
Exposure to software engineering CI/CD processes, including implementation of automated testing, build, release, deployment, containerization, and configuration management.
Experience using JIRA, Confluence, and other tools to manage Agile and SAFe project delivery.
Strong communication, teamwork, and mentoring skills, with the ability to build, coach, and guide high-performing data engineering teams focused on AI/ML objectives.

Desirable Skills/Experience:

Demonstrated experience in developing knowledge graphs (e.g., with Neo4j) and making data AI-ready for Retrieval-Augmented Generation (RAG) and Generative AI (GenAI) applications.

When we put unexpected teams in the same room, we fuel ambitious thinking with the power to inspire life-changing medicines. In-person working gives us the platform we need to connect, work at pace and challenge perceptions. Thats why we work, on average, a minimum of three days per week from the office. But that doesnt mean were not flexible. We balance the expectation of being in the office while respecting individual flexibility. Join us in our unique and ambitious world.

At AstraZeneca, youll find yourself at the heart of innovation where impactful work meets large-scale transformation. We connect across the business to influence patient outcomes positively while driving pioneering change towards becoming a digital enterprise. Collaborate with leading experts using innovative techniques to turn complex information into practical insights that improve lives globally. Our inclusive team grows with diversity bringing together different functions to decode business needs effectively. Here is where you can raise your personal profile through publishing work or showcasing your achievements.

Ready to make a difference? Apply now to join our dynamic team!