Data Scientist - Generative AI/LLM (5-7 yrs)

Years of Experience : 5+ Years.

Key Responsibilities :

- Scope, develop, expand, operate, and maintain scalable, reliable and safe generative AI solutions.

- Design and execute prompt engineering experiments to optimize Large Language Models (LLMs) for various use cases.

- Collaborate with SMEs to evaluate prompt effectiveness and align AI solutions with business needs.

- Understand and apply offline and online evaluation metrics for LLMs, ensuring continuous model improvements.

- Evaluate production models using live data in the absence of ground metrics, implementing robust monitoring systems.

- Monitor LLM applications for model drift, hallucinations, and performance degradation.

- Ensure smooth integration of LLMs into existing workflows, providing real-time insights and predictive analytics.

For Data Scientist :

- Proven experience in data science, with expertise in managing structured and unstructured data.

- Proficiency in statistical techniques, predictive analytics, and reporting results.

- Experience in applied science in fields like Natural Language Processing (NLP), Machine Learning (ML), Deep Learning (DL), or Multimodal Analysis.

- Strong background in software development, data modeling, or data engineering.

- Deep understanding of building and scaling ML models, specifically LLMs.

- Familiarity with open-source tools such as PyTorch, statistical analysis, and data visualization tools.

- Experience with vector databases and graph databases is a plus.

For AI/ML Engineer :

- Solid experience in business analytics, data science, software development, and data engineering.

- Expertise in Python and frameworks such as PyTorch, TensorFlow, or ONNX.

- Hands-on experience working with LLMs and Generative AI, including model development and inference patterns.

- Proven ability to design scalable systems leveraging LLMs, particularly in distributed computing environments.

Preferred Skills :

- Experience in prompt engineering and prompt optimization.

- Expertise in running experiments to evaluate generative AI performance.

- Knowledge of production-level monitoring tools for ML models, including drift detection and mitigation strategies.

- Excellent problem-solving skills and ability to work cross-functionally with data scientists, engineers, and SMEs.

- Experience with safety, security and responsible use of AI.

- Experience with red-teaming (adversarial testing) of generative AI.

- Experience with developing AI applications with sensitive data such as PHI, PII and highly confidential data.