Python Software Developer

Key Responsibilities:

• Develop and maintain automated web scraping scripts using Python libraries such as Beautiful Soup, Scrapy, and Selenium.
• Optimize scraping pipelines for performance, scalability, and resource efficiency.
• Handle dynamic websites, CAPTCHA-solving, and implement IP rotation techniques for uninterrupted scraping.
• Process and clean raw data, ensuring accuracy and integrity in extracted datasets.
• Collaborate with cross-functional teams to understand data requirements and deliver actionable insights.
• Leverage APIs when web scraping is not feasible, managing authentication and request optimization.
• Document processes, pipelines, and troubleshooting steps for maintainable and reusable scraping solutions.
• Ensure compliance with legal and ethical web scraping practices, implementing security safeguards.

Requirements:

• Education:
  Bachelors degree in Computer Science, Engineering, or a related field.

• Experience:
  2+ years of Python development experience, with at least 1 year focused on web scraping.

• Technical Skills:

• Proficiency in Python and libraries like Beautiful Soup, Scrapy, and Selenium.
• Experience with regular expressions (Regex) for data parsing.
• Strong knowledge of HTTP protocols, cookies, headers, and user-agent rotation.
• Familiarity with databases (SQL and NoSQL) for storing scraped data.
• Hands-on experience with data manipulation libraries such as pandas and NumPy.
• Experience working with APIs and managing third-party integrations.
• Familiarity with version control systems like Git.

• Bonus Skills:

• Knowledge of containerization tools like Docker.
• Experience with distributed scraping solutions and task queues (e.g., Celery, RabbitMQ).
• Basic understanding of data visualization tools.

• Non-Technical Skills:

• Strong analytical and problem-solving skills.
• Excellent communication and documentation skills.
• Ability to work independently and collaboratively in a team environmen