Senior Python Data Scraping Engineer

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Senior Python Data Scraping Engineer in Israel.

This role is designed for a highly technical professional who specializes in building robust, scalable, and accurate web data extraction systems within a hybrid AI-human workflow environment. You will work on complex scraping challenges involving dynamic websites, large datasets, and structured data delivery at scale. The position combines hands-on engineering with problem-solving, requiring strong Python expertise and the ability to adapt scraping strategies to evolving web environments. You will collaborate with AI-driven agents while ensuring human-level accuracy, validation, and quality control. This is a remote freelance opportunity where independence, precision, and technical creativity are essential. Your work will directly contribute to building reliable datasets that support advanced AI and analytics applications.

Accountabilities

• Own end-to-end web scraping and data extraction workflows across complex and dynamic websites.
• Build, maintain, and optimize Python-based scraping systems using tools such as BeautifulSoup, Selenium, and API integrations.
• Extract structured datasets from dynamic content sources including JavaScript-rendered pages, AJAX calls, and infinite scroll interfaces.
• Utilize internal tools (such as Apify and OpenRouter) alongside custom-built automation workflows to improve efficiency and scalability.
• Ensure high data quality through validation checks, normalization processes, and cross-source consistency verification.
• Handle large-scale scraping operations using batching, parallelization, and performance optimization techniques.
• Monitor scraping stability and adapt quickly to changes in website structure or anti-bot mechanisms.
• Deliver clean, structured datasets in formats such as CSV, JSON, or spreadsheet-based outputs.
• Apply systematic quality control processes before final data delivery.

Requirements

• Minimum 5\+ years of experience in data engineering, web scraping, automation, or software development.
• Strong hands-on experience with Python scraping frameworks such as BeautifulSoup, Selenium, or equivalent tools.
• Proven ability to extract data from complex and inconsistent web structures, including dynamic and JavaScript-heavy sites.
• Experience working with APIs, proxies, and anti-bot mitigation techniques.
• Strong background in data cleaning, normalization, transformation, and validation.
• Experience delivering structured datasets in formats such as CSV, JSON, or Google Sheets.
• Familiarity with cloud environments such as AWS or equivalent platforms.
• Experience using containerization tools like Docker in production workflows.
• Exposure to LLM-based frameworks such as LangChain, OpenRouter, or similar automation tools.
• Strong analytical mindset with exceptional attention to detail and data accuracy.
• Ability to work independently, troubleshoot issues, and manage tasks without supervision.
• Upper-intermediate or higher English proficiency (B2\+ required).
• Nice to have: GitHub portfolio showcasing relevant scraping or automation projects.

Benefits

• Fully remote freelance opportunity with flexible working schedule.
• Work on cutting-edge hybrid AI \+ human systems in a fast-growing technical field.
• Opportunity to collaborate with advanced AI tools and automation frameworks.
• Competitive compensation of up to $30/hour depending on experience and performance.
• Performance-based bonuses rewarding quality and consistency.
• Exposure to complex real-world data extraction and AI-driven workflows.
• High autonomy with the ability to choose your working pace and environment.
• Opportunity to build highly valuable expertise in AI-assisted data engineering.

How Jobgether Works

We use an

AI-powered matching process

to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice:

By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.