Python Developer

Job Title: Web Scraping Developer for LinkedIn Data Extraction

About the Project
- We are seeking an experienced Web Scraping Developer to create a custom application designed to query Google Search Results specifically for LinkedIn profiles. The application will focus on extracting URLs of LinkedIn profiles associated with the roofing industry in California, targeting positions such as CEO, founder, or owner. This project aims to generate a comprehensive list of 1,000 unique LinkedIn profile URLs that fit the specified criteria for further analysis and outreach.

Responsibilities
- Develop and implement a web scraping application capable of querying Google Search with the specific query string: site:linkedin.com/in/ "Roofing" "ceo" or "founder" or "owner" "California".
- Ensure the application can efficiently handle and bypass potential scraping blocks or challenges, including CAPTCHAs and rate limits, while adhering to legal and ethical guidelines.
- Implement a mechanism to validate and filter the extracted LinkedIn profile URLs to ensure they match the specified criteria and are unique.
- Provide a clean, deduplicated list of 1,000 LinkedIn profile URLs in a structured format (e.g., CSV or Excel) for easy analysis and use.
- Incorporate error handling and logging mechanisms to ensure the application runs smoothly and any issues are promptly addressed.
- Comply with all relevant laws and LinkedIn's terms of service regarding data scraping and usage.

Qualifications
- Proven experience in web scraping and data extraction, with specific expertise in handling Google Search results.
- Strong programming skills, with proficiency in Python and libraries such as BeautifulSoup, Scrapy, or Selenium.
- Knowledge of web technologies (HTML, CSS, JavaScript) and understanding of how to navigate and parse web pages.
- Familiarity with techniques to avoid detection and blocking by web servers (e.g., rotating user agents, proxy usage).
- Attention to detail and ability to ensure the accuracy and relevance of collected data.
- Knowledge of data storage and manipulation, proficient in handling various data formats.
- Understanding of ethical web scraping practices and legal implications, particularly concerning LinkedIn's policies and data privacy laws.

Interested candidates should submit their resume, a cover letter highlighting relevant experiences, and any examples of previous web scraping projects. Preference will be given to applicants who can demonstrate experience in similar projects or have a keen understanding of the technical and ethical challenges involved in web scraping, particularly from Google Search results and LinkedIn.