web scraping business

Building a Web Scraping Based SaaS Business, part 1

This is not a tutorial on how to scrape the web. This is something new I’m trying out. I’ve decided to document the whole process of creating my new “business”.  And as you would think from the post’s title it is based on online gathered…

scrapy beautifulsoup
css selector

How to Write the Best XPATH and CSS Selectors for Your Web Scraper

Selectors are one of the most important pieces of your scraper. Well-written selectors make your web scraper work efficiently and fast. When the website’s layout changes your scraper’s selectors need to be changed as well. Then, in a well-established scraping environment the only things that…

web scraping legal

Is Web Scraping Legal? Top 3 Legal Issues in Web Scraping

Before 2000, web scraping was a gray area in the legal system of US. There was no significant precedent around web scraping. The first time a company was sued for web scraping related activities happened on December 10, 1999, Ebay v. Bidder’s Edge. Bidder’s Edge was…

Html parsing

The Ultimate Resource Guide to Html Parsers

Html parsing is the backbone of every web scraping software because you need to parse html everytime. I realized that some of you are struggling with finding the right parsing library for your scraping project. This ultimate resource may help you. I gathered the best…