Attila Toth

css selector

How to Write the Best XPATH and CSS Selectors for Your Web Scraper

Selectors are one of the most important pieces of your scraper. Well-written selectors make your web scraper work efficiently and fast. When the website’s layout changes your scraper’s selectors need to be changed as well. Then, in a well-established scraping environment the only things that have to be changed are the selectors. In this post …

How to Write the Best XPATH and CSS Selectors for Your Web Scraper Read More »

beautifulsoup

Web Scraping in Python with Beautifulsoup

I’m often asked, “Which web scraping library should I choose?” I usually answer choose the one that is the most popular in your programming language. If it’s java then choose Jsoup. If Python BeautifulSoup is your best bet. BeautifulSoup Installation You can easily install the most recent version of beautifulsoup with pip: pip install beautifulsoup4 …

Web Scraping in Python with Beautifulsoup Read More »

web scraping legal

Is Web Scraping Legal? Top 3 Legal Issues in Web Scraping

Before 2000, web scraping was a gray area in the legal system of US. There was no significant precedent around web scraping. The first time a company was sued for web scraping related activities happened on December 10, 1999, Ebay v. Bidder’s Edge. Bidder’s Edge was an aggregator of auction listings. Users could search through tons …

Is Web Scraping Legal? Top 3 Legal Issues in Web Scraping Read More »