jsoup

Web Scraping in Java with Jsoup

When I was starting out as a programmer and as a web scraper I was addicted to Java. I didn’t care that other languages existed. I was so stubborn that in my hobby projects I literally used Java for everything. I wrote desktop applications, web…

Scraping javascript

How to Scrape Javascript in Java

Nowadays the most popular websites have some kind of dynamic elements and they use javascript to display information. Chances are you have to crawl a website full of javascript content. Designing our web scraper, we should look for simple and pure html web pages to…

Install scrapy

How to install Scrapy on Ubuntu 16.04

Today I’m going to show you how to install Scrapy framework in your system in less than 5 minutes. In ubuntu it’s very easy to install packages so it should be the same now. Install Scrapy On Ubuntu The very first step is to make…