How to install Scrapy on Ubuntu 16.04

Install scrapy

Today I’m going to show you how to install Scrapy framework in your system in less than 5 minutes. In ubuntu it’s very easy to install packages so it should be the same now.

Install Scrapy On Ubuntu

The very first step is to make sure that each dependency of Scrapy is installed locally in your system.

Scrapy dependencies

You will need these packages installed already:

  • python-dev
  • python-pip
  • libxml2-dev
  • libxslt1-dev
  • zlib1g-dev
  • libffi-dev
  • libssl-dev

If you’re not sure these packages are already installed or you want to make sure they are, just open a terminal with Ctrl+alt and run this command:

This command will install every dependency you will need.

Install Scrapy with pip

Now you can easily install scrapy with pip, run this command:

You can test if Scrapy installed properly by running this:

Now you should see which Scrapy version is installed now.

Scrapy: command not found

In case you see something like “scrapy: command not found” or Scrapy just doesn’t work then follow this guide:

Install lxml and gcc with these commands:

Install twisted:

Install pyOpenSSL:

Install pycrypto:

Install easy_install:

Finally, install Scrapy:

You’re done! Again, you can test that Scrapy is installed correctly with this command:

If you cannot install scrapy for some reason let me know in the comments and I will help you!

So now you’ve just setup Scrapy on your system. If you want to learn how to build awesome scrapers with Scrapy just fill in the form below and get started!