scrapy mysql

Gathering URLs To Scrape From Database

I have a project where a script dynamically updates a database with URLs the scraper has to scrape. This database contains hundreds of URLs. I had to find a way to fetch all the URLs from the db with scrapy then run the spider on…