Home

This project is intended to be used as documentation for the ACLED crawling project.

Prerequisites

It is expected that the user first has an installed and is running an instance of JQM. For instructions on how to do this, please refer to the full JQM documentation found here https://jqm.readthedocs.io/en/jqm-all-2.1.0/.

This documentation comprises three main sections that are briefly described below.

1. JQM - job queue management

This section covers the project specific aspects of managing a JQM instance.

2. Crawling/scraping

This covers topics regarding the crawler and scraper management.

3. JEF crawler monitoring

This last topic covers updating the JEF crawler monitor with new crawl databases.