node-crawler-cn-Wallstreet

A node crawler using cheerio & nightmare to scrape the news & the article of news from wallstreetcn, and save the data to json file.

Installation

git clone git@github.com:jackmew/node-crawler-wallstreetcn.git  

Usage

cd node-crawler-wallstreetcn  
node index.js

Specify:
newstype: [global,shares,bonds,commodities,forex,enterprise,economy,charts,china,us,europe,japan]
numOfRecord: 5

node index.js forex 10

Dev mode to show the log .

DEBUG=nightmare node index.js commodities 8

Then it will save json file to /output.

Todo

• Command Line Interface