Skip to content

Add script only for crawling stats data #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
hugg95 wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Add script only for crawling stats data #52

hugg95 wants to merge 2 commits into from

Conversation

@hugg95
Copy link
Contributor

@hugg95 hugg95 commented Jun 10, 2019 •
edited
Loading

Add a script named stats_parse.py only for crawling stats data from http://www.stats.gov.cn/tjsj/tjbz/tjyqhdmhcxhfdm/${year}.

  1. this script reads the latest row that has schema stats from sources.tsv
  2. this script will generate a data file in .tsv format under folder stats/
  3. this script not designed for crawling mca data

Just for your information.
Thanks.

hugg95 added 2 commits June 1, 2019 19:10
@hugg95
Add a new script for crawling stats code
ada95e0 
    add stats_parse.py just for stats data

scripts/stats_parse.py
@hugg95
Update the chinese words use unicode format
1dfee8d 
scripts/stats_parse.py
@hugg95 hugg95 mentioned this pull request Jun 10, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hugg95