BERT-for-20NewsGroups

《2021医学健康数据分析与挖掘》课程论文 -- 基于BERT的20NewsGroups数据集新闻分类实验

Virtual Environment

You can build a virtual environment for project operation.

# Building a virtual environment
pip3 install virtualenv
pip3 install virtualenvwrapper

virtualenv -p /usr/local/bin/python3.6 $env_name --clear  

# active venv.
source $env_name/bin/activate  

# deactive venv.
deactivate

Requirements

pip3 install -r requirements.txt

If you cannot download torch automatically through requirements.txt, you can delete the torch version information and get the command line of torch installation from the torch official website. Note that the installed torch version needs to be the same as that in requirenemts.txt.

OSX

pip3 install torch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2

Linux and Windos

# CUDA 11.0
pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

# CUDA 10.2
pip install torch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2

# CUDA 10.1
pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

# CUDA 9.2
pip install torch==1.7.1+cu92 torchvision==0.8.2+cu92 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

# CPU only
pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Default Run

Create Dic. Before running, you need to build two folders, logging and models, in the project folder

Model training and evaluation

python3 main.py

modify hyperparameters
You can modify the model hyperparameters by editing the config.py file.
vi config.py

Training Log

The training log files are stored in the logging folder, corresponding to the training logs of the BERT-base and BERT-large versions respectively.