-
Notifications
You must be signed in to change notification settings - Fork 172
Description
代码:
from pyabsa import DatasetItem, ModelSaveOption, DeviceTypeOption
import warnings
忽略警告信息
warnings.filterwarnings("ignore")
from pyabsa.framework.trainer_class.trainer_template import Trainer
from pyabsa.tasks.AspectTermExtraction.configuration.atepc_configuration import ATEPCConfigManager
from autocuda import auto_cuda
config = ATEPCConfigManager.get_atepc_config_english()
dataset = '137.all'
dataset = '100.CustomDataset'
添加sklearn的评估指标
from sklearn.metrics import precision_score, recall_score, f1_score, accuracy_score
import numpy as np
config.show_metric=True, #启用这个可以展示更多指标?
config.output_dim=3, #据说这个是方面分类的个数?
config.cross_validate_fold = 5 # 5折交叉验证
config.max_seq_length=512
config.batch_size = 8
config.learning_rate = 1e-5 # 降低学习率(当前为2e-5)
config.l2reg = 1e-4 # 增强L2正则化(当前为1e-5)
config.early_stop_by = "ate_f1" # 默认可能监控APC_ACC,改为监控ATE_F1
config.max_grad_norm = 1.0 # 防止梯度爆炸
config.use_crf = True # 改善序列标注的连贯性
config.patience = 2
config.log_step = -1
config.seed = [1]
config.verbose = False # If verbose == True, PyABSA will output the model strcture and seversal processed data examples
config.notice = (
"This is an training example for aspect term extraction" # for memos usage
)
config.checkpoint_save_mode = ModelSaveOption.SAVE_FULL_MODEL # 保存完整模型
trainer = ATEPC.ATEPCTrainer(
config=config,
dataset=dataset,
auto_device=DeviceTypeOption.AUTO, # use cuda if available
checkpoint_save_mode=ModelSaveOption.SAVE_FULL_MODEL, # save state dict only instead of the whole model
load_aug=False, # there are some augmentation dataset for integrated datasets, you use them by setting load_aug=True to improve performance
)
问题:
- 指标显示不全,如Epoch: 0| loss_apc:1.3189 | loss_ate:0.0577 |: 100%|█| 613/613 [06:13<00:00, 1.64it/s, APC_ACC: 84.49(max:84.49) | A
Epoch: 1 | Smooth Loss: 0.7048: 100%|██████████████████████████████████████████████▉| 612/613 [05:39<00:00, 1.78it/s]
根据教程,每个Epoch都应该显示Epoch: 0| loss_apc:0.0099 | loss_ate:0.0139 |: 100%|██████████| 226/226 [01:02<00:00, 3.59it/s, APC_ACC: 85.96(max:87.21) | APC_F1: 78.04(max:81.65) | ATE_F1: 82.87(max:84.58)]
但是这里APC_F1和ATE_F1都没有显示出来,如图:
- 为什么模型只能输出ATE Classification Report,而没有输出APC Classification Report的表格呢?应该怎样设置?我想知道APC任务的各项指标。如图:
- 我曾设置了多个随机种子进行训练,如[1,2,3]但每个随机种子训练后似乎是独立输出指标,也就是说,不管连续训练几次,Std和IQR的值都是0,和教程中的情况不符,请问应该怎样设置,才能使这两个值正常计算!
您的解答将对我非常有帮助,提前感谢!