Skip to content

ATEPC训练指标显示不全 #411

@Kaia010

Description

@Kaia010

代码:
from pyabsa import DatasetItem, ModelSaveOption, DeviceTypeOption
import warnings

忽略警告信息

warnings.filterwarnings("ignore")

from pyabsa.framework.trainer_class.trainer_template import Trainer
from pyabsa.tasks.AspectTermExtraction.configuration.atepc_configuration import ATEPCConfigManager
from autocuda import auto_cuda

config = ATEPCConfigManager.get_atepc_config_english()
dataset = '137.all'

dataset = '100.CustomDataset'

添加sklearn的评估指标

from sklearn.metrics import precision_score, recall_score, f1_score, accuracy_score
import numpy as np

config.show_metric=True, #启用这个可以展示更多指标?
config.output_dim=3, #据说这个是方面分类的个数?
config.cross_validate_fold = 5 # 5折交叉验证
config.max_seq_length=512
config.batch_size = 8
config.learning_rate = 1e-5 # 降低学习率(当前为2e-5)
config.l2reg = 1e-4 # 增强L2正则化(当前为1e-5)
config.early_stop_by = "ate_f1" # 默认可能监控APC_ACC,改为监控ATE_F1
config.max_grad_norm = 1.0 # 防止梯度爆炸
config.use_crf = True # 改善序列标注的连贯性
config.patience = 2
config.log_step = -1
config.seed = [1]
config.verbose = False # If verbose == True, PyABSA will output the model strcture and seversal processed data examples
config.notice = (
"This is an training example for aspect term extraction" # for memos usage
)
config.checkpoint_save_mode = ModelSaveOption.SAVE_FULL_MODEL # 保存完整模型

trainer = ATEPC.ATEPCTrainer(
config=config,
dataset=dataset,
auto_device=DeviceTypeOption.AUTO, # use cuda if available
checkpoint_save_mode=ModelSaveOption.SAVE_FULL_MODEL, # save state dict only instead of the whole model
load_aug=False, # there are some augmentation dataset for integrated datasets, you use them by setting load_aug=True to improve performance
)

问题:

  1. 指标显示不全,如Epoch: 0| loss_apc:1.3189 | loss_ate:0.0577 |: 100%|█| 613/613 [06:13<00:00, 1.64it/s, APC_ACC: 84.49(max:84.49) | A
    Epoch: 1 | Smooth Loss: 0.7048: 100%|██████████████████████████████████████████████▉| 612/613 [05:39<00:00, 1.78it/s]
    根据教程,每个Epoch都应该显示Epoch: 0| loss_apc:0.0099 | loss_ate:0.0139 |: 100%|██████████| 226/226 [01:02<00:00, 3.59it/s, APC_ACC: 85.96(max:87.21) | APC_F1: 78.04(max:81.65) | ATE_F1: 82.87(max:84.58)]
    但是这里APC_F1和ATE_F1都没有显示出来,如图:
Image
  1. 为什么模型只能输出ATE Classification Report,而没有输出APC Classification Report的表格呢?应该怎样设置?我想知道APC任务的各项指标。如图:
Image
  1. 我曾设置了多个随机种子进行训练,如[1,2,3]但每个随机种子训练后似乎是独立输出指标,也就是说,不管连续训练几次,Std和IQR的值都是0,和教程中的情况不符,请问应该怎样设置,才能使这两个值正常计算!

您的解答将对我非常有帮助,提前感谢!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions