The emotion classification model's performance is almost the same as a random guess

[YipengUva]

Hi, I repeat the emotion classification experiment and get terrible results. I couldn't what is the issue.

1. The experiment is repeated using the command line "!python3 experiments/run_clf_multihead.py --text-key Tweet --train data/semeval/train.csv --val data/semeval/val.csv --process-fn process_tweet".

2. Then, I got a series of classifiers in transformer_multihead from the 1)step.

3. Then I used "!python3 run_classifier.py --load transformer_multihead/model_ep0.clf --text-key Tweet --data data/semeval/val.csv --model transformer --write-results results/semeval/val_result.csv" on the validation set.

4. The performance is evaulated with respect to balanced accuracy, f1 score and ROC using metrics module from sklearn package. The results are shown as follows.

                        anger	anticipation	disgust	fear	joy	sadness	surprise	trust
   

balanced accuracy 0.500876 0.500000 0.537070 0.500000 0.500000 0.500000 0.499412 0.500593
f1_score 0.525000 0.245545 0.488992 0.240318 0.622084 0.460469 0.000000 0.092672
ROC 0.537700 0.450639 0.549253 0.474326 0.508107 0.481694 0.504079 0.500841

Is anything I can do to make it work?

Regards, Yipeng

[YipengUva]

Furthermore, I also tried the pre-trained model transformer_semeval.clf using the command line "!python3 run_classifier.py --load path-to-downloaded-models/transformer_semeval.clf --text-key Tweet --data data/semeval/test.csv --model transformer --write-results results.csv" on Jupyter notebook, the results are also terrible.

[hendrixmar]

Furthermore, I also tried the pre-trained model transformer_semeval.clf using the command line "!python3 run_classifier.py --load path-to-downloaded-models/transformer_semeval.clf --text-key Tweet --data data/semeval/test.csv --model transformer --write-results results.csv" on Jupyter notebook, the results are also terrible.

Did you solve the problem?. Im currently dealing with the same issue

[Saumajit]

@YipengUva I am also trying to use the finetuned classifier for inference by running the same command you mentioned. It is showing me segmentation fault (core dumped). Do you have any idea how to fix this ? Also what did you do to fix your issue?

[YipengUva]

Sorry, I didn't have this problem for this task. As for how to fix, you can search it on google. It seems that multiple cores are occupied by other process or other terminal on the server. Either, I didn't fix the issue. It still has the performance similar as random guess. Regards, Yipeng

…

On Aug 10 2020, at 7:18 am, Saumajit Saha ***@***.***> wrote: @YipengUva (https://github.com/YipengUva) I am also trying to use the finetuned classifier for inference by running the same command you mentioned. It is showing me segmentation fault (core dumped). Do you have any idea how to fix this ? Also what did you do to fix your issue? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (#75 (comment)), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AD52CBQEM43FESDU4KRPDHLR77XQZANCNFSM4MZ2HJPA).