More detail about how to use your inference?

[Tamiflu233]

I try to run the train_script mistral_lora_D_R_H_F_desc.sh and get the ckpt called D_R_H_F_desc_lr5e-5_gpu.And then I run AutoRE.sh but fail to do the inference with the error
ValueError:
Can't find 'adapter_config.json' at 'AutoRE/ckpt/mistral/relation/checkpoint-1200'
Can you provide more about how to inferece?

[bigdante]

If you change it to Mistral, the step of relation should be changed. you can check the checkpoint step in the folder. (same to head and fact step).
Besides, I am still checking my code. If anything wrong, you can contact me anytime.
Best~

[Tamiflu233]

These are the ckpts I get after I run mistral_lora_D_R_H_F_desc.sh.But the code in inference.py seems to need to read the weights under mistral/relation, mistral/subject, and mistral/fact. Did I run the fine-tuning script incorrectly?

[bigdante]

In fact, you can try "mistral_loras_D_R_H_F_desc.sh", it is best relation extraction paradigm I found, with 3 loras to do the job.
so far I only provide the inference code for this, and I will try to put all inference code for all re paradigms tonight. and maybe you can retry it tomorrow.
sorry about my mistake.
Best~

[Tamiflu233]

I run your new AutoRE.sh and get an error:inference.py: error :unrecognized arguments: --finetuning_type lora

I only change lora_test="loras_D_R_H_F_desc" to lora_test="lora_D_R_H_F_desc"

[bigdante]

I run your new AutoRE.sh and get an error:inference.py: error :unrecognized arguments: --finetuning_type lora I only change lora_test="loras_D_R_H_F_desc" to lora_test="lora_D_R_H_F_desc"

delete the --finetune_type(I just fixed it), it is used in finetuning. I changed the basic.py but forgot to delete this param.
Thank you for pointing out the mistakes.
Best~

[Tamiflu233]

And I also get the error:'Namespace' object has no attribute 'with_relation_desc'. My lora path is `model_path="AutoRE/ckpt/mistral/D_R_H_F_desc_lr5e-5_gpu/checkpoint-38100"`

[bigdante]

And I also get the error:'Namespace' object has no attribute 'with_relation_desc'. My lora path is model_path="AutoRE/ckpt/mistral/D_R_H_F_desc_lr5e-5_gpu/checkpoint-38100"

I used it to test if relation description is important to RE. You can just delete it.
I have fixed it and push the code.

Best~

[Tamiflu233]

Thank you for your help, but it seems I've encountered another issue. Could it be because of an error in my data path?

[bigdante]

Thank you for your help, but it seems I've encountered another issue. Could it be because of an error in my data path?

My old code is quite redundant. Therefore, I used this new code, and I might have accidentally deleted some parts during the process, which caused errors. I have made the corrections. I think I need to verify all the code by myself tomorrow. I apologize again, as I have been a bit busy with other things recently.

[Tamiflu233]

Thx for you patience.And I have some other questions.First off, I'm curious about how your generative approach deals with the issue where the generated triples have relationships and entities that mean the same as the real labels, but are phrased differently, especially the relationships. How does your code handle this when calculating the F1 score? Also, I'm wondering if I need to configure any extra parameters in AutoRE.sh to see the F1 results. How can I easily check the accuracy of AutoRE's predictions?

[bigdante]

Thx for you patience.And I have some other questions.First off, I'm curious about how your generative approach deals with the issue where the generated triples have relationships and entities that mean the same as the real labels, but are phrased differently, especially the relationships. How does your code handle this when calculating the F1 score? Also, I'm wondering if I need to configure any extra parameters in AutoRE.sh to see the F1 results. How can I easily check the accuracy of AutoRE's predictions?

Good question. Actually, if the triplet facts are different from the labels, I consider them to be incorrect. You can check the metric I used in the paper. Only if the triple facts are exactly same to the labels, I consider them as true positive. So in fact, AutoRE performs better than I reported in the paper, coz I have found lots of triplet facts to be correct but not in the labels.

As for the f1 results, you can check the "cal_result_lora_facts" func in "report_result.py", it will calculate the final result. And you can use "report_relations_result" to print the result.

Best~

[Tamiflu233]

Why does this line of code in the image use 'lora_D_R_F_for_test' instead of 'lora_D_R_H_F_for_test'?

[bigdante]

Why does this line of code in the image use 'lora_D_R_F_for_test' instead of 'lora_D_R_H_F_for_test'?

sorry, In fact, I have fixed the code, but I encountered a network issue with the push（And I forgot to push it again,hhhhh）. You can switch the function to the corresponding mode directly. And I will update the code right now.

[Tamiflu233]

What's the meaning of `data_from`?I see `"data_from" : "redocred_test"` and `"data_from" : "redocred_train"` in redocred_test.json.Does this mean that during model testing, they're not just using the test data from Redocred, but also incorporating some of the training data used during fine-tuning?

[bigdante]

No, I used to use datas of other source, "data_from" helps me to figure out which dataset the sample is from.
And here, It works as an id to filter the sample i have already processed.
Best~