To better reproduce your work, can you share the script for testing on MMAU?
I have two problems:
- Which prompt do you use for generating the answers on MTP? Is it something like: ". Please choose the answer from the following options: [ans1, ans2, ans3, ans4]. Please respond in two parts: and . The section should be further divided into four parts: , , , and ."?
- If the model responds with "The answer is C." and the third option is correct answer, would you consider this a correct answer?
To better reproduce your work, can you share the script for testing on MMAU?
I have two problems: