Skip to content

MMAU test script #19

@Jacoblincc

Description

@Jacoblincc

To better reproduce your work, can you share the script for testing on MMAU?
I have two problems:

  1. Which prompt do you use for generating the answers on MTP? Is it something like: ". Please choose the answer from the following options: [ans1, ans2, ans3, ans4]. Please respond in two parts: and . The section should be further divided into four parts: , , , and ."?
  2. If the model responds with "The answer is C." and the third option is correct answer, would you consider this a correct answer?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions