summary renamed to judge

[jonas-becker]

No description provided.

[github-actions]

Coverage Report

Name                                                        Stmts   Miss Branch BrPart  Cover
---------------------------------------------------------------------------------------------
mallm/__init__.py                                               1      0      0      0   100%
mallm/agents/__init__.py                                        0      0      0      0   100%
mallm/agents/agent.py                                          90     46     20      3    45%
mallm/agents/draftProposer.py                                  11      1      2      1    85%
mallm/agents/judge.py                                          59     44     20      1    20%
mallm/agents/panelist.py                                       11      6      2      0    38%
mallm/coordinator.py                                          163     73     54      7    52%
mallm/decision_protocols/__init__.py                            0      0      0      0   100%
mallm/decision_protocols/approval_voting.py                    47     31     10      0    28%
mallm/decision_protocols/consensus.py                          52     11     10      1    74%
mallm/decision_protocols/consensus_voting.py                   74     56     20      0    19%
mallm/decision_protocols/cumulative_voting.py                  57     39     18      0    24%
mallm/decision_protocols/judge.py                              29     14      4      0    45%
mallm/decision_protocols/protocol.py                          150    117     60      1    16%
mallm/decision_protocols/ranked_voting.py                      45     30     12      0    26%
mallm/decision_protocols/simple_voting.py                      47     31     10      0    28%
mallm/discussion_paradigms/__init__.py                          0      0      0      0   100%
mallm/discussion_paradigms/collective_refinement.py            61     46     14      1    21%
mallm/discussion_paradigms/debate.py                           86     70     36      1    14%
mallm/discussion_paradigms/memory.py                           20      7      2      1    64%
mallm/discussion_paradigms/paradigm.py                         92     71     30      1    18%
mallm/discussion_paradigms/relay.py                            22      9      2      1    58%
mallm/discussion_paradigms/report.py                           26     13      4      1    47%
mallm/evaluation/__init__.py                                    0      0      0      0   100%
mallm/evaluation/evaluator.py                                 220    185    122      1    11%
mallm/evaluation/metrics/__init__.py                            0      0      0      0   100%
mallm/evaluation/metrics/bertscore.py                           9      2      0      0    78%
mallm/evaluation/metrics/bleu.py                                9      2      0      0    78%
mallm/evaluation/metrics/ifeval.py                             90     59     18      0    29%
mallm/evaluation/metrics/meteor.py                             20      9      2      0    50%
mallm/evaluation/metrics/metric.py                             10      1      0      0    90%
mallm/evaluation/metrics/qa.py                                 52     27     14      0    38%
mallm/evaluation/metrics/rouge.py                              10      3      0      0    70%
mallm/evaluation/plotting/__init__.py                           0      0      0      0   100%
mallm/evaluation/plotting/plots.py                            139    122     22      1    11%
mallm/models/Chat.py                                           76     47     22      0    30%
mallm/models/__init__.py                                        2      0      0      0   100%
mallm/models/discussion/CriticalResponseGenerator.py           44     30     12      0    25%
mallm/models/discussion/FreeTextResponseGenerator.py           63     41     14      0    29%
mallm/models/discussion/ReasoningResponseGenerator.py          46     29     10      0    30%
mallm/models/discussion/ResponseGenerator.py                  151     96     40      1    29%
mallm/models/discussion/SimpleResponseGenerator.py             55     37     14      0    26%
mallm/models/discussion/SplitFreeTextResponseGenerator.py      36     24      4      0    30%
mallm/models/discussion/__init__.py                             0      0      0      0   100%
mallm/models/personas/ExpertGenerator.py                       31     21      8      0    26%
mallm/models/personas/IPIPPersonaGenerator.py                  49     40     20      0    13%
mallm/models/personas/MockGenerator.py                          9      0      0      0   100%
mallm/models/personas/NoPersonaGenerator.py                    11      2      0      0    82%
mallm/models/personas/PersonaGenerator.py                       9      1      0      0    89%
mallm/models/personas/__init__.py                               0      0      0      0   100%
mallm/scheduler.py                                            306    306     52      0     0%
mallm/scripts/__init__.py                                       0      0      0      0   100%
mallm/scripts/batch_mallm.py                                  106    106     24      0     0%
mallm/utils/CustomFormatter.py                                 10     10      0      0     0%
mallm/utils/__init__.py                                         0      0      0      0   100%
mallm/utils/config.py                                          97     42     30      1    44%
mallm/utils/dicts.py                                           29      0      0      0   100%
mallm/utils/enums.py                                           10      0      0      0   100%
mallm/utils/evaluation/__init__.py                              0      0      0      0   100%
mallm/utils/evaluation/instructions.py                        698    398    144      0    36%
mallm/utils/evaluation/instructions_registry.py                19      5      4      0    61%
mallm/utils/evaluation/instructions_util.py                    61     40     12      0    29%
mallm/utils/functions.py                                       12     12      6      0     0%
mallm/utils/task_instructions.py                                1      0      0      0   100%
mallm/utils/types.py                                           91     17     14      0    70%
mallm/utils/utils.py                                           35     35      0      0     0%
---------------------------------------------------------------------------------------------
TOTAL                                                        3759   2464    938     24    29%