[FEATURE] Add ImageReward Metric

[begumcig]

✨ What You’ll Do

Adding more metrics makes Pruna’s evaluations way more insightful, so we’re super excited to welcome ImageReward (paper) (repo) into our metric lineup!

---

🛠️ Dependency

• Add image-reward to your pyproject.toml so that the ImageReward model can be imported.

---

📐 Metric Class

For the full deep-dive on Pruna metrics (class structure, state handling, tests, registration), check out our guide:
Customize a Metric

Quick Overview

1. File & Class

   • Create pruna/evaluation/metrics/metric_image_reward.py
   • Inherit from StatefulMetric

2. Configuration

   • Choose a sensible call_type and set metric_name = "image_reward"
   • Define and register any internal state via self.add_state(...)

3. Implement Core Methods

   • update(x, gt, outputs)
     • Receive inputs, ground truths, and model outputs
     • Feed the generated images into the ImageReward model to compute per-image scores
     • Accumulate scores in your state variables
   • compute()
     • Aggregate your state (e.g., average all scores)
     • Return a result with name "image_reward" and the aggregated float

4. Registration

   • Use the @MetricRegistry.register("image_reward") decorator so users can invoke your metric by name

5. Testing & Docs

   • Write unit tests under tests/evaluation/metrics/test_image_reward.py
   • Update user_manual/evaluation.rst to mention "image_reward"

---

✅ Acceptance Criteria

• Single Load: The ImageReward model is loaded once in the metric’s constructor.
• Correct Output: compute(...) returns a sensible result (e.g. if available a public benchmark score!)
• Tests & Docs: All existing and new unit tests pass, and the documentation is updated!

---

❓ Questions?

Feel free to jump into our Discord channel if you hit any roadblocks, can’t wait to see your contribution! 🎉

[Ayyanaruto]

Hi @begumcig, would it be a good idea to first create a proposed solution for this problem either in a Word or Notion document which I can share here or on Discord, instead of directly creating a PR? Once you're satisfied with the approach, I can proceed with the implementation

[begumcig]

Hi @Ayyanaruto! Sharing your ideas in a doc is a great idea! Notion always makes things easier to follow 🙂

You can check out our general How To Contribute and the updated Customize a Metric guide. I just noticed the one I originally linked was outdated, sorry about that 😅

Suggested flow:
1. Take a look at the metric guidelines: they’re usually enough to get started!
2. If it helps, feel free to sketch your plan in Notion or similar.
3. Open a draft PR and link your doc there if you’ve made one. This makes it easy to give feedback early, and you can still build the implementation right in the same PR.

Excited to see your take on this! Let us know if anything’s unclear! 🎉