Add image-generation task exclusion to leaderboard views#52
Add image-generation task exclusion to leaderboard views#52riceharvest wants to merge 3 commits intopinchbench:mainfrom
Conversation
|
@riceharvest is attempting to deploy a commit to the Kilo Code Team on Vercel. A member of the Team first needs to authorize it. |
| const bestSubmissions = [] | ||
| const batchSize = 10 | ||
|
|
||
| for (let i = 0; i < transformedEntries.length; i += batchSize) { |
There was a problem hiding this comment.
WARNING: Performance — N individual API calls on every cold-cache page load
When excludeImageGen=true, this server component sequentially fetches submission details for every leaderboard entry (batched 10 at a time). If there are 50+ models on the leaderboard, that's 5+ serial rounds of API calls blocking the page response. Even with ISR (revalidate: 60), every 60 seconds a user will hit a cold cache and experience significant latency.
Consider:
- Adding a dedicated API endpoint (e.g.
/leaderboard?excludeTasks=task_13_image_gen) that computes adjusted scores server-side, avoiding the N+1 fetch pattern entirely. - If a new endpoint isn't feasible, moving this computation to the client side (similar to how
TaskHeatmapalready fetches submission data client-side) so the page renders immediately and scores update asynchronously. - At minimum, using
Promise.allfor all entries at once instead of sequential batches — the batching here throttles throughput but doesn't reduce total call count.
Code Review SummaryStatus: 3 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
SUGGESTION
Other Observations (not in diff)Issues found in unchanged code that cannot receive inline comments:
Files Reviewed (8 files)
Fix these issues in Kilo Cloud Reviewed by claude-opus-4.6 · 717,149 tokens |
|
Addressed the redundant filtering note in by passing the already-filtered directly to . On the N+1 point in : agreed this is not ideal architecturally. For this PR I kept the implementation server-side so the filtered view stays deterministic and consistent with the submission-detail pages, while also falling back to the original entry if a submission detail fetch fails. If we want to optimize further, the next step should be a dedicated backend/API path that returns pre-adjusted leaderboard data (or task-level aggregates) so we can avoid per-entry submission fetches during SSR. |
|
Addressed the redundant filtering note in On the N+1 point in |
|
Follow-up cleanup pushed in
That should address the heatmap re-fetch performance note while keeping the existing behavior unchanged. |
|
👋 Hi @riceharvest! I'm @olearycrew's OpenClaw bot, just dropping in with a quick update. Thanks for addressing the review feedback on this PR! The changes look good. One thing to flag: the PR now has merge conflicts — likely from some recent merges to main. Would you be able to rebase on main to resolve those when you get a chance? Also, there's a Vercel deploy authorization pending, but that's on the maintainer side — nothing you need to do there. Appreciate your contribution! 🦀 |
Summary
Adds an
excludeImageGenleaderboard mode that removes image-generation tasks from score calculations and related views.This is implemented dynamically from submission data rather than hardcoding task counts, so it stays correct if tasks are added later or if more excluded tasks are introduced.
What changed
Leaderboard filtering
excludeImageGenquery-param support on the main leaderboard pageShared task exclusion logic
lib/task-metadata.tsUI updates
Fixes included
components/task-heatmap.tsxFiles changed
app/page.tsxapp/submission/[id]/page.tsxcomponents/leaderboard-header.tsxcomponents/leaderboard-view.tsxcomponents/task-breakdown.tsxcomponents/task-heatmap.tsxlib/task-metadata.tslib/transforms.tsValidation
Build
npm run build✅Route smoke tests
/✅/?excludeImageGen=true✅/about✅/runs✅/claim✅/claim/success✅/claim/error✅/user/test✅/submission/5d73c775-fb81-4df1-ac2f-a08434541601✅/submission/5d73c775-fb81-4df1-ac2f-a08434541601?excludeImageGen=true✅Browser checks