Skip to content

[QUESTION] KV budget for longbench #16

@GWS0428

Description

@GWS0428

Hi, I have a question regarding the kv budget size used to evaluate longbench.

In the paper, it states '256' budget size is used.
However, I'm not sure whether it means 256 chunks (total of 2048 kv) or 256 kv.
I was confused because, in the case of outlier budget, the paper states that outlier budget was 48, but it was actually 48 * 8 in your code.

What's the exact kv budget size used in longbench evaluation?

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions