Clarification on Setting 'linear_step_weight' in the 'force_scheduler' Function

[haoweiz23]

Hi, could you provide any insights or guidance on how to configure the 'linear_step_weight' parameter in the 'force_scheduler' function? I couldn't find any explanation or experiments about it in your paper.

[haoweiz23]

Additionally, I noticed that the "force_scheduler" is utilized within the 'force_init' function, which appears to lag behind the 'global_force_fresh' function (link). This could potentially lead to issues, such as only the first layer being force-refreshed in certain steps, as the 'cal_threshold' value is updated in the first layer.

[Shenyi-Z]

Hello, thank you for your attention!

1. The results of tuning the linear_step_weight parameter are shown in Appendix Figure 11(f) under the sensitivity analysis section. Since it is not closely related to the main topic we are discussing, token-wise caching, we did not elaborate on it in the main text.

2. Regarding the force_scheduler: Your concern is very reasonable. However, this misalignment issue does not seem to have occurred in previous experiments. I will check the code version and make adjustments in the upcoming release. It seems appropriate to set a flag specifically to update at layer0 to indicate the computation type for deeper layers.