About the results in model_zoo

[Tingji2419]

Thanks for your great work on the reproduciton of QLIB benchmark. But I have a little questions about the results.

First, about the metrics. The values in the table are much higher than those in the original QLIB benchmark (https://github.com/microsoft/qlib/tree/main/examples/benchmarks), e.g. the IC of Transformer model in Alpha360 dataset is 0.08938, which is greater than 0.0114 in the original QLIB benchmark.

Second, about fairness of the training process. Simple LSTMs and Transformers work better than more complex sota models such as Informer and PatchTST. Can you provide details and setup of the data preprocessing and training?

Looking forward to the reply.

[QuantML-C]

Hello

For the first question, I guess the main difference comes from the stock universe and time period, in my case, I use all a-share and the test period is 2021-2023.

dataset for all the ts models:

dataset:
class: TSDatasetH
module_path: qlib.data.dataset
kwargs:
handler:
class: Alpha158
module_path: qlib.contrib.data.handler
kwargs: *data_handler_config
segments:
train: [2016-01-01, 2019-12-31]
valid: [2020-01-01, 2020-12-31]
test: [2021-01-01, 2023-12-31]
step_len: 60