As a prominent data modality task, time series forecasting plays a pivotal role in diverse applications. With the remarkable advancements in Large Language Models (LLMs), the adoption of LLMs as the foundational architecture for time series modeling has gained significant attention. Although existing models achieve some success, they rarely both model time and frequency characteristics in a pretraining-finetuning paradigm leading to suboptimal performance in predictions of complex time series, which requires both modeling periodicity and prior pattern knowledge of signals. We propose MoFE-Time, an innovative time series forecasting model that integrates time and frequency domain features within a Mixture of Experts (MoE) network. Moreover, we use the pretraining-finetuning paradigm as our training framework to effectively transfer prior pattern knowledge across pretraining and finetuning datasets with different periodicity distributions. Our method introduces both frequency and time cells as experts after attention modules and leverages the MoE routing mechanism to construct multidimensional sparse representations of input signals. In experiments on six public benchmarks, MoFE-Time has achieved new state-of-the-art performance, reducing MSE and MAE by 6.95% and 6.02% compared to the representative methods Time-MoE. Beyond the existing evaluation benchmarks, we have developed a proprietary dataset, NEV-sales, derived from real-world business scenarios. Our method achieves outstanding results on this dataset, underscoring the effectiveness of the MoFE-Time model in practical commercial applications.
The checkpoint has been released on Google Drive:
https://drive.google.com/drive/folders/1AkhMj8atS90m59WjNwMIOD2odJBr1PyA
Huggingface checkpoint will be released soon ^-^
paper: https://arxiv.org/abs/2507.06502
Experimental results demonstrate that our method achieve SOTA on six public datasets and NEV-sales, which demonstrates the effectiveness of the MoFE-Time model.
-
data prepare
download from huggingface https://huggingface.co/datasets/Maple728/Time-300B
-
Install Pytorch and other dependencies.
pip install -r requirements.txt -
start pretrain
sh ./src/pretrain_and_eval_ds.shPretrain on Multiple Nodes
sh ./src/pretrain_and_eval_nodes.sh
```sh ./src/fine_tune_ds.sh```
🙋 Please let us know if you find out a mistake or have any suggestions!
🌟 If you find the MOFE-Time models helpful in your research, please consider to star this repository and cite the corresponding
@misc{liu2025mofetimemixturefrequencydomain,
title={MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models},
author={Yiwen Liu and Chenyu Zhang and Junjie Song and Siqi Chen and Sun Yin and Zihan Wang and Lingming Zeng and Yuji Cao and Junming Jiao},
year={2025},
eprint={2507.06502},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2507.06502},
}
- TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis, in arXiv 2024. [paper] [GitHub Repo]
- Towards Neural Scaling Laws for Time Series Foundation Models, arXiv 2024. [paper]
- Foundation Models for Time Series Analysis: A Tutorial and Survey, in KDD 2024. [paper] [Tutorial]
- What Can Large Language Models Tell Us about Time Series Analysis, in ICML 2024. [paper]
- Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects, in TPAMI 2024. [paper] [Website]
- Transformers in Time Series: A Survey, in IJCAI 2023. [paper] [GitHub Repo]
- A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection, in TPAMI 2024. [paper] [Website]
We appreciate the following GitHub repos a lot for their valuable code and efforts.
Time-MoE (https://github.com/Time-MoE/Time-MoE)
Time-Series-Library （https://github.com/thuml/Time-Series-Library）
Chronos (https://github.com/amazon-science/chronos-forecasting)
Times-FM (https://github.com/google-research/timesfm)
Moirai(https://github.com/SalesforceAIResearch/uni2ts)
If you have any questions or want to use the code, please contact caoyuji@lixiang.com or liuyiwen@lixiang.com
asLLR: LLM Based Leads Raking In Auto Sales（https://github.com/alg-znsy-li/as_llr）
理想汽车-智能商业-算法团队