Skip to content

Levi-Ackman/TeCh

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Code for ICLR26 (Oral):

Decentralized Attention Fails Centralized Signals: Rethink Transformers for Medical Time Series

Introduction

1. Mismatch between centralized MedTS signals and decentralized Attention.

image

2. Reprogram decentralized Attention into the centralized CoTAR block.

image

3. TeCh: a unified CoTAR-based framework that captures temporal, channel, or both via adaptive tokenization.

image

4. Improved effectiveness, higher efficiency, and stronger robustness.

image

Usage

  1. Install requirements.
pip install -r requirements.txt
  1. Prepare data. You can download all datasets from Medformer. All the datasets are well pre-processed (except for the TDBrain dataset, which requires permission first) and can be used easily thanks to their efforts. Then, place all datasets under the folder
./dataset
  1. Train the model. We provide the experiment scripts of all benchmarks under the folder
./scripts
  1. For example, you can use the command line below to get the result of APAVA. The whole training history is under the './logs' folder.
bash ./scripts/APAVA.sh

Citation

If you find this repo helpful, please cite our paper.

@inproceedings{
yu2026tech,
title={Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series},
author={Guoqi Yu and Juncheng Wang and Chen Yang and Jing Qin and Angelica I Aviles-Rivero and Shujun Wang},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=oZJFY2BQt2}
}

Acknowledgement

This project is built on the code in the repo Medformer. Thanks a lot for their amazing work!

Please also star their project and cite their paper if you find this repo useful.

@article{wang2024medformer,
  title={Medformer: A multi-granularity patching transformer for medical time-series classification},
  author={Wang, Yihe and Huang, Nan and Li, Taida and Yan, Yujun and Zhang, Xiang},
  journal={Advances in Neural Information Processing Systems},
  volume={37},
  pages={36314--36341},
  year={2024}
}

About

Official implementation of ICLR26 (Oral): Decentralized Attention Fails Centralized Signals: Rethink Transformers for Medical Time Series

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors