-
-
-
-
-
+ 
+
+
+by ModelScope
+
+ English ｜ 中文
+
+
+
+
+
+
+
+ English Documentation ｜ 中文文档 +
+ +## ✨ What is Twinkle? + +Twinkle✨ is a lightweight, client-server training framework engineered +with modular, high-cohesion interfaces. Whether you are executing locally +with `torchrun`, or scaling training across Ray clusters, +Twinkle✨ eliminates infrastructure friction by encapsulating +training logic into standardized APIs. Beyond simple +abstraction, Twinkle✨ serves as a robust backend and gateway to enable serverless Training-as-a-Service (TaaS). +It offers interfaces that constitute a _superset_ of [Tinker](https://thinkingmachines.ai/tinker/) APIs, +thereby making it possible to access a Twinkle✨ training service via Tinker client or native Twinkle✨ client +which offers more functionalities. + +🧩 Decoupled Architecture: Standardized Interfaces, backward compatible with Tinker APIs.
|
+
+## Installation
+
+### Install with package:
+
+```shell
+pip install 'twinkle-kit'
+```
+
+### Install from Source:
+
+```shell
+git clone https://github.com/modelscope/twinkle.git
+cd twinkle
+pip install -e .
+```
+
+## Tutorials
+
+| Training Type | Model Framework | Cookbook Path |
+| --------------------------------- | --------------- | ------------------------------------------------- |
+| FSDP finetuning | transformers | [Script](cookbook/transformers/fsdp2.py) |
+| FSDP MoE finetuning | transformers | [Script](cookbook/transformers/fsdp2_moe.py) |
+| ep/sp FSDP MoE finetuning | transformers | [Script](cookbook/transformers/ep_fsdp_qwen3_moe.py) |
+| EP MoE finetuning | transformers | [Script](cookbook/transformers/ep_fsdp_qwen3_moe.py) |
+| pp/tp/cp finetuning | megatron | [Script](cookbook/megatron/tp.py) |
+| pp/tp/cp MoE finetuning | megatron | [Script](cookbook/megatron/tp_moe.py) |
+| tinker client finetuning | megatron | [Script](cookbook/client/tinker/megatron) |
+| tinker client finetuning/sampling | transformers | [Script](cookbook/client/tinker/transformer) |
+| twinkle client finetuning | megatron | [Script](cookbook/client/twinkle/megatron) |
+| twinkle client finetuning | transformer | [Script](cookbook/client/twinkle/transformer) |
+
+## Changelog
+
+- 🎉2026-02-13 Initial version of Twinkle✨ released, including SFT/PT/RL support for text models and serverless training capabilities on [ModelScope](https://modelscope.cn).
+
+## Training as a Service on ModelScope
+
+We are rolling out training service built atop Twinkle✨ on ModelScope. It is currently in _Beta_. You may
+sign up for free access by joining the [Twinkle-Explorers](https://modelscope.cn/organization/twinkle-explorers) organization, and
+train via API endpoint `base_url=https://www.modelscope.cn/twinkle`. For more details, please refer to
+our [documentation](docs/source_en/Usage%20Guide/ModelScope-Official-Resources.md).
+
+## Supported Hardware
+
+| Hardware Environment | Notes |
+| -------------------- | ---------------------------------------------------------------- |
+| Nvidia GPUs | ✅ Support for BF16/Flash-Attn may be incomplete in earlier GPUs |
+| Ascend NPU | ✅ Some operators may not supported |
+| PPU | ✅ |
+| CPU | Supports partial components like dataset, dataloader |
+
+## Supported Models
+
+We will be adding support for more models as new models are released. The following table lists current models
+supported on Twinkle✨ framework.
+
+>[!Note]
+> For serverless training service accessed via `base_url=https://www.modelscope.cn/twinkle`, it currently supports
+> one training base at a time, and currently it is [Qwen3-30B-A3B-Instruct-2507](https://modelscope.cn/models/Qwen/Qwen3-30B-A3B-Instruct-2507).
+
+
+| Model Type | Model ID on [ModelScope](https://modelscope.cn) | Requires | Megatron Support | HF Model ID |
+| ------------------- |--------------------------------------------------------------------------------------------------------------------------| -------------------- | ---------------- | ---------------------------------------------------------------------------------------------------------- |
+| qwen3 series | [Qwen/Qwen3-0.6B-Base](https://modelscope.cn/models/Qwen/Qwen3-0.6B-Base)~32B | transformers>=4.51 | ✅ | [Qwen/Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base) |
+| qwen3_moe series | [Qwen/Qwen3-30B-A3B-Base](https://modelscope.cn/models/Qwen/Qwen3-30B-A3B-Base) | transformers>=4.51 | ✅ | [Qwen/Qwen3-30B-A3B-Base](https://huggingface.co/Qwen/Qwen3-30B-A3B-Base) |
+| | [Qwen/Qwen3-30B-A3B](https://modelscope.cn/models/Qwen/Qwen3-30B-A3B)~235B | transformers>=4.51 | ✅ | [Qwen/Qwen3-30B-A3B](https://huggingface.co/Qwen/Qwen3-30B-A3B) |
+| qwen2 series | [Qwen/Qwen2-0.5B-Instruct](https://modelscope.cn/models/Qwen/Qwen2-0.5B-Instruct) ~72B | transformers>=4.37 | ✅ | [Qwen/Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct) |
+| | [Qwen/Qwen2.5-0.5B-Instruct](https://modelscope.cn/models/Qwen/Qwen2.5-0.5B-Instruct)~72B | transformers>=4.37 | ✅ | [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) |
+| | [Qwen/Qwen2.5-0.5B](https://modelscope.cn/models/Qwen/Qwen2.5-0.5B)~72B | transformers>=4.37 | ✅ | [Qwen/Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) |
+| qwen2_moe series | [Qwen/Qwen1.5-MoE-A2.7B-Chat](https://modelscope.cn/models/Qwen/Qwen1.5-MoE-A2.7B-Chat) | transformers>=4.40 | ✅ | [Qwen/Qwen1.5-MoE-A2.7B-Chat](https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat) |
+| chatglm4 series | [ZhipuAI/glm-4-9b-chat](https://modelscope.cn/models/ZhipuAI/glm-4-9b-chat) | transformers>=4.42 | ✘ | [zai-org/glm-4-9b-chat](https://huggingface.co/zai-org/glm-4-9b-chat) |
+| | [ZhipuAI/LongWriter-glm4-9b](https://modelscope.cn/models/ZhipuAI/LongWriter-glm4-9b) | transformers>=4.42 | ✘ | [zai-org/LongWriter-glm4-9b](https://huggingface.co/zai-org/LongWriter-glm4-9b) |
+| glm_edge series | [ZhipuAI/glm-edge-1.5b-chat](https://modelscope.cn/models/ZhipuAI/glm-edge-1.5b-chat) | transformers>=4.46 | ✘ | [zai-org/glm-edge-1.5b-chat](https://huggingface.co/zai-org/glm-edge-1.5b-chat) |
+| | [ZhipuAI/glm-edge-4b-chat](https://modelscope.cn/models/ZhipuAI/glm-edge-4b-chat) | transformers>=4.46 | ✘ | [zai-org/glm-edge-4b-chat](https://huggingface.co/zai-org/glm-edge-4b-chat) |
+| internlm2 series | [Shanghai_AI_Laboratory/internlm2-1_8b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-1_8b) | transformers>=4.38 | ✘ | [internlm/internlm2-1_8b](https://huggingface.co/internlm/internlm2-1_8b) |
+| | [Shanghai_AI_Laboratory/internlm2-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-7b) | transformers>=4.38 | ✘ | [internlm/internlm2-chat-7b](https://huggingface.co/internlm/internlm2-chat-7b) |
+| deepseek_v1 | [deepseek-ai/deepseek-vl-7b-chat](https://modelscope.cn/models/deepseek-ai/deepseek-vl-7b-chat) | transformers>=4.39.4 | ✅ | —— |
+| | [deepseek-ai/DeepSeek-V2-Lite](https://modelscope.cn/models/deepseek-ai/DeepSeek-V2-Lite) | transformers>=4.39.3 | ✅ | [deepseek-ai/DeepSeek-V2-Lite](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite) |
+| | [deepseek-ai/DeepSeek-V2.5](https://modelscope.cn/models/deepseek-ai/DeepSeek-V2.5) | transformers>=4.39.3 | ✅ | [deepseek-ai/DeepSeek-V2.5](https://huggingface.co/deepseek-ai/DeepSeek-V2.5) |
+| | [deepseek-ai/DeepSeek-R1](https://modelscope.cn/models/deepseek-ai/DeepSeek-R1) | transformers>=4.39.3 | ✅ | [deepseek-ai/DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1) |
+| deepSeek-r1-distill | [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://modelscope.cn/models/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) ~32B | transformers>=4.37 | ✅ | [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) |
+
+For a more detailed model support list 👉 [Quick Start.md](https://github.com/modelscope/twinkle/blob/dev/docs/source/%E4%BD%BF%E7%94%A8%E6%8C%87%E5%BC%95/%E5%BF%AB%E9%80%9F%E5%BC%80%E5%A7%8B.md)
+
+## Sample Code
+
+### Train with Ray
+
+```python
+from peft import LoraConfig
+import twinkle
+from twinkle import DeviceMesh, DeviceGroup
+from twinkle.dataloader import DataLoader
+from twinkle.dataset import Dataset, DatasetMeta
+from twinkle.model import TransformersModel
+from twinkle.preprocessor import SelfCognitionProcessor
+
+device_group = [DeviceGroup(name='default',ranks=8,device_type='cuda')]
+device_mesh = DeviceMesh.from_sizes(fsdp_size=4, dp_size=2)
+# local for torchrun
+twinkle.initialize(mode='ray', groups=device_group, global_device_mesh=device_mesh)
+
+
+def train():
+ # to load model from Hugging Face, use 'hf://...'
+ base_model = 'ms://Qwen/Qwen2.5-7B-Instruct'
+ # 1000 samples
+ dataset = Dataset(dataset_meta=DatasetMeta('ms://swift/self-cognition', data_slice=range(1000)))
+ # Set template to prepare encoding
+ dataset.set_template('Template', model_id=base_model)
+ # Preprocess the dataset to standard format
+ dataset.map(SelfCognitionProcessor('twinkle LLM', 'ModelScope Community'))
+ # Encode dataset
+ dataset.encode()
+ # Global batch size = 8, for GPUs, so 1 sample per GPU
+ dataloader = DataLoader(dataset=dataset, batch_size=8, min_batch_size=8)
+ # Use a TransformersModel
+ model = TransformersModel(model_id=base_model, remote_group='default')
+
+ lora_config = LoraConfig(
+ r=8,
+ lora_alpha=32,
+ target_modules='all-linear'
+ )
+
+ # Add a lora to model, with name `default`
+ # Comment this to use full-parameter training
+ model.add_adapter_to_model('default', lora_config, gradient_accumulation_steps=2)
+ # Add Optimizer for lora `default`
+ model.set_optimizer(optimizer_cls='AdamW', lr=1e-4)
+ # Add LRScheduler for lora `default`
+ model.set_lr_scheduler(scheduler_cls='CosineWarmupScheduler', num_warmup_steps=5,
+ num_training_steps=len(dataloader))
+ for step, batch in enumerate(dataloader):
+ # Do forward and backward
+ model.forward_backward(inputs=batch)
+ # Step
+ model.clip_grad_and_step()
+ if step % 20 == 0:
+ # Print metric
+ metric = model.calculate_metric(is_training=True)
+ print(f'Current is step {step} of {len(dataloader)}, metric: {metric}')
+ model.save(f'last-checkpoint')
+
+
+if __name__ == '__main__':
+ train()
+```
+
+### Using Tinker-Like API
+
+```python
+import os
+from tqdm import tqdm
+from tinker import types
+from twinkle_client import init_tinker_compat_client
+from twinkle.dataloader import DataLoader
+from twinkle.dataset import Dataset, DatasetMeta
+from twinkle.preprocessor import SelfCognitionProcessor
+from twinkle.server.tinker.common import input_feature_to_datum
+
+base_model = 'ms://Qwen/Qwen3-30B-A3B-Instruct-2507'
+base_url='http://www.modelscope.cn/twinkle'
+api_key=os.environ.get('MODELSCOPE_TOKEN')
+
+# Use twinkle dataset to load the data
+dataset = Dataset(dataset_meta=DatasetMeta('ms://swift/self-cognition', data_slice=range(500)))
+dataset.set_template('Template', model_id=base_model, max_length=256)
+dataset.map(SelfCognitionProcessor('twinkle Model', 'twinkle Team'), load_from_cache_file=False)
+dataset.encode(batched=True, load_from_cache_file=False)
+dataloader = DataLoader(dataset=dataset, batch_size=8)
+
+# Initialize tinker client
+service_client = init_tinker_compat_client(base_url, api_key)
+training_client = service_client.create_lora_training_client(base_model=base_model[len('ms://'):], rank=16)
+
+# Training loop: use input_feature_to_datum to transfer the input format
+for epoch in range(3):
+ for step, batch in tqdm(enumerate(dataloader)):
+ input_datum = [input_feature_to_datum(input_feature) for input_feature in batch]
+
+ fwdbwd_future = training_client.forward_backward(input_datum, "cross_entropy")
+ optim_future = training_client.optim_step(types.AdamParams(learning_rate=1e-4))
+
+ fwdbwd_result = fwdbwd_future.result()
+ optim_result = optim_future.result()
+
+ training_client.save_state(f"twinkle-lora-{epoch}").result()
+```
+
+## Architecture Design
+
+
+
+ **Twinkle✨** features a decoupled **Client-Server architecture** designed for maximum flexibility.
+ The client-side provides two distinct integration paths:
+
+* **Twinkle✨ Native:** A conforming API that mirrors the server-side interface for seamless end-to-end integration.
+* **Tinker Compatibility:** Full support for the native Tinker API, enabling developers to leverage Twinkle✨’s backend using Tinker client.
+
+This dual-path design ensures access to Twinkle✨’s training services using Tinker API, with a simple modification of the Tinker base URL.
+
+## Multi-Tenancy
+
+**Twinkle✨** supports simultaneous multi-tenant training on a shared base model. Leveraging a **LoRA Pool + Tenant Application** architecture, Twinkle enables up to **N tenants** to train in parallel with complete isolation. This design offers unprecedented flexibility: from the model's perspective, each tenant's session is distinct, supporting heterogeneous configurations including unique **data padding strategies, optimizers, and loss functions**—all running concurrently on the same base model.
+
+*Note: This feature is currently optimized for [LoRA](https://github.com/huggingface/peft).*
+
+
+
+For example:
+
+- Tenant A: Load local private dataset locally, LoRA rank=8, using base model for SFT
+- Tenant B: Load open-source dataset from Hub remotely, LoRA rank=32, using base model for PT
+- Tenant C: Use base model for GRPO loss calculation, using Sampler for sampling
+- Tenant D: Use base model for logps inference
+
+These processes are executed concurrently on a single base model because the **Model and Sampler**
+are integrated as **task-agnostic components** within the Twinkle✨ ecosystem.
+Upon completion, checkpoints are automatically pushed to **ModelScope** or **HuggingFace** repositories
+(private by default). On the server side, Twinkle✨ provides a robust multi-tenant suite
+featuring **automated cluster management** and **dynamic scaling**, making it the
+foundation for building customizable, enterprise-grade training services.
+
+> As a modular framework, Twinkle✨ also supports remote temporary exclusive training, i.e., training in full-parameter mode.
+
+## 🛠️ Twinkle✨ Modular Ecosystem
+
+|
+ Dataset |
+
+ Template |
+
+ DataLoader |
+
+ Preprocessor |
+
+ InputProcessor |
+
|
+ Model |
+
+ Sampler |
+
+ Loss |
+
+ Metric |
+
+ Reward |
+
|
+ Advantage |
+
+ CheckpointEngine |
+
+ Patch |
+
+ Module |
+
+ Kernel |
+
|
+ Server |
+
+ Client |
+
+ Infra |
+
+ Plugin |
+
+ Hub |
+
+
+
+
+ModelScope
+
+ English ｜ 中文
+
+
+
+
+
+
+
|
+
+## 安装
+
+### 使用包安装：
+
+```shell
+pip install 'twinkle-kit'
+```
+
+### 从源码安装：
+
+```shell
+git clone https://github.com/modelscope/twinkle.git
+cd twinkle
+pip install -e .
+```
+
+## 教程
+
+| 训练类型 | 模型框架 | Cookbook 路径 |
+| ---------------------------- | -------- | ------------------------------------------------- |
+| FSDP 微调 | transformers | [脚本](cookbook/transformers/fsdp2.py) |
+| FSDP MoE 微调 | transformers | [脚本](cookbook/transformers/fsdp2_moe.py) |
+| EP MoE 微调 | transformers | [脚本](cookbook/transformers/ep_fsdp_qwen3_moe.py) |
+| pp/tp/cp 微调 | megatron | [脚本](cookbook/megatron/tp.py) |
+| pp/tp/cp MoE 微调 | megatron | [脚本](cookbook/megatron/tp_moe.py) |
+| tinker 客户端微调 | megatron | [脚本](cookbook/client/tinker/megatron) |
+| tinker 客户端微调/采样 | transformers | [脚本](cookbook/client/tinker/transformer) |
+| twinkle 客户端微调 | megatron | [脚本](cookbook/client/twinkle/megatron) |
+| twinkle 客户端微调 | transformer | [脚本](cookbook/client/twinkle/transformer) |
+
+## 更新日志
+
+- 🎉2026-02-13 Twinkle✨ 初始版本发布，包括对文本模型的 SFT/PT/RL 支持以及在 [ModelScope](https://modelscope.cn) 上的无服务器训练能力。
+
+## ModelScope 的训练服务
+
+我们正在 ModelScope 上推出基于 Twinkle✨ 构建的训练服务。目前处于 _Beta_ 阶段。你可以通过加入 [Twinkle-Explorers](https://modelscope.cn/organization/twinkle-explorers) 组织来注册免费访问，并通过 API 端点 `base_url=https://www.modelscope.cn/twinkle` 进行训练。更多详情请参阅我们的[文档](docs/source_zh/使用指引/训练服务.md)。
+
+## 支持的硬件
+
+| 硬件环境 | 备注 |
+| -------- | --------------------------------------------------------------- |
+| Nvidia GPU | ✅ 早期 GPU 对 BF16/Flash-Attn 的支持可能不完整 |
+| 昇腾 NPU | ✅ 部分算子可能不支持 |
+| PPU | ✅ |
+| CPU | 支持部分组件如 dataset、dataloader |
+
+## 支持的模型
+
+随着新模型的发布，我们将添加对更多模型的支持。下表列出了 Twinkle✨ 框架当前支持的模型。
+
+>[!注意]
+> 对于通过 `base_url=https://www.modelscope.cn/twinkle` 访问的无服务器训练服务，目前一次只支持一个训练基座，当前是 [Qwen3-30B-A3B-Instruct-2507](https://modelscope.cn/models/Qwen/Qwen3-30B-A3B-Instruct-2507)。
+
+
+| 模型类型 | [ModelScope](https://modelscope.cn) 上的模型 ID | 要求 | Megatron 支持 | HF 模型 ID |
+| ----------------- |--------------------------------------------------------------------------------------------------------------------------| -------------------- | -------------- | ---------------------------------------------------------------------------------------------------------- |
+| qwen3 系列 | [Qwen/Qwen3-0.6B-Base](https://modelscope.cn/models/Qwen/Qwen3-0.6B-Base)~32B | transformers>=4.51 | ✅ | [Qwen/Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base) |
+| qwen3_moe 系列 | [Qwen/Qwen3-30B-A3B-Base](https://modelscope.cn/models/Qwen/Qwen3-30B-A3B-Base) | transformers>=4.51 | ✅ | [Qwen/Qwen3-30B-A3B-Base](https://huggingface.co/Qwen/Qwen3-30B-A3B-Base) |
+| | [Qwen/Qwen3-30B-A3B](https://modelscope.cn/models/Qwen/Qwen3-30B-A3B)~235B | transformers>=4.51 | ✅ | [Qwen/Qwen3-30B-A3B](https://huggingface.co/Qwen/Qwen3-30B-A3B) |
+| qwen2 系列 | [Qwen/Qwen2-0.5B-Instruct](https://modelscope.cn/models/Qwen/Qwen2-0.5B-Instruct) ~72B | transformers>=4.37 | ✅ | [Qwen/Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct) |
+| | [Qwen/Qwen2.5-0.5B-Instruct](https://modelscope.cn/models/Qwen/Qwen2.5-0.5B-Instruct)~72B | transformers>=4.37 | ✅ | [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) |
+| | [Qwen/Qwen2.5-0.5B](https://modelscope.cn/models/Qwen/Qwen2.5-0.5B)~72B | transformers>=4.37 | ✅ | [Qwen/Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) |
+| qwen2_moe 系列 | [Qwen/Qwen1.5-MoE-A2.7B-Chat](https://modelscope.cn/models/Qwen/Qwen1.5-MoE-A2.7B-Chat) | transformers>=4.40 | ✅ | [Qwen/Qwen1.5-MoE-A2.7B-Chat](https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat) |
+| chatglm4 系列 | [ZhipuAI/glm-4-9b-chat](https://modelscope.cn/models/ZhipuAI/glm-4-9b-chat) | transformers>=4.42 | ✘ | [zai-org/glm-4-9b-chat](https://huggingface.co/zai-org/glm-4-9b-chat) |
+| | [ZhipuAI/LongWriter-glm4-9b](https://modelscope.cn/models/ZhipuAI/LongWriter-glm4-9b) | transformers>=4.42 | ✘ | [zai-org/LongWriter-glm4-9b](https://huggingface.co/zai-org/LongWriter-glm4-9b) |
+| glm_edge 系列 | [ZhipuAI/glm-edge-1.5b-chat](https://modelscope.cn/models/ZhipuAI/glm-edge-1.5b-chat) | transformers>=4.46 | ✘ | [zai-org/glm-edge-1.5b-chat](https://huggingface.co/zai-org/glm-edge-1.5b-chat) |
+| | [ZhipuAI/glm-edge-4b-chat](https://modelscope.cn/models/ZhipuAI/glm-edge-4b-chat) | transformers>=4.46 | ✘ | [zai-org/glm-edge-4b-chat](https://huggingface.co/zai-org/glm-edge-4b-chat) |
+| internlm2 系列 | [Shanghai_AI_Laboratory/internlm2-1_8b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-1_8b) | transformers>=4.38 | ✘ | [internlm/internlm2-1_8b](https://huggingface.co/internlm/internlm2-1_8b) |
+| | [Shanghai_AI_Laboratory/internlm2-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-7b) | transformers>=4.38 | ✘ | [internlm/internlm2-chat-7b](https://huggingface.co/internlm/internlm2-chat-7b) |
+| deepseek_v1 | [deepseek-ai/deepseek-vl-7b-chat](https://modelscope.cn/models/deepseek-ai/deepseek-vl-7b-chat) | transformers>=4.39.4 | ✅ | —— |
+| | [deepseek-ai/DeepSeek-V2-Lite](https://modelscope.cn/models/deepseek-ai/DeepSeek-V2-Lite) | transformers>=4.39.3 | ✅ | [deepseek-ai/DeepSeek-V2-Lite](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite) |
+| | [deepseek-ai/DeepSeek-V2.5](https://modelscope.cn/models/deepseek-ai/DeepSeek-V2.5) | transformers>=4.39.3 | ✅ | [deepseek-ai/DeepSeek-V2.5](https://huggingface.co/deepseek-ai/DeepSeek-V2.5) |
+| | [deepseek-ai/DeepSeek-R1](https://modelscope.cn/models/deepseek-ai/DeepSeek-R1) | transformers>=4.39.3 | ✅ | [deepseek-ai/DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1) |
+| deepSeek-r1-distill | [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://modelscope.cn/models/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) ~32B | transformers>=4.37 | ✅ | [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) |
+
+更详细的模型支持列表 👉 [快速开始.md](https://github.com/modelscope/twinkle/blob/dev/docs/source/%E4%BD%BF%E7%94%A8%E6%8C%87%E5%BC%95/%E5%BF%AB%E9%80%9F%E5%BC%80%E5%A7%8B.md)
+
+## 示例代码
+
+### 使用 Ray 训练
+
+```python
+from peft import LoraConfig
+import twinkle
+from twinkle import DeviceMesh, DeviceGroup
+from twinkle.dataloader import DataLoader
+from twinkle.dataset import Dataset, DatasetMeta
+from twinkle.model import TransformersModel
+from twinkle.preprocessor import SelfCognitionProcessor
+
+device_group = [DeviceGroup(name='default',ranks=8,device_type='cuda')]
+device_mesh = DeviceMesh.from_sizes(fsdp_size=4, dp_size=2)
+# local for torchrun
+twinkle.initialize(mode='ray', groups=device_group, global_device_mesh=device_mesh)
+
+
+def train():
+ # to load model from Hugging Face, use 'hf://...'
+ base_model = 'ms://Qwen/Qwen2.5-7B-Instruct'
+ # 1000 samples
+ dataset = Dataset(dataset_meta=DatasetMeta('ms://swift/self-cognition', data_slice=range(1000)))
+ # Set template to prepare encoding
+ dataset.set_template('Template', model_id=base_model)
+ # Preprocess the dataset to standard format
+ dataset.map(SelfCognitionProcessor('twinkle LLM', 'ModelScope Community'))
+ # Encode dataset
+ dataset.encode()
+ # Global batch size = 8, for GPUs, so 1 sample per GPU
+ dataloader = DataLoader(dataset=dataset, batch_size=8, min_batch_size=8)
+ # Use a TransformersModel
+ model = TransformersModel(model_id=base_model, remote_group='default')
+
+ lora_config = LoraConfig(
+ r=8,
+ lora_alpha=32,
+ target_modules='all-linear'
+ )
+
+ # Add a lora to model, with name `default`
+ # Comment this to use full-parameter training
+ model.add_adapter_to_model('default', lora_config, gradient_accumulation_steps=2)
+ # Add Optimizer for lora `default`
+ model.set_optimizer(optimizer_cls='AdamW', lr=1e-4)
+ # Add LRScheduler for lora `default`
+ model.set_lr_scheduler(scheduler_cls='CosineWarmupScheduler', num_warmup_steps=5,
+ num_training_steps=len(dataloader))
+ for step, batch in enumerate(dataloader):
+ # Do forward and backward
+ model.forward_backward(inputs=batch)
+ # Step
+ model.clip_grad_and_step()
+ if step % 20 == 0:
+ # Print metric
+ metric = model.calculate_metric(is_training=True)
+ print(f'Current is step {step} of {len(dataloader)}, metric: {metric}')
+ model.save(f'last-checkpoint')
+
+
+if __name__ == '__main__':
+ train()
+```
+
+### 使用类 Tinker API
+
+```python
+import os
+from tqdm import tqdm
+from tinker import types
+from twinkle_client import init_tinker_compat_client
+from twinkle.dataloader import DataLoader
+from twinkle.dataset import Dataset, DatasetMeta
+from twinkle.preprocessor import SelfCognitionProcessor
+from twinkle.server.tinker.common import input_feature_to_datum
+
+base_model = 'ms://Qwen/Qwen3-30B-A3B-Instruct-2507'
+base_url='http://www.modelscope.cn/twinkle'
+api_key=os.environ.get('MODELSCOPE_TOKEN')
+
+# Use twinkle dataset to load the data
+dataset = Dataset(dataset_meta=DatasetMeta('ms://swift/self-cognition', data_slice=range(500)))
+dataset.set_template('Template', model_id=base_model, max_length=256)
+dataset.map(SelfCognitionProcessor('twinkle Model', 'twinkle Team'), load_from_cache_file=False)
+dataset.encode(batched=True, load_from_cache_file=False)
+dataloader = DataLoader(dataset=dataset, batch_size=8)
+
+# Initialize tinker client
+service_client = init_tinker_compat_client(base_url, api_key)
+training_client = service_client.create_lora_training_client(base_model=base_model[len('ms://'):], rank=16)
+
+# Training loop: use input_feature_to_datum to transfer the input format
+for epoch in range(3):
+ for step, batch in tqdm(enumerate(dataloader)):
+ input_datum = [input_feature_to_datum(input_feature) for input_feature in batch]
+
+ fwdbwd_future = training_client.forward_backward(input_datum, "cross_entropy")
+ optim_future = training_client.optim_step(types.AdamParams(learning_rate=1e-4))
+
+ fwdbwd_result = fwdbwd_future.result()
+ optim_result = optim_future.result()
+
+ training_client.save_state(f"twinkle-lora-{epoch}").result()
+```
+
+## 架构设计
+
+
+
+**Twinkle✨** 采用解耦的**客户端-服务端架构**设计，以实现最大的灵活性。客户端提供两种不同的集成路径：
+
+* **Twinkle✨ 原生：** 符合服务端接口的 API，实现无缝的端到端集成。
+* **Tinker 兼容：** 完全支持原生 Tinker API，使开发者能够使用 Tinker 客户端来利用 Twinkle✨ 的后端。
+
+这种双路径设计确保可以使用 Tinker API 访问 Twinkle✨ 的训练服务，只需简单修改 Tinker 的 base URL。
+
+## 多租户
+
+**Twinkle✨** 支持在共享基础模型上同时进行多租户训练。利用 **LoRA 池 + 租户应用** 架构，Twinkle 能够让多达 **N 个租户** 在完全隔离的情况下并行训练。这种设计提供了前所未有的灵活性：从模型的角度来看，每个租户的会话是独立的，支持异构配置，包括独特的**数据填充策略、优化器和损失函数**——所有这些都在同一个基础模型上并发运行。
+
+*注意：此功能目前针对 [LoRA](https://github.com/huggingface/peft) 进行了优化。*
+
+
+
+例如：
+
+- 租户 A：在本地加载私有数据集，LoRA rank=8，使用基础模型进行 SFT
+- 租户 B：从 Hub 远程加载开源数据集，LoRA rank=32，使用基础模型进行 PT
+- 租户 C：使用基础模型进行 GRPO 损失计算，使用 Sampler 进行采样
+- 租户 D：使用基础模型进行 logps 推理
+
+这些过程在单个基础模型上并发执行，因为**模型和采样器**作为 Twinkle✨ 生态系统中的**任务无关组件**被集成。完成后，检查点会自动推送到 **ModelScope** 或 **HuggingFace** 仓库（默认为私有）。在服务端，Twinkle✨ 提供强大的多租户套件，具备**自动化集群管理**和**动态扩展**功能，使其成为构建可定制、企业级训练服务的基础。
+
+> 作为模块化框架，Twinkle✨ 也支持远程临时独占训练，即全参数模式训练。
+
+## 🛠️ Twinkle✨ 模块化生态系统
+
+|
+ Dataset |
+
+ Template |
+
+ DataLoader |
+
+ Preprocessor |
+
+ InputProcessor |
+
|
+ Model |
+
+ Sampler |
+
+ Loss |
+
+ Metric |
+
+ Reward |
+
|
+ Advantage |
+
+ CheckpointEngine |
+
+ Patch |
+
+ Module |
+
+ Kernel |
+
|
+ Server |
+
+ Client |
+
+ Infra |
+
+ Plugin |
+
+ Hub |
+