-
-
-
-
-
+ 
+
+
+by ModelScope
+
+ English ｜ 中文
+
+
+
+
+
+
+
+ English Documentation ｜ 中文文档 +
+ +## ✨ What is Twinkle? +Twinkle✨ is a lightweight, client-server training framework engineered +with modular, high-cohesion interfaces. Whether you are executing locally +with `torchrun`, or scaling training across Ray clusters, +Twinkle✨ eliminates infrastructure friction by encapsulating +complex training logic into standardized APIs. Beyond simple +abstraction, Twinkle✨ serves as a robust backend and gateway to enable serverless Training-as-a-Service (TaaS). +It offers interfaces that constitute a _superset_ of [Tinker](https://thinkingmachines.ai/tinker/) APIs, +thereby making is possible to access a Twinkle✨ training service via Tinker client or native Twinkle✨-client +which offers more functionalities. + +🧩 Decoupled Architecture: Standardized Interfaces, backward compatible with Tinker APIs.
+
+ **Twinkle✨** features a decoupled **Client-Server architecture** designed for maximum flexibility.
+ The client-side provides two distinct integration paths:
+ * **Twinkle✨ Native:** A conforming API that mirrors the server-side interface for seamless end-to-end integration.
+ * **Tinker Compatibility:** Full support for the native Tinker API, enabling developers to leverage Twinkle✨’s backend using Tinker client.
+
+This dual-path design ensures access to Twinkle✨’s training services using Tinker API, with a simple modification of the Tinker base URL.
+
+## Multi-Tenancy
+
+**Twinkle✨** supports simultaneous multi-tenant training on a shared base model. Leveraging a **LoRA Pool + Tenant Application** architecture, Twinkle enables up to **N tenants** to train in parallel with complete isolation. This design offers unprecedented flexibility: from the model's perspective, each tenant's session is distinct, supporting heterogeneous configurations including unique **data padding strategies, optimizers, and loss functions**—all running concurrently on the same base model.
+
+*Note: This feature is currently optimized for [LoRA](https://github.com/huggingface/peft).*
+
+
+
+For example:
+
+- Tenant A: Load local private dataset locally, LoRA rank=8, using base model for SFT
+- Tenant B: Load open-source dataset from Hub remotely, LoRA rank=32, using base model for PT
+- Tenant C: Use base model for GRPO loss calculation, using Sampler for sampling
+- Tenant D: Use base model for logps inference
+
+These processes are executed concurrently on a single base model because the **Model and Sampler**
+are integrated as **task-agnostic components** within the Twinkle✨ ecosystem.
+Upon completion, checkpoints are automatically pushed to **ModelScope** or **HuggingFace** repositories
+(private by default). On the server side, Twinkle✨ provides a robust multi-tenant suite
+featuring **automated cluster management** and **dynamic scaling**, making it the
+foundation for building customizable, enterprise-grade training services.
+
+> As a modular framework, Twinkle✨ also supports remote temporary exclusive training, i.e., training in full-parameter mode.
+
+
+## 🛠️ Twinkle✨ Modular Ecosystem
+|
+ Dataset |
+
+ Template |
+
+ DataLoader |
+
+ Preprocessor |
+
+ InputProcessor |
+
|
+ Model |
+
+ Sampler |
+
+ Loss |
+
+ Metric |
+
+ Reward |
+
|
+ Advantage |
+
+ CheckpointEngine |
+
+ Patch |
+
+ Module |
+
+ Kernel |
+
|
+ Server |
+
+ Client |
+
+ Infra |
+
+ Plugin |
+
+ Hub |
+
+
+
+
+
+
+The Modelscope Community
+
+ 中文 ｜ English
+
+
+
+
+
+
+
+ English Documentation ｜ 中文文档 +
+ +
+🧩 松耦合架构 · 标准化接口
+🚀 多运行模式 · torchrun / Ray / HTTP
+🔌 多框架兼容 · Transformers / Megatron
+👥 多租户支持 · 单基座模型部署
+
+
+twinkle的架构由client和server两部分构成，其中client端包含两个使用可能性：
+
+1. 符合twinkle调用API的客户端，其API和server端完全相同
+2. 对原生Tinker API的兼容
+
+这使得开发者可以直接使用Tinker API调用twinkle部署起来的后端训练服务。
+
+## 多租户支持
+
+Twinkle支持多个租户同时使用一个基模型进行训练。这一行为目前仅限于[LoRA](https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora/config.py#L323)。
+Twinkle采用了LoRA池+租户申请的技术方案。这个方案可以支持最大N个租户并行训练互不干扰，并且在模型角度来看，不同租户的训练流程可能不同，在基模中的数据padding方式、optimizer、Loss类型也可以不同。
+
+
+
+例如：
+
+- 租户A：本机加载本地私有数据集，loRA rank=8，使用基模进行SFT
+- 租户B：使用远端加载Hub端开源数据集，LoRA rank=32，使用基模进行PT
+- 租户C：使用基模进行GRPO Loss计算，使用Sampler采样
+- 租户D：使用基模进行logps推理
+
+这些过程可以同时发生在一个基模上，因为模型、Sampler本质上也是twinkle组件的一部分，可以做到任务无关。训练完成后，支持checkpoint推送HuggingFace/ModelScope的模型仓库，默认为私有。twinkle提供了完整的多租户训练解决方案，在server端支持集群化管理和动态扩缩容，可以进行简单定制化后作为企业级服务。
+
+> 作为模块化框架，twinkle本身也可以支持远端临时的独占式训练，即全参数方式。
+
+
+## 支持的组件
+
+| Dataset 数据加载和预处理 |
+ Template 编码和解码 |
+ DataLoader 数据分发和batch化 |
+ Preprocessor 数据ETL |
+ InputProcessor 处理任务特定输入 |
+
| Model 大模型，支持多种框架 |
+ Sampler 采样器 |
+ Loss 残差 |
+ Metric 训练指标集合 |
+ Reward 奖励函数 |
+
| Advantage 优势函数 |
+ CheckpointEngine 权重同步 |
+ Patch 补丁，用于模型修复 |
+ Module 组件，例如Optimizer |
+ Kernel 算子 |
+
| Server 开启后端集群 |
+ Client 客户端代码 |
+ Infra 隔离ray和torchrun差异 |
+ Plugin 使用hub端组件 |
+ Hub 对接HF/MS网络库 |
+
+🧩 松耦合架构 · 标准化接口
+🚀 多运行模式 · torchrun / Ray / HTTP
+🔌 多框架兼容 · Transformers / Megatron
+👥 多租户支持 · 单基座模型部署
+