Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added src/.vuepress/public/img/LargeModel01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/.vuepress/public/img/LargeModel02.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/.vuepress/public/img/LargeModel03.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/.vuepress/public/img/LargeModel04.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/.vuepress/public/img/LargeModel05.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/.vuepress/public/img/LargeModel06.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/.vuepress/public/img/LargeModel07.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/.vuepress/public/img/LargeModel08.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 9 additions & 1 deletion src/.vuepress/sidebar/V2.0.x/en-Tree.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,6 @@ export const enSidebar = {
children: [
{ text: 'Data Sync', link: 'Data-Sync_apache' },
{ text: 'Data Subscription', link: 'Data-subscription' },
{ text: 'AI Capability', link: 'AINode_apache' },
{
text: 'Security Management',
collapsible: true,
Expand Down Expand Up @@ -129,6 +128,15 @@ export const enSidebar = {
},
],
},
{
text: 'AI capability',
collapsible: true,
prefix: 'AI-capability/',
children: [
{ text: 'AINode', link: 'AINode_apache' },
{ text: 'TimeSeries Large Model', link: 'TimeSeries-Large-Model' },
],
},
{
text: 'Tools System',
collapsible: true,
Expand Down
10 changes: 9 additions & 1 deletion src/.vuepress/sidebar/V2.0.x/zh-Tree.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,6 @@ export const zhSidebar = {
children: [
{ text: '数据同步', link: 'Data-Sync_apache' },
{ text: '数据订阅', link: 'Data-subscription' },
{ text: 'AI能力', link: 'AINode_apache' },
{
text: '安全管理',
collapsible: true,
Expand Down Expand Up @@ -116,6 +115,15 @@ export const zhSidebar = {
},
],
},
{
text: 'AI 能力',
collapsible: true,
prefix: 'AI-capability/',
children: [
{ text: 'AINode', link: 'AINode_apache' },
{ text: '时序大模型', link: 'TimeSeries-Large-Model' },
],
},
{
text: '工具体系',
collapsible: true,
Expand Down
10 changes: 9 additions & 1 deletion src/.vuepress/sidebar_timecho/V2.0.x/en-Tree.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,6 @@ export const enSidebar = {
children: [
{ text: 'Data Sync', link: 'Data-Sync_timecho' },
{ text: 'Data Subscription', link: 'Data-subscription' },
{ text: 'AI Capability', link: 'AINode_timecho' },
{
text: 'Security Management',
collapsible: true,
Expand Down Expand Up @@ -142,6 +141,15 @@ export const enSidebar = {
},
],
},
{
text: 'AI capability',
collapsible: true,
prefix: 'AI-capability/',
children: [
{ text: 'AINode', link: 'AINode_timecho' },
{ text: 'TimeSeries Large Model', link: 'TimeSeries-Large-Model' },
],
},
{
text: 'Tools System',
collapsible: true,
Expand Down
10 changes: 9 additions & 1 deletion src/.vuepress/sidebar_timecho/V2.0.x/zh-Tree.ts
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,6 @@ export const zhSidebar = {
children: [
{ text: '数据同步', link: 'Data-Sync_timecho' },
{ text: '数据订阅', link: 'Data-subscription' },
{ text: 'AI能力', link: 'AINode_timecho' },
{
text: '安全管理',
collapsible: true,
Expand Down Expand Up @@ -125,6 +124,15 @@ export const zhSidebar = {
},
],
},
{
text: 'AI 能力',
collapsible: true,
prefix: 'AI-capability/',
children: [
{ text: 'AINode', link: 'AINode_timecho' },
{ text: '时序大模型', link: 'TimeSeries-Large-Model' },
],
},
{
text: '工具体系',
collapsible: true,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

-->

# AI Capability
# AINode

AINode is the third internal node after ConfigNode and DataNode in Apache IoTDB, which extends the capability of machine learning analysis of time series by interacting with DataNode and ConfigNode of IoTDB cluster, supports the introduction of pre-existing machine learning models from the outside to be registered, and uses the registered models in the It supports the process of introducing existing machine learning models from outside for registration, and using the registered models to complete the time series analysis tasks on the specified time series data through simple SQL statements, which integrates the model creation, management and inference in the database engine. At present, we have provided machine learning algorithms or self-developed models for common timing analysis scenarios (e.g. prediction and anomaly detection).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

-->

# AI Capability
# AINode

AINode is the third internal node after ConfigNode and DataNode in Apache IoTDB, which extends the capability of machine learning analysis of time series by interacting with DataNode and ConfigNode of IoTDB cluster, supports the introduction of pre-existing machine learning models from the outside to be registered, and uses the registered models in the It supports the process of introducing existing machine learning models from outside for registration, and using the registered models to complete the time series analysis tasks on the specified time series data through simple SQL statements, which integrates the model creation, management and inference in the database engine. At present, we have provided machine learning algorithms or self-developed models for common timing analysis scenarios (e.g. prediction and anomaly detection).

Expand Down Expand Up @@ -444,45 +444,11 @@ Total line number = 4

In the result set, each row's label corresponds to the output of the anomaly detection model after inputting each group of 24 rows of data.

### 4.6 AINode-Timer Model Import Steps
1. Open the IoTDB CLI console and verify that the ConfigNode, DataNode, and AINode statuses are all ​Running.

Check command:
```sql
show cluster
```

![](/img/ainode-timer-1.png)

2. Model file storage path: It is recommended to place the model files in the same directory as the AINode installation package.
You may create a new folder to store model files.
3. Register the model

Use the following SQL statement:

```sql
create model <model_name> using uri <uri>
```

Example (for the Timer model):
### 4.6 TimeSeries Large Models Import Steps

```sql
create model Timer using uri <uri>
```

Note: When importing the Timer model, the name must be "Timer" (case-sensitive), otherwise it will not be recognized by the IoTDB visualization console.

![](/img/ainode-timer-2.png)

4. Verify model registration success

Check command:

```sql
show models
```
AINode currently supports a variety of time series large models. For deployment and usage, please refer to [TimeSeries Large Models](../AI-capability/TimeSeries-Large-Model)

![](/img/ainode-timer-3.png)

## 5. Privilege Management

Expand Down
117 changes: 117 additions & 0 deletions src/UserGuide/Master/Tree/AI-capability/TimeSeries-Large-Model
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
<!--

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

-->

# TimeSeries Large Model

## Introduction

A time series large model is a foundational model specifically designed for time series analysis. The IoTDB team has independently developed time series large models, which are pre-trained on massive time series data using technologies such as transformer structures. These models can understand and generate time series data across various domains and are applicable to applications like time series forecasting, anomaly detection, and time series imputation. Unlike traditional time series analysis techniques, time series large models possess the capability to extract universal features and provide technical services based on zero-shot analysis and fine-tuning for a wide range of analytical tasks.

The team's related technologies of time series large models have been published in top international machine learning conferences.

## Application Scenarios

- **Time Series Forecasting**: Provides forecasting services for time series data in industrial production, natural environments, and other areas, helping users to understand future trends in advance.
- **Data Imputation**: For missing segments in time series, perform context imputation to enhance the continuity and completeness of the dataset.
- **Anomaly Detection**: Utilizing regression analysis technology, monitor time series data in real-time and provide timely warnings for potential anomalies.

![](/img/LargeModel08.png)

## Timer Model

The Timer model not only demonstrates excellent few-shot generalization and multi-task adaptation capabilities but also gains a rich knowledge base through pre-training, endowing it with the universal capability to handle a variety of downstream tasks, featuring the following:

- **Generalization**: The model can be fine-tuned using a small number of samples to achieve leading predictive performance in the industry.
- **Versatility**: The model is designed flexibly to adapt to various task requirements and supports variable input and output lengths, enabling it to play a role in various application scenarios.
- **Scalability**: As the number of model parameters increases or the scale of pre-training data expands, the model's performance continues to improve, ensuring the model can optimize its predictive effects with the growth of time and data volume.

![](/img/LargeModel02.png)

## Timer-XL Model

Timer-XL is an upgraded version of Timer that further extends the network structure and achieves comprehensive breakthroughs in multiple dimensions:

- **Long Context Support**: This model breaks through the limitations of traditional time series forecasting models, supporting the processing of thousands of tokens (equivalent to tens of thousands of time points) of input, effectively addressing the bottleneck of context length.
- **Multi-variable Forecasting Scenario Coverage**: Supports a variety of forecasting scenarios, including non-stationary time series forecasting, multi-variable prediction tasks, and predictions involving covariates, meeting diverse business needs.
- **Large-scale Industrial Time Series Dataset**: Pre-trained using a massive industrial IoT time series dataset that has a large volume, excellent quality, and rich domain characteristics, covering energy, aerospace, steel, transportation, and more.


## Effect Demonstration

Time series large models can adapt to real time series data from various fields and scenarios, showing excellent processing effects in various tasks. Here are the real performances on different data:

**Time Series Forecasting:**

Utilizing the predictive capabilities of the time series large model, it can accurately predict the future trend of time series. As shown in the figure, the blue curve represents the predicted trend, and the red curve represents the actual trend, with the two curves highly matching.

![](/img/LargeModel03.png)

**Data Imputation:**:

Using the time series large model to perform predictive imputation for missing data segments.

![](/img/LargeModel04.png)


**Anomaly Detection:**:

Utilizing the time series large model to accurately identify anomalies that deviate significantly from the normal trend.

![](/img/LargeModel05.png)

## Deployment Usage

1. Open the IoTDB CLI console and verify that the ConfigNode, DataNode, and AINode statuses are all ​Running.

Check command:

```sql
show cluster
```

![](/img/ainode-timer-1.png)

2. Model file storage path: It is recommended to place the model files in the same directory as the AINode installation package.
You may create a new folder to store model files.

3. Register the model

Use the following SQL statement:

```sql
create model <model_name> using uri <uri>
```

Example (for the Timer model):

```sql
create model Timer using uri <uri>
```

4. Verify model registration success

Check command:

```sql
show models
```

![](/img/LargeModel06.png)
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

-->

# AI Capability
# AINode

AINode is the third internal node after ConfigNode and DataNode in Apache IoTDB, which extends the capability of machine learning analysis of time series by interacting with DataNode and ConfigNode of IoTDB cluster, supports the introduction of pre-existing machine learning models from the outside to be registered, and uses the registered models in the It supports the process of introducing existing machine learning models from outside for registration, and using the registered models to complete the time series analysis tasks on the specified time series data through simple SQL statements, which integrates the model creation, management and inference in the database engine. At present, we have provided machine learning algorithms or self-developed models for common timing analysis scenarios (e.g. prediction and anomaly detection).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

-->

# AI Capability
# AINode

AINode is the third internal node after ConfigNode and DataNode in Apache IoTDB, which extends the capability of machine learning analysis of time series by interacting with DataNode and ConfigNode of IoTDB cluster, supports the introduction of pre-existing machine learning models from the outside to be registered, and uses the registered models in the It supports the process of introducing existing machine learning models from outside for registration, and using the registered models to complete the time series analysis tasks on the specified time series data through simple SQL statements, which integrates the model creation, management and inference in the database engine. At present, we have provided machine learning algorithms or self-developed models for common timing analysis scenarios (e.g. prediction and anomaly detection).

Expand Down Expand Up @@ -444,45 +444,11 @@ Total line number = 4

In the result set, each row's label corresponds to the output of the anomaly detection model after inputting each group of 24 rows of data.

### 4.6 AINode-Timer Model Import Steps
1. Open the IoTDB CLI console and verify that the ConfigNode, DataNode, and AINode statuses are all ​Running.

Check command:
```sql
show cluster
```

![](/img/ainode-timer-1.png)

2. Model file storage path: It is recommended to place the model files in the same directory as the AINode installation package.
You may create a new folder to store model files.
3. Register the model

Use the following SQL statement:

```sql
create model <model_name> using uri <uri>
```

Example (for the Timer model):
### 4.6 TimeSeries Large Models Import Steps

```sql
create model Timer using uri <uri>
```

Note: When importing the Timer model, the name must be "Timer" (case-sensitive), otherwise it will not be recognized by the IoTDB visualization console.

![](/img/ainode-timer-2.png)

4. Verify model registration success

Check command:

```sql
show models
```
AINode currently supports a variety of time series large models. For deployment and usage, please refer to [TimeSeries Large Models](../AI-capability/TimeSeries-Large-Model)

![](/img/ainode-timer-3.png)

## 5. Privilege Management

Expand Down
Loading