Skip to content

Deployment and Maintenance Module English PR #575

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
qiaojialin merged 1 commit into from
Feb 19, 2025
Merged

Deployment and Maintenance Module English PR #575

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
349 changes: 349 additions & 0 deletions src/UserGuide/Master/Table/Deployment-and-Maintenance/Cluster-Deployment_apache.md
View file
Open in desktop

Large diffs are not rendered by default.

486 changes: 248 additions & 238 deletions ...UserGuide/Master/Table/Deployment-and-Maintenance/Cluster-Deployment_timecho.md
View file
Open in desktop

Large diffs are not rendered by default.

102 changes: 59 additions & 43 deletions src/UserGuide/Master/Table/Deployment-and-Maintenance/Database-Resources.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,50 +19,50 @@

-->
# Database Resources
## CPU
## 1 CPU
<table style="text-align: center;">
<tbody>
<tr>
<th rowspan="2">Number of timeseries (frequency<=1HZ)</th>
<th rowspan="2">Number of timeseries (frequency&lt;=1HZ)</th>
<th rowspan="2">CPU</th>
<th colspan="3">Number of nodes</th>
</tr>
<tr>
<th>standalone mode</th>
<th>Double active</th>
<th>standalone</th>
<th>Dual-Active</th>
<th>Distributed</th>
</tr>
<tr>
<td>Within 100000</td>
<td>2core-4core</td>
<td>2-4 cores</td>
<td>1</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>Within 300000</td>
<td>4core-8core</td>
<td>4-8 cores</td>
<td>1</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>Within 500000</td>
<td>8core-26core</td>
<td>8-16 cores</td>
<td>1</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>Within 1000000</td>
<td>16core-32core</td>
<td>16-32 cores</td>
<td>1</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>Within 2000000</td>
<td>32core-48core</td>
<td>32-48 cores</td>
<td>1</td>
<td>2</td>
<td>3</td>
Expand All @@ -81,50 +81,50 @@
</tbody>
</table>

## Memory
## 2 Memory
<table style="text-align: center;">
<tbody>
<tr>
<th rowspan="2">Number of timeseries (frequency<=1HZ)</th>
<th rowspan="2">Number of timeseries (frequency&lt;=1HZ)</th>
<th rowspan="2">Memory</th>
<th colspan="3">Number of nodes</th>
</tr>
<tr>
<th>standalone mode</th>
<th>Double active</th>
<th>standalone</th>
<th>Dual-Active</th>
<th>Distributed</th>
</tr>
<tr>
<td>Within 100000</td>
<td>4G-8G</td>
<td>4-8G</td>
<td>1</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>Within 300000</td>
<td>12G-32G</td>
<td>12-32G</td>
<td>1</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>Within 500000</td>
<td>24G-48G</td>
<td>24-48G</td>
<td>1</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>Within 1000000</td>
<td>32G-96G</td>
<td>32-96G</td>
<td>1</td>
<td>2</td>
<td>3</td>
</tr>
<tr>
<td>Within 2000000</td>
<td>64G-128G</td>
<td>64-128G</td>
<td>1</td>
<td>2</td>
<td>3</td>
Expand All @@ -143,19 +143,23 @@
</tbody>
</table>

## Storage (Disk)
### Storage space
Calculation formula: Number of measurement points * Sampling frequency (Hz) * Size of each data point (Byte, different data types may vary, see table below) * Storage time (seconds) * Number of copies (usually 1 copy for a single node and 2 copies for a cluster) ÷ Compression ratio (can be estimated at 5-10 times, but may be higher in actual situations)
## 3 Storage (Disk)
### 3.1 Storage space
Calculation Formula:

```Plain
Storage Space = Number of Measurement Points * Sampling Frequency (Hz) * Size of Each Data Point (Bytes, see the table below) * Storage Duration * Replication Factor / Compression Ratio
```

Data Point Size Calculation Table:

<table style="text-align: center;">
<tbody>
<tr>
<th colspan="4">Data point size calculation</th>
</tr>
<tr>
<th>data type</th>
<th>Data Type</th>
<th>Timestamp (Bytes)</th>
<th> Value (Bytes)</th>
<th> Total size of data points (in bytes)
<th> Total Data Point Size (Bytes)
</th>
</tr>
<tr>
Expand All @@ -165,36 +169,48 @@ Calculation formula: Number of measurement points * Sampling frequency (Hz) * Si
<td>9</td>
</tr>
<tr>
<td> INT32/FLOAT</td>
<td> INT32 / FLOAT (Single Precision)</td>
<td>8</td>
<td>4</td>
<td>12</td>
</tr>
<tr>
<td>INT64/DOUBLE</td>
<td>INT64 / DOUBLE (Double Precision)</td>
<td>8</td>
<td>8</td>
<td>16</td>
</tr>
<tr>
<td>TEXT</td>
<td>TEXT (String)</td>
<td>8</td>
<td>The average is a</td>
<td>Average = a</td>
<td>8+a</td>
</tr>
</tbody>
</table>
Example:

- Scenario: 1,000 devices, 100 measurement points per device, i.e. 100,000 sequences in total. Data type is INT32. Sampling frequency is 1Hz (once per second). Storage duration is 1 year. Replication factor is 3.
- Full Calculation:
```Plain
1,000 devices * 100 measurement points * 12 bytes per data point * 86,400 seconds per day * 365 days per year * 3 replicas / 10 compression ratio = 11 TB
```
- Simplified Calculation:
```Plain
1,000 * 100 * 12 * 86,400 * 365 * 3 / 10 = 11 TB
```
### 3.2 Storage Configuration

- For systems with > 10 million measurement points or high query loads, SSD is recommended.

## 4 Network (NIC)
When the write throughput does not exceed 10 million points per second, a gigabit network card is required. When the write throughput exceeds 10 million points per second, a 10-gigabit network card is required.

| **Write** **Throughput** **(Data Points/Second)** | **NIC** **Speed** |
| ------------------------------------------------- | -------------------- |
| < 10 million | 1 Gbps (Gigabit) |
| ≥ 10 million | 10 Gbps (10 Gigabit) |

## 5 Additional Notes

Example: 1000 devices, each with 100 measurement points, a total of 100000 sequences, INT32 type. Sampling frequency 1Hz (once per second), storage for 1 year, 3 copies.
- Complete calculation formula: 1000 devices * 100 measurement points * 12 bytes per data point * 86400 seconds per day * 365 days per year * 3 copies / 10 compression ratio / 1024 / 1024 / 1024 / 1024 =11T
- Simplified calculation formula: 1000 * 100 * 12 * 86400 * 365 * 3 / 10 / 1024 / 1024 / 1024 / 1024 =11T
### Storage Configuration
If the number of nodes is over 10000000 or the query load is high, it is recommended to configure SSD
## Network (Network card)
If the write throughput does not exceed 10 million points/second, configure 1Gbps network card. When the write throughput exceeds 10 million points per second, a 10Gbps network card needs to be configured.
| **Write throughput (data points per second)** | **NIC rate** |
| ------------------- | ------------- |
| <10 million | 1Gbps |
| >=10 million | 10Gbps |
## Other instructions
IoTDB has the ability to scale up clusters in seconds, and expanding node data does not require migration. Therefore, you do not need to worry about the limited cluster capacity estimated based on existing data. In the future, you can add new nodes to the cluster when you need to scale up.
- IoTDB supports second-level cluster scaling . Data migration is not required when adding new nodes, so there is no need to worry about limited cluster capacity based on current data estimates. You can add new nodes to the cluster when scaling is needed in the future.
52 changes: 52 additions & 0 deletions src/UserGuide/Master/Table/Deployment-and-Maintenance/Deployment-form_apache.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
<!--

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

-->
# Deployment-Mode_apache

IoTDB has two operation modes: standalone mode and cluster mode.

## 1 Standalone Mode

An IoTDB standalone instance includes 1 ConfigNode and 1 DataNode, referred to as 1C1D.

- **Features**: Easy for developers to install and deploy, with lower deployment and maintenance costs, and convenient operation.
- **Applicable scenarios**: Situations with limited resources or where high availability is not a critical requirement, such as edge servers.
- **Deployment method**:[Stand-Alone Deployment](../Deployment-and-Maintenance/Stand-Alone-Deployment_apache.md)


## 2 Cluster Mode

An IoTDB cluster instance consists of 3 ConfigNodes and no fewer than 3 DataNodes, typically 3 DataNodes, referred to as 3C3D. In the event of partial node failures, the remaining nodes can still provide services, ensuring high availability of the database service, and the database performance can be improved with the addition of nodes.

- **Features**: High availability and scalability, with the ability to enhance system performance by adding DataNodes.
- **Applicable scenarios**: Enterprise-level application scenarios that require high availability and reliability.
- **Deployment method**: [Cluster Deployment](../Deployment-and-Maintenance/Cluster-Deployment_apache.md)

## 3 Summary of Features

| **Dimension** | **Stand-Alone Mode** | **Cluster Mode** |
| :-------------------------- | :----------------------------------------------------- | :----------------------------------------------------------- |
| Applicable Scenario | Edge deployment, low requirement for high availability | High-availability business, disaster recovery scenarios, etc. |
| Number of Machines Required | 1 | ≥3 |
| Security and Reliability | Cannot tolerate single-point failure | High, can tolerate single-point failure |
| Scalability | Scalable by adding DataNodes to improve performance | Scalable by adding DataNodes to improve performance |
| Performance | Scalable with the number of DataNodes | Scalable with the number of DataNodes |

- The deployment steps for standalone mode and cluster mode are similar (adding ConfigNodes and DataNodes one by one), with the only differences being the number of replicas and the minimum number of nodes required to provide services.
63 changes: 63 additions & 0 deletions src/UserGuide/Master/Table/Deployment-and-Maintenance/Deployment-form_timecho.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
<!--

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

-->
# Deployment-Mode_apache

IoTDB has two operation modes: standalone mode and cluster mode.

## 1 Standalone Mode

An IoTDB standalone instance includes 1 ConfigNode and 1 DataNode, i.e., 1C1D.

- **Features**: Easy for developers to install and deploy, with low deployment and maintenance costs and convenient operations.
- **Use Cases**: Scenarios with limited resources or low high-availability requirements, such as edge servers.
- **Deployment Method**: [Stand-Alone Deployment](../Deployment-and-Maintenance/Stand-Alone-Deployment_timecho.md)

## 2 Dual-Active Mode

Dual-Active Deployment is a feature of TimechoDB, where two independent instances synchronize bidirectionally and can provide services simultaneously. If one instance stops and restarts, the other instance will resume data transfer from the breakpoint.

> An IoTDB Dual-Active instance typically consists of 2 standalone nodes, i.e., 2 sets of 1C1D. Each instance can also be a cluster.

- **Features**: The high-availability solution with the lowest resource consumption.
- **Use Cases**: Scenarios with limited resources (only two servers) but requiring high availability.
- **Deployment Method**: [Dual-Active Deployment](../Deployment-and-Maintenance/Dual-Active-Deployment_timecho.md)

## 3 Cluster Mode

An IoTDB cluster instance consists of 3 ConfigNodes and no fewer than 3 DataNodes, typically 3 DataNodes, i.e., 3C3D. If some nodes fail, the remaining nodes can still provide services, ensuring high availability of the database. Performance can be improved by adding DataNodes.

- **Features**: High availability, high scalability, and improved system performance by adding DataNodes.
- **Use Cases**: Enterprise-level application scenarios requiring high availability and reliability.
- **Deployment Method**: [Cluster Deployment](../Deployment-and-Maintenance/Cluster-Deployment_timecho.md)



## 4 Feature Summary

| **Dimension** | **Stand-Alone Mode** | **Dual-Active Mode** | **Cluster Mode** |
| :-------------------------- | :------------------------------------------------------- | :------------------------------------------------------ | :------------------------------------------------------ |
| Use Cases | Edge-side deployment, low high-availability requirements | High-availability services, disaster recovery scenarios | High-availability services, disaster recovery scenarios |
| Number of Machines Required | 1 | 2 | ≥3 |
| Security and Reliability | Cannot tolerate single-point failure | High, can tolerate single-point failure | High, can tolerate single-point failure |
| Scalability | Can expand DataNodes to improve performance | Each instance can be scaled as needed | Can expand DataNodes to improve performance |
| Performance | Can scale with the number of DataNodes | Same as one of the instances | Can scale with the number of DataNodes |

- The deployment steps for Stand-Alone Mode and Cluster Mode are similar (adding ConfigNodes and DataNodes one by one), with differences only in the number of replicas and the minimum number of nodes required to provide services.
Loading