From ef985b57d72361b77cede6a5666170742ede3e0b Mon Sep 17 00:00:00 2001 From: liyuheng Date: Mon, 13 Jan 2025 11:06:00 +0800 Subject: [PATCH 1/3] try --- src/zh/UserGuide/latest/User-Manual/Load-Balance.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/zh/UserGuide/latest/User-Manual/Load-Balance.md b/src/zh/UserGuide/latest/User-Manual/Load-Balance.md index 494a39e27..e98265910 100644 --- a/src/zh/UserGuide/latest/User-Manual/Load-Balance.md +++ b/src/zh/UserGuide/latest/User-Manual/Load-Balance.md @@ -39,7 +39,7 @@ IoTDB 是一个分布式数据库，数据的均衡分布对集群的磁盘空 1. 推荐仅在 IoTDB 1.3.3 以及更高版本使用 Region 迁移功能。 2. 仅在共识协议为 IoTConsensus、Ratis 时支持 Region 迁移（iotdb-system.properties中的`schema_region_consensus_protocol_class` 和 `data_region_consensus_protocol_class`）。 3. Region 迁移会占用硬盘和网络带宽等系统资源，推荐在低业务负载时进行。 -4. Region 迁移过程会占用 WAL 文件，阻塞本共识组 WAL 文件的删除，如果 WAL 文件总量达到`wal_file_size_threshold_in_byte`，则会阻塞写入。 +4. 在理想情况下，Region 迁移不影响用户侧读写。特殊情况下，Region 迁移可能阻塞写入，这种情况的具体鉴别与处理方式见使用说明。 ### 使用说明 @@ -99,3 +99,10 @@ IoTDB 是一个分布式数据库，数据的均衡分布对集群的磁盘空 It costs 0.003s ``` +- **阻塞写入**：IoTConsensus 的 region 迁移不直接阻塞写入，但由于过程中需要阻塞 WAL 文件的清理，如果 WAL 文件堆积达到阈值`wal_throttle_threshold_in_byte`，那么当前 DataNode 会暂停写入，直到 WAL 文件恢复到阈值以下。 + 如果迁移过程中观察到类似于`WAL 过大`的写入报错，可以将`wal_throttle_threshold_in_byte`调大到 500GB 或更大以允许继续写入。使用 SQL 语句： + ```plain + IoTDB> set configuration "wal_throttle_threshold_in_byte"="536870912000" + Msg: The statement is executed successfully. + ``` + From c4614582d198f4c0692f48489052b879b8aaece4 Mon Sep 17 00:00:00 2001 From: liyuheng Date: Mon, 13 Jan 2025 11:19:48 +0800 Subject: [PATCH 2/3] try --- src/zh/UserGuide/latest/User-Manual/Load-Balance.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/zh/UserGuide/latest/User-Manual/Load-Balance.md b/src/zh/UserGuide/latest/User-Manual/Load-Balance.md index e98265910..a9093e0be 100644 --- a/src/zh/UserGuide/latest/User-Manual/Load-Balance.md +++ b/src/zh/UserGuide/latest/User-Manual/Load-Balance.md @@ -99,8 +99,11 @@ IoTDB 是一个分布式数据库，数据的均衡分布对集群的磁盘空 It costs 0.003s ``` -- **阻塞写入**：IoTConsensus 的 region 迁移不直接阻塞写入，但由于过程中需要阻塞 WAL 文件的清理，如果 WAL 文件堆积达到阈值`wal_throttle_threshold_in_byte`，那么当前 DataNode 会暂停写入，直到 WAL 文件恢复到阈值以下。 - 如果迁移过程中观察到类似于`WAL 过大`的写入报错，可以将`wal_throttle_threshold_in_byte`调大到 500GB 或更大以允许继续写入。使用 SQL 语句： +- **阻塞写入**： + + IoTConsensus 的 region 迁移不直接阻塞写入，但由于过程中需要阻塞 WAL 文件的清理，如果 WAL 文件堆积达到阈值`wal_throttle_threshold_in_byte`，那么当前 DataNode 会暂停写入，直到 WAL 文件恢复到阈值以下。 + + 如果迁移过程中由于 WAL 达到阈值造成写入报错（例如报错信息为 The write is rejected because the wal directory size has reached the threshold），可以将`wal_throttle_threshold_in_byte`调大到 500GB 或更大以允许继续写入。使用 SQL 语句： ```plain IoTDB> set configuration "wal_throttle_threshold_in_byte"="536870912000" Msg: The statement is executed successfully. From e5d13e7c8b795c27718b512f5af605c4c7da368b Mon Sep 17 00:00:00 2001 From: liyuheng Date: Mon, 13 Jan 2025 11:29:11 +0800 Subject: [PATCH 3/3] copy --- .../Master/Tree/User-Manual/Load-Balance.md | 13 +++++++++++-- .../V2.0.1/Tree/User-Manual/Load-Balance.md | 13 +++++++++++-- src/UserGuide/latest/User-Manual/Load-Balance.md | 13 +++++++++++-- .../Master/Tree/User-Manual/Load-Balance.md | 11 ++++++++++- .../V2.0.1/Tree/User-Manual/Load-Balance.md | 11 ++++++++++- 5 files changed, 53 insertions(+), 8 deletions(-) diff --git a/src/UserGuide/Master/Tree/User-Manual/Load-Balance.md b/src/UserGuide/Master/Tree/User-Manual/Load-Balance.md index 17cfe0397..45ae3299b 100644 --- a/src/UserGuide/Master/Tree/User-Manual/Load-Balance.md +++ b/src/UserGuide/Master/Tree/User-Manual/Load-Balance.md @@ -39,8 +39,8 @@ Here is a schematic diagram of the region migration process : 1. It is recommended to only use the Region Migration feature on IoTDB 1.3.3 and higher versions. 2. Region migration is only supported when the consensus protocol is IoTConsus or Ratis (in iotdb system. properties, the `schema_region_consensus_protocol_class` and`data_region_consensus_protocol_class`). -3. Region migration will occupy system resources such as hard drives and network bandwidth, and it is recommended to perform it during low business loads. -4. The region migration process will occupy WAL files, blocking the deletion of WAL files in this consensus group. If the total number of WAL files reaches `wal_file_size_threshold_in_byte`, writing will be blocked. +3. Region migration consumes system resources such as disk space and network bandwidth. It is recommended to perform the migration during periods of low business load. +4. Under ideal circumstances, Region migration does not affect user-side read or write operations. In special cases, Region migration may block writes. For detailed identification and handling of such situations, please refer to the user guide. ### Instructions for use @@ -93,3 +93,12 @@ Here is a schematic diagram of the region migration process : Total line number = 3 It costs 0.003s ``` +- **Block Write**: + The region migration in IoTConsensus does not directly block writes. However, since the process requires suspending the cleanup of WAL files, if the accumulated WAL files exceed the threshold defined by wal_throttle_threshold_in_byte, the current DataNode will pause writing until the WAL files fall below the threshold. + + If a write error occurs during the migration due to the WAL exceeding the threshold (e.g., an error message like “The write is rejected because the wal directory size has reached the threshold”), you can increase the wal_throttle_threshold_in_byte value to 500GB or more to allow continued writing. Use the following SQL statement: + + ```plain + IoTDB> set configuration "wal_throttle_threshold_in_byte"="536870912000" + Msg: The statement is executed successfully. + ``` \ No newline at end of file diff --git a/src/UserGuide/V2.0.1/Tree/User-Manual/Load-Balance.md b/src/UserGuide/V2.0.1/Tree/User-Manual/Load-Balance.md index 17cfe0397..45ae3299b 100644 --- a/src/UserGuide/V2.0.1/Tree/User-Manual/Load-Balance.md +++ b/src/UserGuide/V2.0.1/Tree/User-Manual/Load-Balance.md @@ -39,8 +39,8 @@ Here is a schematic diagram of the region migration process : 1. It is recommended to only use the Region Migration feature on IoTDB 1.3.3 and higher versions. 2. Region migration is only supported when the consensus protocol is IoTConsus or Ratis (in iotdb system. properties, the `schema_region_consensus_protocol_class` and`data_region_consensus_protocol_class`). -3. Region migration will occupy system resources such as hard drives and network bandwidth, and it is recommended to perform it during low business loads. -4. The region migration process will occupy WAL files, blocking the deletion of WAL files in this consensus group. If the total number of WAL files reaches `wal_file_size_threshold_in_byte`, writing will be blocked. +3. Region migration consumes system resources such as disk space and network bandwidth. It is recommended to perform the migration during periods of low business load. +4. Under ideal circumstances, Region migration does not affect user-side read or write operations. In special cases, Region migration may block writes. For detailed identification and handling of such situations, please refer to the user guide. ### Instructions for use @@ -93,3 +93,12 @@ Here is a schematic diagram of the region migration process : Total line number = 3 It costs 0.003s ``` +- **Block Write**: + The region migration in IoTConsensus does not directly block writes. However, since the process requires suspending the cleanup of WAL files, if the accumulated WAL files exceed the threshold defined by wal_throttle_threshold_in_byte, the current DataNode will pause writing until the WAL files fall below the threshold. + + If a write error occurs during the migration due to the WAL exceeding the threshold (e.g., an error message like “The write is rejected because the wal directory size has reached the threshold”), you can increase the wal_throttle_threshold_in_byte value to 500GB or more to allow continued writing. Use the following SQL statement: + + ```plain + IoTDB> set configuration "wal_throttle_threshold_in_byte"="536870912000" + Msg: The statement is executed successfully. + ``` \ No newline at end of file diff --git a/src/UserGuide/latest/User-Manual/Load-Balance.md b/src/UserGuide/latest/User-Manual/Load-Balance.md index 17cfe0397..3453ea107 100644 --- a/src/UserGuide/latest/User-Manual/Load-Balance.md +++ b/src/UserGuide/latest/User-Manual/Load-Balance.md @@ -39,8 +39,8 @@ Here is a schematic diagram of the region migration process : 1. It is recommended to only use the Region Migration feature on IoTDB 1.3.3 and higher versions. 2. Region migration is only supported when the consensus protocol is IoTConsus or Ratis (in iotdb system. properties, the `schema_region_consensus_protocol_class` and`data_region_consensus_protocol_class`). -3. Region migration will occupy system resources such as hard drives and network bandwidth, and it is recommended to perform it during low business loads. -4. The region migration process will occupy WAL files, blocking the deletion of WAL files in this consensus group. If the total number of WAL files reaches `wal_file_size_threshold_in_byte`, writing will be blocked. +3. Region migration consumes system resources such as disk space and network bandwidth. It is recommended to perform the migration during periods of low business load. +4. Under ideal circumstances, Region migration does not affect user-side read or write operations. In special cases, Region migration may block writes. For detailed identification and handling of such situations, please refer to the user guide. ### Instructions for use @@ -93,3 +93,12 @@ Here is a schematic diagram of the region migration process : Total line number = 3 It costs 0.003s ``` +- **Block Write**: + The region migration in IoTConsensus does not directly block writes. However, since the process requires suspending the cleanup of WAL files, if the accumulated WAL files exceed the threshold defined by wal_throttle_threshold_in_byte, the current DataNode will pause writing until the WAL files fall below the threshold. + + If a write error occurs during the migration due to the WAL exceeding the threshold (e.g., an error message like “The write is rejected because the wal directory size has reached the threshold”), you can increase the wal_throttle_threshold_in_byte value to 500GB or more to allow continued writing. Use the following SQL statement: + + ```plain + IoTDB> set configuration "wal_throttle_threshold_in_byte"="536870912000" + Msg: The statement is executed successfully. + ``` diff --git a/src/zh/UserGuide/Master/Tree/User-Manual/Load-Balance.md b/src/zh/UserGuide/Master/Tree/User-Manual/Load-Balance.md index 494a39e27..87a23d2ce 100644 --- a/src/zh/UserGuide/Master/Tree/User-Manual/Load-Balance.md +++ b/src/zh/UserGuide/Master/Tree/User-Manual/Load-Balance.md @@ -39,7 +39,7 @@ IoTDB 是一个分布式数据库，数据的均衡分布对集群的磁盘空 1. 推荐仅在 IoTDB 1.3.3 以及更高版本使用 Region 迁移功能。 2. 仅在共识协议为 IoTConsensus、Ratis 时支持 Region 迁移（iotdb-system.properties中的`schema_region_consensus_protocol_class` 和 `data_region_consensus_protocol_class`）。 3. Region 迁移会占用硬盘和网络带宽等系统资源，推荐在低业务负载时进行。 -4. Region 迁移过程会占用 WAL 文件，阻塞本共识组 WAL 文件的删除，如果 WAL 文件总量达到`wal_file_size_threshold_in_byte`，则会阻塞写入。 +4. 在理想情况下，Region 迁移不影响用户侧读写。特殊情况下，Region 迁移可能阻塞写入，这种情况的具体鉴别与处理方式见使用说明。 ### 使用说明 @@ -99,3 +99,12 @@ IoTDB 是一个分布式数据库，数据的均衡分布对集群的磁盘空 It costs 0.003s ``` +- **阻塞写入**： + + IoTConsensus 的 region 迁移不直接阻塞写入，但由于过程中需要阻塞 WAL 文件的清理，如果 WAL 文件堆积达到阈值`wal_throttle_threshold_in_byte`，那么当前 DataNode 会暂停写入，直到 WAL 文件恢复到阈值以下。 + + 如果迁移过程中由于 WAL 达到阈值造成写入报错（例如报错信息为 The write is rejected because the wal directory size has reached the threshold），可以将`wal_throttle_threshold_in_byte`调大到 500GB 或更大以允许继续写入。使用 SQL 语句： + ```plain + IoTDB> set configuration "wal_throttle_threshold_in_byte"="536870912000" + Msg: The statement is executed successfully. + ``` diff --git a/src/zh/UserGuide/V2.0.1/Tree/User-Manual/Load-Balance.md b/src/zh/UserGuide/V2.0.1/Tree/User-Manual/Load-Balance.md index 494a39e27..99730de97 100644 --- a/src/zh/UserGuide/V2.0.1/Tree/User-Manual/Load-Balance.md +++ b/src/zh/UserGuide/V2.0.1/Tree/User-Manual/Load-Balance.md @@ -39,7 +39,7 @@ IoTDB 是一个分布式数据库，数据的均衡分布对集群的磁盘空 1. 推荐仅在 IoTDB 1.3.3 以及更高版本使用 Region 迁移功能。 2. 仅在共识协议为 IoTConsensus、Ratis 时支持 Region 迁移（iotdb-system.properties中的`schema_region_consensus_protocol_class` 和 `data_region_consensus_protocol_class`）。 3. Region 迁移会占用硬盘和网络带宽等系统资源，推荐在低业务负载时进行。 -4. Region 迁移过程会占用 WAL 文件，阻塞本共识组 WAL 文件的删除，如果 WAL 文件总量达到`wal_file_size_threshold_in_byte`，则会阻塞写入。 +4. 在理想情况下，Region 迁移不影响用户侧读写。特殊情况下，Region 迁移可能阻塞写入，这种情况的具体鉴别与处理方式见使用说明。 ### 使用说明 @@ -99,3 +99,12 @@ IoTDB 是一个分布式数据库，数据的均衡分布对集群的磁盘空 It costs 0.003s ``` +- **阻塞写入**： + + IoTConsensus 的 region 迁移不直接阻塞写入，但由于过程中需要阻塞 WAL 文件的清理，如果 WAL 文件堆积达到阈值`wal_throttle_threshold_in_byte`，那么当前 DataNode 会暂停写入，直到 WAL 文件恢复到阈值以下。 + + 如果迁移过程中由于 WAL 达到阈值造成写入报错（例如报错信息为 The write is rejected because the wal directory size has reached the threshold），可以将`wal_throttle_threshold_in_byte`调大到 500GB 或更大以允许继续写入。使用 SQL 语句： + ```plain + IoTDB> set configuration "wal_throttle_threshold_in_byte"="536870912000" + Msg: The statement is executed successfully. + ``` \ No newline at end of file