Add thread multiplier configurations to worker/config_native.properties#218
Add thread multiplier configurations to worker/config_native.properties#218karthikeyann wants to merge 1 commit intomainfrom
Conversation
Added configuration options for thread multipliers in HTTP and connector executors.
|
@devavret @zoltan @majetideepak please review them, in context of
|
There was a problem hiding this comment.
Pull request overview
Adds thread multiplier configuration entries to the worker native config template to allow tuning executor thread counts relative to hardware concurrency.
Changes:
- Documented and introduced multiple
*-hw-multiplierthread settings inconfig_native.properties. - Enabled several of these multipliers by setting explicit values (e.g., HTTP + exchange executors, connector IO).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| # Each multiplier value affects number of threads used for each module. | ||
| # hw_concurrency x multiplier. | ||
| # 1.0 is default for each multiplier config. |
There was a problem hiding this comment.
The comment says "1.0 is default for each multiplier config", but later comments list different defaults (e.g., connector defaults to 0.0, driver to 4.0, spiller to 1.0). Please correct this line to reflect that defaults vary per setting (or update the per-setting default comments to match reality).
| # 1.0 is default for each multiplier config. | |
| # Default values differ per multiplier; see per-setting comments below. |
| # If 0.0 then connector preload/prefetch is disabled. | ||
| # 0.0 is default. | ||
| connector.num-io-threads-hw-multiplier=0.01 | ||
| # Number of threeads for Driver CPU executor |
There was a problem hiding this comment.
Typo in comment: "threeads" should be "threads".
| # Number of threeads for Driver CPU executor | |
| # Number of threads for Driver CPU executor |
| #driver.num-cpu-threads-hw-multiplier | ||
| # Number of threads for Spiller CPU executor | ||
| # If 0.0 then spilling is disabled. | ||
| # 1.0 is default. | ||
| #spiller.num-cpu-threads-hw-multiplier |
There was a problem hiding this comment.
The commented-out examples for driver/spiller multipliers omit an '=' and example/default value (unlike the connector example above). This makes it harder to discover the intended configuration value and increases the chance of misconfiguration; consider adding the explicit default/example values (e.g., "...=4.0" / "...=1.0") or otherwise standardizing how these properties are documented.
| #driver.num-cpu-threads-hw-multiplier | |
| # Number of threads for Spiller CPU executor | |
| # If 0.0 then spilling is disabled. | |
| # 1.0 is default. | |
| #spiller.num-cpu-threads-hw-multiplier | |
| #driver.num-cpu-threads-hw-multiplier=4.0 | |
| # Number of threads for Spiller CPU executor | |
| # If 0.0 then spilling is disabled. | |
| # 1.0 is default. | |
| #spiller.num-cpu-threads-hw-multiplier=1.0 |
|
What is the context of this? Were the high default values causing slowdowns? |
Added configuration options for thread multipliers in HTTP and connector executors.
There are total 10 hw-multiplier in https://github.com/prestodb/presto/blob/e1006483a1d6f5470a8d4229b7805cf08301d92e/presto-native-execution/presto_cpp/main/common/Configs.h#L198
Listed only 8 of them.
Enabled only 5 of them.