enhanced gas station registry consistency & maintenance features

[qrayven]

Description

This PR introduces several important features to improve the reliability, maintainability, and operational flexibility of the IOTA Gas Station. The changes focus on preventing registry inconsistencies, enabling safe configuration changes, and providing better multi-instance support.

Features

1. Cold Parameters System

The gas station now tracks "cold parameters" — configuration values that require a full coin registry rescan when changed. Currently, this includes the target_init_balance setting.

When the gas station starts, it compares the current configuration against previously stored cold parameters in Redis. If any changes are detected, the system requires explicit acknowledgment via the --allow-reinit flag before proceeding with a full rescan. This prevents accidental misconfiguration from silently corrupting the coin registry.

2. Storage Namespaces

Redis keys are now prefixed with a network-aware namespace that combines the fullnode host, port, and sponsor address. This prevents data collisions when running multiple gas station instances against different networks using the same Redis instance. For example, a devnet instance and a testnet instance can now safely share the same Redis database without interfering with each other.

3. Automatic Key Migration

To ensure backward compatibility, the system automatically detects and migrates data from the old key format to the new namespaced format on startup. The migration runs atomically via Lua scripts, preserving any existing data in the new namespace. The process logs detailed statistics about how many keys were migrated, skipped, or encountered errors.

4. Maintenance Mode

A distributed maintenance lock mechanism now protects registry operations across multiple gas station instances. When performing a full rescan or registry cleanup, the system first acquires an exclusive maintenance lock in Redis. While this lock is held, all coin reservation requests from any instance return a clear error message indicating the gas station is temporarily unavailable. The lock includes a time-to-live to prevent permanent deadlocks if an instance crashes during maintenance.

5. Consistency Checks

Before each scheduled rescan cycle, the gas station now performs a quick consistency check comparing the Redis registry state against the actual network state. It checks both the total coin count and aggregate balance, using configurable divergence thresholds (defaulting to 20%). If the registry has drifted significantly from reality — perhaps due to external transfers or registry corruption — the system logs detailed warnings with actionable remediation steps, suggesting the use of --allow-reinit and --delete-coin-registry flags.

6. Target Init Balance Validation

A new safety check validates that the configured target_init_balance is large enough relative to network gas costs. This prevents creating coins so small that gas fees would consume them entirely, which would lead to registry inconsistencies. If the validation fails, initialization stops with a clear error message explaining the minimum required balance.

7. Delete Registry CLI Flag

A new --delete-coin-registry flag allows operators to force a complete clean re-initialization of the coin registry. When combined with --allow-reinit, this triggers maintenance mode, deletes all existing registry data, and performs a fresh scan of on-chain coins. This is useful for recovering from corrupted registry states or when making significant configuration changes.

CLI Changes

Three new flags have been added:

• --allow-reinit (-a): Permits automatic reinitialization when cold parameters change
• --delete-coin-registry (-d): Forces deletion of the existing registry before starting (requires --allow-reinit)
• --ignore-locks: Ignores the maintenance lock and initialization lock.

Breaking Changes

• Redis key format changed: Keys now include a network namespace prefix. The automatic migration handles existing data transparently, but operators should be aware of this structural change.
• Maintenance mode behavior: During full rescans, coin reservation requests will temporarily fail with a descriptive error rather than silently waiting or returning stale data.

Testing

Added unit tests for consistency check calculations including edge cases with zero values and large numbers. Added integration tests covering various Redis migration scenarios including partial migrations and key conflicts. Added tests for the new aggregate coin stats fetching functionality used by consistency checks.

closes #144