Analysis: Database connection count validation for pool resizing in network partitions

[Copilot]

Problem

In multinode deployments, network partitions cause unnecessary pool expansion. When Server3 becomes unreachable due to partition (but remains operational), Servers 1-2 expand pools from 10→15 connections each. Since Server3 continues serving other clients with 10 connections, total connections exceed configured limit (10+15+15=40 vs 30 configured).

Solution Analysis

Query database for actual connection count before resizing pools. High count (≥85% threshold) indicates network partition; low count indicates true failure.

Documents Created

Core Analysis (documents/analysis/connection-count-validation/)

• DECISION_SUMMARY.md - Executive summary with go/no-go recommendation
• ANALYSIS.md - Deep dive covering:
  • 8 critical questions (multi-user scenarios, query failures, threshold selection)
  • 4 design opinions (simplicity, observability, escape hatches, XA/non-XA handling)
  • 5 risk assessments (complexity, permissions, heuristic limitations)
  • 4 alternative approaches evaluated and rejected
• DATABASE_CONNECTION_COUNT_QUERIES.md - SQL queries for PostgreSQL, MySQL, Oracle, SQL Server, DB2, H2, CockroachDB with permission requirements
• IMPLEMENTATION_GUIDE.md - Architecture, component specs, integration points, 6-week rollout plan

Key Findings

Feasibility: Supported across all major databases with minimal overhead (10-100ms per validation, executes only on cluster health changes)

Recommendation: Implement as opt-in feature (disabled by default) with fail-open behavior

Configuration:

ojp.pool.resize.validation.enabled=false
ojp.pool.resize.validation.connectionThreshold=0.85
ojp.pool.resize.validation.failureMode=PROCEED
ojp.pool.resize.validation.queryTimeout=5000

Example Query (PostgreSQL):

SELECT COUNT(*) FROM pg_stat_activity 
WHERE usename = CURRENT_USER AND pid != pg_backend_pid()
-- Result ≥26 (85% of 30) → Skip resize (partition)
-- Result <26 → Proceed with resize (failure)

Trade-offs

Benefits: Prevents connection limit violations, negligible overhead, works universally

Costs: ~500-1000 LOC, heuristic-based (not 100% accurate), database permissions needed for Oracle/SQL Server/DB2

Limitations: Single database user per deployment, doesn't cover all partition edge cases (documented)

Implementation Estimate

3-4 weeks: Core logic → Database support → Testing → Beta → Release

Original prompt

Do an analysis on what it would take to query the database for the number of connections open to the current user before resizing the ojp server pool in case of a loss of another ojp server node. For example if there was three ojp server nodes with 10 max connections each, currently if one goes down the other two raise to 15 connections max each to compensate, I want you to analyse what would be required to, before expanding or contracting the pool we query the DB and actually check the amount of current connetions allocated to the current user.
This is to deal with the situation of network partition where a client cannot see an ojp server because of a network partition but the server is up and potentially serving other clients, in such scenario we do not want to alter the pools, and by checking the db for the number of open connections to the current user we can figure it out if a change is actually required or not.
Document any questions, concerns, opinions, suggestions or criticisms about this design you might have.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.