Enable db connection pooling

[golharam]

To keep the database connection alive when your APIServer is idle, you need to configure SQLAlchemy's connection pool parameters in core/db.py. The current configuration creates an engine without pool management settings, which causes stale connections.

Recommended Solution

Update create_engine() with these parameters:

from sqlmodel import create_engine, Session
from core.config import get_settings

# Connect to db with connection pool management
engine = create_engine(
    str(get_settings().SQLALCHEMY_DATABASE_URI),
    echo=False,
    pool_pre_ping=True,           # Test connections before using them
    pool_recycle=3600,             # Recycle connections after 1 hour (3600 seconds)
    pool_size=5,                   # Number of connections to maintain in pool
    max_overflow=10                # Max connections beyond pool_size
)

Key Parameters Explained

1. pool_pre_ping=True (Most Important)

   • Tests each connection before using it with a simple SELECT statement
   • Automatically discards stale/disconnected connections
   • Prevents "MySQL server has gone away" or "connection closed" errors
   • Minimal performance overhead

2. pool_recycle=3600

   • Recycles connections after 3600 seconds (1 hour)
   • Should be set LESS than your database's wait_timeout setting
   • For MySQL: default wait_timeout is 28800 seconds (8 hours)
   • For PostgreSQL: default idle_in_transaction_session_timeout varies

3. pool_size=5

   • Maintains 5 persistent connections in the pool
   • Adjust based on your application's concurrency needs

4. max_overflow=10

   • Allows up to 10 additional temporary connections during traffic spikes
   • Total max connections = pool_size + max_overflow = 15

Database-Specific Considerations

For PostgreSQL:

• pool_recycle should be less than server's idle_in_transaction_session_timeout
• Consider setting to 1800-3600 seconds

For MySQL:

• pool_recycle should be less than wait_timeout (typically 28800)
• Consider setting to 3600-7200 seconds

For SQLite:

• Connection pooling parameters have no effect (SQLite uses file-based connections)
• Current configuration is fine for SQLite

Testing the Fix

After implementing these changes, you can verify the solution:

1. Start your server and let it sit idle beyond the previous timeout period
2. Make a request to any endpoint that uses the database
3. The connection should work without "server has gone away" errors
4. Check logs for any connection-related warnings

The pool_pre_ping=True parameter is the most critical setting and will solve most idle connection timeout issues with minimal configuration changes.