LeanPool

A lightweight, process-free resource pool for Ruby using concurrent-ruby. Inspired by Elixir's nimble_pool, LeanPool provides efficient resource management without per-resource processes, making it perfect for managing sockets, HTTP connections, ports, and other resources.

Author: Junaid Farooq | Repository: half-blood-labs/lean_pool

Features

🚀 Zero Overhead: Direct resource access without data copying between processes
🔒 Thread-Safe: Built on concurrent-ruby for reliable thread safety
⚡ Efficient: Lazy initialization and smart resource reuse
🎯 Simple API: Clean, Ruby-idiomatic interface
🔧 Flexible: Works with any resource type (sockets, connections, ports, etc.)
🎛️ Resource Selection Strategies: FIFO, LIFO, Random, and LRU (Least Recently Used)
⚡ Priority-Based Checkout: High-priority requests get resources first
♻️ Resource Recycling: Automatic resource replacement based on usage count or age
📡 Event Callbacks: Monitor pool lifecycle with before/after checkout, resource creation, etc.

Installation

Add this line to your application's Gemfile:

gem 'lean_pool'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install lean_pool

Usage

Real-World Examples

See examples/real_world_use_cases.rb for comprehensive examples showing:

Database connection pooling for web applications
HTTP API client pooling for microservices
File handle pooling for log processing
Redis connection pooling with automatic recycling
Priority-based resource access for background jobs
How concurrent-ruby enables thread safety
Performance benefits and use cases

Run the examples:

$ bundle exec ruby examples/real_world_use_cases.rb

Basic Example

require 'lean_pool'

# Create a pool of Redis connections
pool = LeanPool::Pool.new(size: 5) do
  Redis.new(host: "localhost", port: 6379)
end

# Use the pool
pool.checkout do |redis|
  redis.get("my_key")
  redis.set("my_key", "value")
end

HTTP Connection Pool

LeanPool includes a built-in HTTP connection pool for easy HTTP/HTTPS requests:

require 'lean_pool'

# Create an HTTP pool
http_pool = LeanPool::HTTPPool.new("api.example.com", 443, size: 10, use_ssl: true)

# Perform GET requests
response = http_pool.get("/users")
puts response[:status]  # => 200
puts response[:body]
puts response[:headers]

# Perform POST requests with JSON
response = http_pool.post(
  "/users",
  body: '{"name":"John","email":"john@example.com"}',
  headers: { "Content-Type" => "application/json" }
)

# With custom timeout
response = http_pool.get("/slow-endpoint", timeout: 30.0)

# Check pool statistics
stats = http_pool.stats
# => { size: 10, available: 8, in_use: 2, total: 10 }

# Shutdown when done
http_pool.shutdown

Custom HTTP Pool Implementation

You can also create your own HTTP pool wrapper:

require 'lean_pool'
require 'net/http'

class CustomHTTPPool
  def self.new_pool(host, port, size: 10)
    LeanPool::Pool.new(
      size: size,
      timeout: 5.0,
      lazy: true,
      pool_state: { host: host, port: port }
    ) do |state|
      Net::HTTP.new(state[:host], state[:port])
    end
  end

  def self.get(pool, path, opts = {})
    pool.checkout(timeout: opts[:timeout] || 5.0) do |http|
      http.start unless http.started?
      response = http.get(path)
      
      if response.code == "200"
        { status: response.code.to_i, body: response.body, headers: response.to_hash }
      else
        { remove: true }
      end
    end
  end
end

# Usage
pool = CustomHTTPPool.new_pool("api.example.com", 443)
result = CustomHTTPPool.get(pool, "/users")
puts result[:body]

With Custom State

pool = LeanPool::Pool.new(
  size: 10,
  pool_state: { database: "production", timeout: 30 }
) do |state|
  DatabaseConnection.new(
    database: state[:database],
    timeout: state[:timeout]
  )
end

pool.checkout do |db|
  db.query("SELECT * FROM users")
end

Pool Statistics

pool = LeanPool::Pool.new(size: 5) { Redis.new }

stats = pool.stats
# => { size: 5, available: 3, in_use: 2, total: 5 }

Shutdown and Reload

# Gracefully shutdown the pool
pool.shutdown do |resource|
  resource.close if resource.respond_to?(:close)
end

# Reload the pool (useful after forking)
pool.reload do |resource|
  resource.quit if resource.respond_to?(:quit)
end

Resource Selection Strategies

LeanPool supports multiple resource selection strategies:

# FIFO (First In, First Out) - default
pool = LeanPool::Pool.new(size: 5, strategy: :fifo) { Redis.new }

# LIFO (Last In, First Out)
pool = LeanPool::Pool.new(size: 5, strategy: :lifo) { Redis.new }

# Random selection
pool = LeanPool::Pool.new(size: 5, strategy: :random) { Redis.new }

# LRU (Least Recently Used)
pool = LeanPool::Pool.new(size: 5, strategy: :lru) { Redis.new }

# Change strategy at runtime
pool.strategy = :lru

Priority-Based Checkout

High-priority requests get resources first when the pool is exhausted:

pool = LeanPool::Pool.new(size: 1) { Connection.new }

# Low priority (higher number = lower priority)
pool.checkout(priority: 10) do |conn|
  # This will wait if pool is full
end

# High priority (lower number = higher priority)
pool.checkout(priority: 0) do |conn|
  # This will get the resource before lower priority requests
end

Resource Recycling

Automatically replace resources after a certain number of uses or age:

# Recycle after 100 uses
pool = LeanPool::Pool.new(size: 5, max_uses: 100) { Connection.new }

# Recycle after 1 hour
pool = LeanPool::Pool.new(size: 5, max_age: 3600) { Connection.new }

# Recycle when either condition is met
pool = LeanPool::Pool.new(
  size: 5,
  max_uses: 100,
  max_age: 3600
) { Connection.new }

Event Callbacks

Monitor pool lifecycle with event callbacks:

pool = LeanPool::Pool.new(size: 5) { Connection.new }

# Before checkout
pool.on(:before_checkout) do |pool|
  puts "Checking out from pool"
end

# After checkout
pool.on(:after_checkout) do |pool, resource|
  puts "Got resource: #{resource}"
end

# Resource created
pool.on(:on_resource_created) do |pool, resource|
  puts "New resource created: #{resource}"
end

# Resource removed
pool.on(:on_resource_removed) do |pool, resource|
  puts "Resource removed: #{resource}"
end

# Pool exhausted
pool.on(:on_pool_exhausted) do |pool|
  puts "Pool is full!"
end

# Multiple callbacks for the same event
pool.on(:before_checkout) { puts "Callback 1" }
pool.on(:before_checkout) { puts "Callback 2" }

Available events:

:before_checkout - Called before a resource is checked out
:after_checkout - Called after a resource is successfully checked out
:before_checkin - Called before a resource is checked in
:after_checkin - Called after a resource is checked in
:on_resource_created - Called when a new resource is created
:on_resource_removed - Called when a resource is removed from the pool
:on_pool_exhausted - Called when the pool is exhausted (all resources in use)

Error Handling

begin
  pool.checkout(timeout: 2.0) do |resource|
    resource.perform_operation
  end
rescue LeanPool::TimeoutError => e
  puts "Pool is busy, try again later"
rescue LeanPool::ShutdownError => e
  puts "Pool is shutdown"
rescue LeanPool::ResourceError => e
  puts "Failed to create resource: #{e.message}"
end

Comparison with Other Pooling Solutions

Feature	LeanPool	connection_pool	pond
Process-free	✅	❌	❌
Direct resource access	✅	❌	❌
Zero data copying	✅	❌	❌
Thread-safe	✅	✅	✅
Lazy initialization	✅	✅	✅
Built on concurrent-ruby	✅	❌	❌

When to Use LeanPool

Use LeanPool when:

You need direct access to resources (sockets, ports, HTTP connections)
You want to avoid data copying between processes
You're managing resources that don't need per-resource processes
You need high-performance resource pooling

Don't use LeanPool when:

You're managing processes (use process-based pools instead)
You need multiplexing (HTTP/2, etc.)
Resources require per-resource process isolation

Requirements

Ruby >= 3.3.0
concurrent-ruby >= 1.3.0

Development

After checking out the repo, run:

$ bundle install
$ bundle exec rspec

Running Tests Locally

Quick test run:

$ bundle exec rspec

With documentation format:

$ bundle exec rspec --format documentation

Run specific test files:

$ bundle exec rspec spec/lean_pool/pool_spec.rb
$ bundle exec rspec spec/lean_pool/http_pool_spec.rb
$ bundle exec rspec spec/lean_pool/errors_spec.rb

Simulate CI locally:

$ ./.github/workflows/test-local.sh

This script will:

Check Ruby version
Install dependencies
Validate syntax of all Ruby files
Run RuboCop linting
Run the full test suite
Display test summary

Using Rake:

$ bundle exec rake        # Runs tests + linting
$ bundle exec rake spec  # Runs only tests

Testing

The project includes comprehensive test coverage with over 222 test cases covering:

Pool initialization and configuration
Resource checkout and lifecycle management
Thread safety and concurrent operations
Timeout handling and error recovery
HTTP connection pooling (with WebMock for reliable testing)
Resource selection strategies (FIFO, LIFO, Random, LRU)
Priority-based checkout
Resource recycling (max_uses, max_age)
Event callbacks
Integration scenarios and real-world use cases
Edge cases and boundary conditions

Run the full test suite:

$ bundle exec rspec

Project Status

✅ CI/CD: Automated testing with GitHub Actions (testing on Ruby 3.3, 4.0)
✅ Test Coverage: Comprehensive test suite with 222+ test cases
✅ Documentation: Complete YARD documentation for all modules
✅ Thread Safety: Built on concurrent-ruby for reliable concurrency
✅ Production Ready: Suitable for production use
✅ Code Quality: RuboCop linting and syntax validation
✅ Advanced Features: Priority-based checkout, resource strategies, recycling, and event callbacks

CI/CD Status

The project uses GitHub Actions for continuous integration:

Test Matrix: Tests run on Ruby 3.3 and 4.0
Linting: RuboCop code quality checks
Syntax Validation: Automatic Ruby syntax checking
Status Badge:

View the latest CI runs: GitHub Actions

Status: The gem is now published on RubyGems.org! Badges will automatically update to show download counts and build status.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/half-blood-labs/lean_pool.

Author

Junaid Farooq

Email: fjunaid252@gmail.com
GitHub: @half-blood-labs

License

The gem is available as open source under the terms of the MIT License.

Inspiration

This gem is inspired by nimble_pool from the Elixir ecosystem, adapted for Ruby's concurrency model using concurrent-ruby.

Name		Name	Last commit message	Last commit date
Latest commit History 51 Commits
.github		.github
examples		examples
lib		lib
spec		spec
.gitignore		.gitignore
.rspec		.rspec
.rspec_status		.rspec_status
.rubocop.yml		.rubocop.yml
CHANGELOG.md		CHANGELOG.md
Gemfile		Gemfile
LICENSE.txt		LICENSE.txt
QUICKSTART.md		QUICKSTART.md
README.md		README.md
Rakefile		Rakefile
lean_pool.gemspec		lean_pool.gemspec

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

LeanPool

Features

Installation

Usage

Real-World Examples

Basic Example

HTTP Connection Pool

Custom HTTP Pool Implementation

With Custom State

Pool Statistics

Shutdown and Reload

Resource Selection Strategies

Priority-Based Checkout

Resource Recycling

Event Callbacks

Error Handling

Comparison with Other Pooling Solutions

When to Use LeanPool

Requirements

Development

Running Tests Locally

Testing

Project Status

CI/CD Status

Contributing

Author

License

Inspiration

About

Uh oh!

Releases

Packages

Languages

License

half-blood-labs/lean_pool

Folders and files

Latest commit

History

Repository files navigation

LeanPool

Features

Installation

Usage

Real-World Examples

Basic Example

HTTP Connection Pool

Custom HTTP Pool Implementation

With Custom State

Pool Statistics

Shutdown and Reload

Resource Selection Strategies

Priority-Based Checkout

Resource Recycling

Event Callbacks

Error Handling

Comparison with Other Pooling Solutions

When to Use LeanPool

Requirements

Development

Running Tests Locally

Testing

Project Status

CI/CD Status

Contributing

Author

License

Inspiration

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages