Skip to content

Releases: Betterment/delayed

v2.0.2 - improved index creation lock-handling

05 Jan 22:09
Immutable release. Only release title and notes can be modified.
838c5a5

Choose a tag to compare

What's Changed

  • fix: Allow statement timeouts on index creation to retry (within migration wait timeout) by @smudge in #86

Full Changelog: v2.0.1...v2.0.2

v2.0.1

05 Jan 19:58
Immutable release. Only release title and notes can be modified.
d8a44e1

Choose a tag to compare

What's Changed

  • fix: Ensure that migration lock timeout loop does not bail after the first lock timeout. by @smudge in #85

Full Changelog: v2.0.0...v2.0.1

v2.0.0 - The "Database-Driven Development" Release!

19 Dec 22:16
Immutable release. Only release title and notes can be modified.
62df905

Choose a tag to compare

What's Changed

Numerous changes focused on index coverage and query performance went into this release.

At a high level, this release consists of:

  • New optional-but-encouraged DB indexes (available via rake delayed:install:migrations) that improve index coverage of all queries.
  • Adjustments to queries to improve selectivity and reduce cost (# of scanned/filtered rows).
  • First class support for HOT updates in PostgreSQL (during job pickup query).
  • Daylight savings time fixes in non-UTC time (:local) contexts. (#81)
  • Job timeouts can no longer be rescued from within the perform method as a StandardError (#66)

YMMV (and will depend heavily on the contents of your queue and the number of workers you run). In real-world at-scale testing against a PostgreSQL (RDS Aurora) backed queue, in an environment where the table had many millions of future-scheduled and/or failed rows (i.e. "non-claimable" rows), improvements were observed across the board:

  • The worker pickup query saw a 100-1000x query time improvement.
  • The "monitor" queries saw on net a 10-100x improvement.
  • Overall CPU usage and disk I/O of an active (but not back-logged) PostgreSQL queue dropped significantly.

The full list of changes is as follows:

  • test: Add "golden" tests for worker/monitor SQL by @smudge in #61
  • refactor: Add 'lock_timeout' & clarify that 'max_run_time' is a process-wide config by @smudge in #62
  • refactor: Remove 'ready_scope' and improve remaining scopes. by @smudge in #63
  • fix(tests): timing issue on test flake by @smudge in #65
  • fix: max_run_time timeout error should not be rescuable as a StandardError by @smudge in #66
  • test: Use snapshots for easier SQL testing/iteration. by @smudge in #67
  • test: Normalize DB versions used in CI by @smudge in #68
  • test: Snapshot test all EXPLAIN query plans by @smudge in #69
  • feat(perf): New indexes for job pickup & monitoring by @smudge in #70
  • fix: Ensure that upsert_index is reversible by @smudge in #72
  • fix(monitor): exclude failed jobs from metrics that shouldn't count them by @smudge in #73
  • fix(monitor): Obey lock timeout when reporting claimed/unclaimed rows by @smudge in #74
  • fix: Include 'claimed' as part of 'claimed_by' scope by @smudge in #75
  • perf(monitor): use 'attempts > 0' for better indexability by @smudge in #76
  • perf: HOT updates for PostgreSQL, index locked_at for everyone else. by @smudge in #77
  • perf(worker): avoid sequential scan during worker shutdown by @smudge in #79
  • perf: Improve selectivity of pickup query / "claimed" states by @smudge in #80
  • perf: remove legacy/unused "delayed_jobs_priority" index by @smudge in #78
  • fix: prevent DST + :local from breaking job backoff by @smudge in #81
  • fix: Improve upsert_index to only drop+rebuild if there is no matching & valid index by @smudge in #83
  • test(migration path): ensure that new queries are compatible with old index by @smudge in #84
  • perf(monitor): avoid querying the same thing twice, avoid seq scans by @smudge in #82
  • build: Ship 2.0.0! by @smudge in #71

Full Changelog: v1.2.1...v2.0.0

v1.2.1

06 Oct 19:23
33a485b

Choose a tag to compare

What's Changed

  • fix: Fix set_name callback, and improve spec coverage by @smudge in #60

Full Changelog: v1.2.0...v1.2.1

v1.2.0 - `delayed_jobs.name` !!!

03 Oct 14:16
8a35903

Choose a tag to compare

What's Changed

This release adds a delayed_jobs.name column that will be populated by default when each job is enqueued. While this column starts as nullable (to ensure zero downtime is required to make this change), it may someday be promoted to a required column (TBD).

The purpose of this column is twofold:

  1. Improve downstream developers' ability to manually operate the queue without needing to query deep into the handler column.
  2. Power richer observability & instrumentation tooling (e.g. top N job types in the table, etc)

Full Changelog: v1.1.0...v1.2.0

v1.1.0

12 Sep 21:13
d46a0fb

Choose a tag to compare

What's Changed

New Contributors

Full Changelog: v1.0.0...v1.1.0

v1.0.0 🎉

08 May 20:17
7e5a112

Choose a tag to compare

What's Changed

  • Delayed 1.0: Lifecycle & job cleanup improvements by @smudge in #55

Full Changelog: v0.8.0...v1.0.0

v0.8.0 - `Priority#to_f`

03 Apr 17:26
427bc30

Choose a tag to compare

What's Changed

  • 0.8.0 - Priority#to_f, plus some regression coverage by @smudge in #54

Full Changelog: v0.7.2...v0.8.0

v0.7.2 - thread callback hotfix!

02 Apr 22:22
2ea5f07

Choose a tag to compare

What's Changed

  • Upgrade betterlint to 1.18.0 by @dkubb in #52
  • [fix] Thread callback ordering by @smudge in #53

New Contributors

  • @dkubb made their first contribution in #52

Full Changelog: v0.7.1...v0.7.2

v0.7.1

24 Jan 22:11
f0fcc71

Choose a tag to compare

What's Changed

  • Add Ruby 3.3 to the CI matrix by @rzane in #50
  • Fix Rails 8.0 deprecation by @rzane in #51

Full Changelog: v0.7.0...v0.7.1