You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
The key has expired.
Using SCAN instead of KEYS at remove_queue (#1)
*** Why is this change necessary?
If you have a big collection of keys to be fetched and deleted from
Redis two problems will occur:
1. The server will be blocked due to the use of KEYS command since it
takes a long time to return for big collections;
2. Using the splat ("*") operator for a large array raises a
SystemStackError
Ultimately the queue is removed but the keys remain there:
```
2.3.8 :057 > Resque.size :test_queue
=> 300001
2.3.8 :058 > Resque.remove_queue :test_queue
SystemStackError: stack level too deep
from /root/project/vendor/bundle/ruby/2.3.0/gems/resque-unique_in_queue-2.0.1/lib/resque/uniqu
e_in_queue/queue.rb:68:in `cleanup'
from /root/project/vendor/bundle/ruby/2.3.0/gems/resque-unique_in_queue-2.0.1/lib/resque/uniqu
e_in_queue/resque_ext/resque.rb:35:in `remove_queue_with_unique_in_queue_cleanup'
from /root/project/lib/ext/hacks/resque.rb:115:in `remove_queue_rewrite'
from (irb):58
from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands/console.
rb:110:in `start'
from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands/console.
rb:9:in `start'
from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands/commands
_tasks.rb:68:in `console'
from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands/commands
_tasks.rb:39:in `run_command!'
from /root/project/vendor/bundle/ruby/2.3.0/gems/railties-4.2.11.1/lib/rails/commands.rb:17:in
`<top (required)>'
from script/rails:9:in `require'
from script/rails:9:in `<main>'
2.3.8 :059 > Resque.size :test_queue
=> 0
2.3.8 :060 > Resque.redis.keys("#{Resque::UniqueInQueue.configuration&.unique_in_queue_key_base}:queue:test_queue:job:*").count
=> 300001
```
*** How does it address the issue?
1. Replacing KEYS by SCAN command
2. Removing splat operator.
*** What side effects does this change have?
The removal process for large collections will take a lot longer:
```
irb(main):001:0> Benchmark.measure { Resque.remove_queue :test_queue }
=> #<Benchmark::Tms:0x000055b793b3cd60 @label="", @real=167.7889306979996, @cstime=0.0, @cutime=0.0, @stime=0.7600000000000002, @utime=11.500000000000002, @total=12.260000000000002>
```