Skip to content

Make the CI Queue reporters work in parallel #355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ChrisBr merged 2 commits into from
Aug 21, 2025
Merged

Make the CI Queue reporters work in parallel #355

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ef14fca
Make the CI Queue reporters work in parallel
gmcgibbon Aug 19, 2025
52229d6
Don't run fork test in truffleruby
gmcgibbon Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions ruby/lib/minitest/queue/order_reporter.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,29 @@
class Minitest::Queue::OrderReporter < Minitest::Reporters::BaseReporter
def initialize(options = {})
@path = options.delete(:path)
@file = nil
super
end

def start
@file = File.open(@path, 'w+')
super
file.truncate(0)
Copy link
Contributor

@davidstosik davidstosik Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is #start called only once before forking?

If not, is there a risk that you truncate the few first logs in the case the first tests ran very quickly?

Also, is it okay not to call super?

Copy link
Member Author

@gmcgibbon gmcgibbon Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is called once before forking, so the truncate will behave like opening the file in write mode and rewriting sections of it in a non-forked setup.

Copy link
Member Author

@gmcgibbon gmcgibbon Aug 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we call https://github.com/minitest/minitest/blob/7c907429e8e7ec7c3b6ee0c7045e67249f4ad505/lib/minitest.rb#L864

Probably not needed, but I see most of the time it is called. I'll add it back just in case.

end

def before_test(test)
super
@file.puts("#{test.class.name}##{test.name}")
@file.flush
file.puts("#{test.class.name}##{test.name}")
file.flush
end

def report
@file.close
file.close
end

private

def file
@file ||= File.open(@path, 'a+')
end
end

2 changes: 1 addition & 1 deletion ruby/test/integration/grind_redis_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def test_grind_command_runs_tests

def test_grind_max_time
grind_count = 1000000
timeout = RUBY_ENGINE == "truffleruby" ? 10 : 1
timeout = self.class.truffleruby? ? 10 : 1

start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
_, err = capture_subprocess_io do
Expand Down
2 changes: 1 addition & 1 deletion ruby/test/integration/minitest_redis_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -673,7 +673,7 @@ def test_test_data_time_reporter
.sort_by { |h| "#{h[:test_id]}##{h[:test_index]}" }
.first

start_delta = RUBY_ENGINE == "truffleruby" ? 15 : 5
start_delta = self.class.truffleruby? ? 15 : 5
assert_in_delta start_time.to_i, failure[:test_start_timestamp], start_delta, "start time"
assert_in_delta end_time.to_i, failure[:test_finish_timestamp], 5
assert failure[:test_finish_timestamp] > failure[:test_start_timestamp]
Expand Down
25 changes: 25 additions & 0 deletions ruby/test/minitest/queue/order_reporter_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,41 @@ class OrderReporterTest < Minitest::Test

def setup
@reporter = OrderReporter.new(path: log_path)
end

def test_start
@reporter.start
@reporter.report
assert_equal [], File.readlines(log_path).map(&:chomp)
end

def test_before_test
@reporter.start
@reporter.before_test(runnable('a'))
@reporter.before_test(runnable('b'))
@reporter.report
assert_equal ['Minitest::Test#a', 'Minitest::Test#b'], File.readlines(log_path).map(&:chomp)
end

unless truffleruby?
def test_forking
Comment on lines +26 to +27
Copy link
Contributor

@davidstosik davidstosik Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean to run this test only when not running the test suite with Truffle Ruby?
Does that mean the feature implemented in this PR is not compatible with Truffle Ruby?
Do we need a fallback? Will ci-queue not work with Truffle Ruby anymore?

Copy link
Member Author

@gmcgibbon gmcgibbon Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fork isn't implemented in truffle ruby. CI queue doesn't fork explicitly anywhere, we're just making sure the logging behaves well when distributed across multiple processes in forks or other threads.

pid = fork do
@reporter.start
end
pids = 5.times.map do
fork do
@reporter.before_test(runnable(Process.pid))
@reporter.report
end
end
(pids + [pid]).map do |pid|
Process.waitpid(pid)
end

assert_equal pids.map { |pid| "Minitest::Test##{pid}" }.sort, File.readlines(log_path).map(&:chomp).sort
end
end

private

def delete_log
Expand Down
9 changes: 9 additions & 0 deletions ruby/test/test_helper.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,12 @@
Dir[File.expand_path('../support/**/*.rb', __FILE__)].sort.each do |file|
require file
end


class Minitest::Test
class << self
def truffleruby?
RUBY_ENGINE == "truffleruby"
end
end
end
Loading