BuildStream doesn't quit on ctrl+c then `quit`

[BenjaminSchubert]

When running BuildStream interactively, the quit feature seems broken and will not stop running.

What did you do?

cd <my-v2 project>
bst build
^C
quit

What did you expect?

BuildStream to finish building the few packages it was building, then stop

What happened instead?

BuildStream continued

[abderrahim]

I noticed this too. Might be related to #1727

[gtristan]

BuildStream to finish building the few packages it was building, then stop

This is no longer the correct expectation, and indeed will be the result of #1727 which addresses #534 (Failed builds not pushing artifacts on quit). Essentially without this, the default/majority behavior makes sharing of failed build artifacts is useless because failed builds do not get pushed.

As of #1727, the expectation is that imperative tasks and pre-imperative tasks will no longer be queued after quitting, while post-imperative tasks will continue to be queued and processed. This only really applies to the build command, and the behavior will be that no further elements will be built, but any already built elements will continue to be pushed.

Normally I would leave this open and wait for further info, but since this is casting doubt on the already delayed 2.0 release I'll close this and ask that if you find that pre-imperative tasks continue to be processed/queued after quitting, to please reopen this.

[BenjaminSchubert]

Hey, sorry I realize I should have been more thorough in the description.

What I saw is more build jobs getting enqueued (and not just one round of them), which I believe breaks the above expectations you outlined?

[gtristan]

Yes that would break the expectation indeed :)

I'll We'll have to check if we have tests for this, and possibly add more tests to verify this.

[abderrahim]

So I took the time to go down this rabbit hole. What's happening here is that are "ready" in the build queue but haven't been scheduled yet (waiting for resources) will still be scheduled even after the quit command is issued.

More specifically, at

buildstream/src/buildstream/_scheduler/scheduler.py

Line 389 in 6c06e1b

ready.extend(chain.from_iterable(q.harvest_jobs() for q in reversed(queues)))

all considered queues (including the build queue) are harvested for ready jobs, and those jobs are scheduled.

#2005 is my attempt to fix this.