Skip to content

Fix #433 Prevent per-page hangs & avoid killing job on maxbackoff#438

Open
akshan-main wants to merge 1 commit intoallenai:mainfrom
akshan-main:request_timeout_and_backoff_fix
Open

Fix #433 Prevent per-page hangs & avoid killing job on maxbackoff#438
akshan-main wants to merge 1 commit intoallenai:mainfrom
akshan-main:request_timeout_and_backoff_fix

Conversation

@akshan-main
Copy link

Closes #433

Changes proposed in this pull request:

  • apost() now takes a timeout_s param and wraps the entire network path in asyncio.timeout(), so a stalled server cant block forever
  • When max backoff is exhausted, we return None instead of sys.exit(1) - the existing fallback path (make_fallback_result) handles it from there, so the rest of the PDF still gets processed
  • New --request_timeout_s CLI flag (default 120s) to control per-request timeout

Before submitting

  • I've read and followed all steps in the Making a pull request
    section of the CONTRIBUTING docs.
  • I've updated or added any relevant docstrings following the syntax described in the
    Writing docstrings section of the CONTRIBUTING docs.
  • If this PR fixes a bug, I've added a test that will fail without my fix.
  • If this PR adds a new feature, I've added tests that sufficiently cover my new functionality.

@akshan-main
Copy link
Author

@jakep-allenai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Tends to get stuck at the last few pages

1 participant