Add simple logging with timestamps to identify high-traffic scheduled tasks

[Copilot]

One of the scheduled tasks is consuming excessive bandwidth, likely downloading gigabytes of email data. Need instrumentation to identify which task.

Changes

Simplified Logging Approach

• Logs to PHP error_log (outputs to Docker stderr)
• All log entries include timestamps in Y-m-d H:i:s format
• Tracks response time for each scheduled task execution
• Counts number of actions/items processed per task
• No database, no UI, no additional infrastructure

Task Integration
All 5 scheduled tasks now log timing and action counts with timestamps:

$startTime = microtime(true);
$taskName = 'scheduled-email-receiver';
error_log(date('Y-m-d H:i:s') . " [$taskName] Starting task");

// Task execution
$result = $emailReceiver->processNextFolder();
$foldersProcessed++;

$duration = round(microtime(true) - $startTime, 3);
error_log(date('Y-m-d H:i:s') . " [$taskName] Task completed in {$duration}s - Processed $foldersProcessed folders");

IMAP Action Details
For scheduled-imap-handling, individual IMAP action durations are logged with timestamps:

• createFolders execution time
• processSentFolder execution time
• processInbox execution time

This helps identify which specific IMAP operation is consuming the most time/bandwidth.

Usage

View Docker logs to identify which task takes the longest time, indicating high bandwidth usage:

docker logs organizer 2>&1 | grep "scheduled-"

Example output:

2025-12-26 21:30:15 [scheduled-email-receiver] Starting task
2025-12-26 21:30:59 [scheduled-email-receiver] Task completed in 45.234s - Processed 10 folders
2025-12-26 21:31:00 [scheduled-imap-handling] Starting task
2025-12-26 21:31:02 [scheduled-imap-handling] createFolders completed in 2.145s
2025-12-26 21:31:41 [scheduled-imap-handling] processSentFolder completed in 38.912s
2025-12-26 21:32:33 [scheduled-imap-handling] processInbox completed in 52.678s
2025-12-26 21:32:34 [scheduled-imap-handling] Task completed in 93.735s - Processed 25 threads

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Organizer - scheduled tasks using a lot of Internet bandwidth</issue_title>
<issue_description>One of the scheduled tasks in organizer is using a lot of Internet bandwidth. Add logging to figure out which. It is likely something that is receiving gigabytes of email messages (some of the emails in the system are large).</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Organizer - scheduled tasks using a lot of Internet bandwidth #108

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

---

Note

Adds lightweight instrumentation to scheduled tasks to identify high-bandwidth operations.

• Adds timestamped error_log start/complete/fail messages with execution duration across scheduled-email-receiver, scheduled-email-sending, scheduled-thread-follow-up, scheduled-email-extraction, and scheduled-imap-handling
• Counts processed items per run (folders, extractions) and includes status in logs
• Logs per-action timings in scheduled-imap-handling for createFolders, processSentFolder, and processInbox
• Improves error paths with duration-stamped failure logs and preserves admin notifications; validates extraction type and returns JSON error when invalid

^{Written by Cursor Bugbot for commit cb5a93c. This will update automatically on new commits. Configure here.}

[HNygard]

@copilot This is way overkill. Just log response time and number of different imap actions on scheduled tasks. Log to PHP error log (outputs to docker strerr)

[Copilot]

@copilot This is way overkill. Just log response time and number of different imap actions on scheduled tasks. Log to PHP error log (outputs to docker strerr)

Simplified the implementation in commit d0b6784. Now using error_log() to log response times and action counts for each scheduled task. The IMAP handling task also logs individual action durations (createFolders, processSentFolder, processInbox). All database infrastructure, UI, and tests removed.

[HNygard]

@copilot : Timestamp the error_log() calls.

[Copilot]

Added timestamps in commit 61d5fab. All error_log() calls now include date('Y-m-d H:i:s') prefix.

[cursor]

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on January 27

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

[cursor]

Folder count underreports by one due to uncounted call

The $foldersProcessed counter is initialized to 0 and only incremented inside the loop (lines 28-45), but processNextFolder() is also called at line 24 before the loop starts. This means the logged folder count will always be off by one — the timing includes all processed folders but the count misses the first one. Since the PR's goal is to identify high-bandwidth tasks, this inaccurate count could mislead debugging efforts.

 

[HNygard]

@copilot : Fix comment

[Copilot]

Pull request overview

This PR adds simple timestamp-based logging to all scheduled tasks in the organizer system to identify which task is consuming excessive bandwidth. The logging outputs to PHP's error_log (Docker stderr) and tracks task duration and action counts for each scheduled task.

Key Changes:

• Added start/end logging with timestamps in Y-m-d H:i:s format to all 5 scheduled tasks
• Implemented duration tracking using microtime(true) for precise timing measurements
• Added detailed IMAP operation timing for the scheduled-imap-handling task to identify which specific operation (createFolders, processSentFolder, or processInbox) consumes the most time

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
scheduled-thread-follow-up.php	Added start/end logging with task duration and success/failure status tracking
scheduled-imap-handling.php	Added task-level logging plus individual timing for three IMAP operations (createFolders, processSentFolder, processInbox)
scheduled-email-sending.php	Added start/end logging with task duration and success/failure status tracking
scheduled-email-receiver.php	Added start/end logging with folder processing counter to track how many folders were successfully processed
scheduled-email-extraction.php	Added start/end logging with extraction counter and invalid extraction type error logging

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[cursor]

Normal idle state incorrectly logged as task failure

The $status variable is set to 'failed' whenever $result['success'] is false, but the underlying classes return success => false with "No threads ready for sending/follow-up" when there's simply nothing to process—a normal idle condition, not an error. The existing code correctly treats this as non-error (skipping admin notification), but the new logging will generate "Task failed" entries during routine operation. This will pollute logs with false failure messages, making it harder to identify actual problems when debugging bandwidth issues.

Additional Locations (1)

• organizer/src/system-pages/scheduled-thread-follow-up.php#L36-L38