#2994 - T4A - Upload File to Student Account - Part 2 #5511

andrewsignori-aot · 2025-12-12T22:37:53Z

Expose new env variables.
- T4A_FOLDER, as planned, was removed from the constant and converted to an env config.
- T4A_ARCHIVE_FOLDER was exposed to allow having all the archived files added to the same shared folder. Currently, all archive operations implicitly look for an "Archive" folder at the same level as the files being processed, which would not work for the T4A and its subfolder, unless an agreement is made to create them ahead of time (either manually or automatically), which is not desired.
File group adjusted to "Tax Documents".
Introduced a small helper to allow some level of identification of SSH issues. The initial goal is to detect a permanent issue where the processing of a T4A batch should be stopped because no further attempt will likely work.
Method renameFile from the SSH changed to allow using an existing SFTP client.
Method archiveFile changed to allow it to optionally receive the "absolute" path where the file should be archived.
Minor and basic SIN validation added to the SIN number received to ensure they are in the expected format, since the job can receive manual input.

E2E Tests

Queue processor for t4a-upload-enqueuer,
√ Should enqueue 3 batches when the batch size is 100, and there are 2 folders, one with 100 files and the other with 101 files.
√ Should log that no directories were found when no directories were listed from the SFTP.
√ Should not try to add bulk jobs when a directory was found from the SFTP, but no files were found.

Queue processor for t4a-upload,
√ Should download a T4A file from the SFTP, create a file in the student account, upload the file to S3 storage, queue a virus scan, send a notification to the student, and archive the processed file on success when the student has the current SIN valid, and matching the file name.
√ Should find a student match when the SIN is valid for one of the SIN validations entries when the student has multiple valid SIN numbers.
√ Should skip the file upload when a file with the same hash was already uploaded for the same student.
√ Should log a warning and archive the file with a not found student when an invalid SIN is found in the T4A file name.
√ Should log a warning and archive the file when multiple student accounts have the same valid SIN.

...backend/apps/queue-consumers/src/processors/t4a/_tests_/e2e/t4a-upload.processor.e2e-spec.ts

sources/packages/backend/libs/integrations/src/t4a/t4a-enqueuer.processing.service.ts

dheepak-aot · 2025-12-16T22:00:10Z

sources/packages/backend/libs/integrations/src/services/ssh/ssh.service.ts

+   * @returns True if the error is an SSH error with the given code, false otherwise.
+   */
+  static hasError(error: unknown, errorCode: SSHErrorCodes): boolean {
+    return !!error && (error as SSHError).code === errorCode;


Was there a reason not to use the class and instance of similar to other errors here?

instance of works when a class is instantiated, similar to what we do in the Web for ApiProcessError.
Here, either the framework throws the error using a class, or the instance of would not work.
Please let me know if I am missing something.

dheepak-aot · 2025-12-16T22:01:04Z

sources/packages/backend/libs/integrations/src/t4a/t4a-upload.processing.service.ts

+      if (SshService.hasError(error, SSHErrorCodes.NotConnected)) {
+        // Stops processing as the SFTP client is disconnected.
+        throw new Error("SFTP client disconnected unexpectedly.", {
+          cause: error,
+        });
+      }


dheepak-aot · 2025-12-16T22:03:16Z

sources/packages/backend/libs/integrations/src/t4a/t4a-upload.processing.service.ts

    }
    if (!students.length) {
-      processSummary.warn(
+      processSummary.info(


dheepak-aot · 2025-12-16T22:11:48Z

sources/packages/backend/libs/test-utils/src/matchers/uuid-matcher.ts

+/**
+ * Validate if a string represents a valid UUID v4.
+ */
+export const uuidV4Matcher: jest.AsymmetricMatcher = {


I didn't get the purpose of this matcher. Can you please help me understand?

It asserts if a string is a GUID and also checks the GUID version as 4 (the one that we have been using).
Instead of using an expected.any(String).

dheepak-aot

Great coverage of E2E tests. Thanks for making the changes. I would suggest to add the new envs to env-example file.

tiago-graf · 2025-12-16T22:37:07Z

sources/packages/backend/libs/integrations/src/services/ssh/ssh.service.ts

+   * @param errorCode Error code to be checked.
+   * @returns True if the error is an SSH error with the given code, false otherwise.
+   */
+  static hasError(error: unknown, errorCode: SSHErrorCodes): boolean {


name could be hasSSHError

It can, but it is a static method, inside a service called SshService, that will be consumed as SshService.hasError. Does it really require SSH? If yes, then the createClient should be renamed to createSSHClient 😉

dheepak-aot · 2025-12-16T22:39:10Z

.github/workflows/env-setup-deploy-secrets.yml

      CRA_RESPONSE_FOLDER: ${{ secrets.CRA_RESPONSE_FOLDER }}
      CRA_PROGRAM_AREA_CODE: ${{ secrets.CRA_PROGRAM_AREA_CODE }}
      CRA_ENVIRONMENT_CODE: ${{ secrets.CRA_ENVIRONMENT_CODE }}
+      T4A_FOLDER: ${{ vars.T4A_FOLDER }}


Please also update the devops makefile to pass these parameters.

tiago-graf

looks great, just a minor comment

sonarqubecloud · 2025-12-16T23:18:40Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions · 2025-12-16T23:19:08Z

Backend Unit Tests Coverage Report

Totals
Statements:	20.39% ( 4306 / 21120 )
Methods:	9.74% ( 252 / 2588 )
Lines:	24.53% ( 3688 / 15034 )
Branches:	10.46% ( 366 / 3498 )

github-actions · 2025-12-16T23:19:21Z

E2E Workflow Workers Coverage Report

Totals
Statements:	75.41% ( 1055 / 1399 )
Methods:	79.31% ( 115 / 145 )
Lines:	78.79% ( 769 / 976 )
Branches:	61.51% ( 171 / 278 )

github-actions · 2025-12-16T23:22:23Z

E2E Queue Consumers Coverage Report

Totals
Statements:	85.47% ( 1588 / 1858 )
Methods:	84.65% ( 182 / 215 )
Lines:	88.47% ( 1266 / 1431 )
Branches:	66.04% ( 140 / 212 )

github-actions · 2025-12-16T23:22:52Z

E2E SIMS API Coverage Report

Totals
Statements:	75.11% ( 8464 / 11269 )
Methods:	74.91% ( 1009 / 1347 )
Lines:	79.26% ( 6162 / 7774 )
Branches:	60.2% ( 1293 / 2148 )

andrewsignori-aot added 2 commits December 12, 2025 14:02

Archiving

9ca5331

T4A_FOLDER env and sin validation

9eeab02

andrewsignori-aot changed the title ~~Queue consumers t4 part 2~~ #2994 - T4A - Upload File to Student Account - Part 2 Dec 12, 2025

andrewsignori-aot self-assigned this Dec 12, 2025

andrewsignori-aot added Queue Consumers Devops Devops labels Dec 12, 2025

andrewsignori-aot added 12 commits December 12, 2025 17:40

Archive config folder

33bd15e

Minor comment update.

7a248d9

First E2E for enqueuer

12ef9ab

Fix E2E

2bb2acf

More E2E tests for enqueuer

4bf36b9

E2E fix

99eca3e

E2E fix

647e8fb

First E2E for T4AUploadProcessor

f99b04d

Added second E2E test

02e8cd7

Added more E2E tests

1b3e2c0

toSorted

7e7ac69

Sonarcloud issue fix

b2913dc

andrewsignori-aot requested a review from Copilot December 16, 2025 00:52

Copilot started reviewing on behalf of andrewsignori-aot December 16, 2025 00:52 View session

This comment was marked as resolved.

Sign in to view

Copilot AI mentioned this pull request Dec 16, 2025

No changes required - JSDoc already compliant #5531

Closed

Copilot suggestions and fixes

21361e2

bcgov deleted a comment from Copilot AI Dec 16, 2025

E2E scenario added and pre-review adjustments

503ccf1

andrewsignori-aot requested a review from Copilot December 16, 2025 02:04

Copilot started reviewing on behalf of andrewsignori-aot December 16, 2025 02:05 View session

This comment was marked as outdated.

Sign in to view

Testing copilot instructions

1a3611f

andrewsignori-aot requested a review from Copilot December 16, 2025 02:18

dheepak-aot reviewed Dec 16, 2025

View reviewed changes

...backend/apps/queue-consumers/src/processors/t4a/_tests_/e2e/t4a-upload.processor.e2e-spec.ts Outdated Show resolved Hide resolved

dheepak-aot reviewed Dec 16, 2025

View reviewed changes

sources/packages/backend/libs/integrations/src/t4a/t4a-enqueuer.processing.service.ts Outdated Show resolved Hide resolved

dheepak-aot reviewed Dec 16, 2025

View reviewed changes

E2E test leftover removal and early return on no files

25ce98c

tiago-graf self-requested a review December 16, 2025 22:19

dheepak-aot approved these changes Dec 16, 2025

View reviewed changes

tiago-graf reviewed Dec 16, 2025

View reviewed changes

dheepak-aot reviewed Dec 16, 2025

View reviewed changes

tiago-graf approved these changes Dec 16, 2025

View reviewed changes

Added missing env variables to make file

4124278

andrewsignori-aot added this pull request to the merge queue Dec 16, 2025

Merged via the queue into main with commit 6b885de Dec 16, 2025
22 checks passed

andrewsignori-aot deleted the queue-consumers-t4-part-2 branch December 16, 2025 23:55

andrewsignori-aot temporarily deployed to DEV December 17, 2025 00:00 — with GitHub Actions Inactive

andrewsignori-aot temporarily deployed to DEV December 17, 2025 00:03 — with GitHub Actions Inactive

#2994 - T4A - Upload File to Student Account - Part 2 #5511

#2994 - T4A - Upload File to Student Account - Part 2 #5511

Uh oh!

Conversation

andrewsignori-aot commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Tests

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrewsignori-aot Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dheepak-aot left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tiago-graf left a comment

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Dec 16, 2025

Quality Gate passed

Uh oh!

github-actions bot commented Dec 16, 2025

Backend Unit Tests Coverage Report

Uh oh!

github-actions bot commented Dec 16, 2025

E2E Workflow Workers Coverage Report

Uh oh!

github-actions bot commented Dec 16, 2025

E2E Queue Consumers Coverage Report

Uh oh!

github-actions bot commented Dec 16, 2025

E2E SIMS API Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andrewsignori-aot commented Dec 12, 2025 •

edited

Loading

andrewsignori-aot Dec 16, 2025 •

edited

Loading