diff --git a/requirements/emailController/addNonHgnEmailSubscription.md b/requirements/emailController/addNonHgnEmailSubscription.md index f5748f142..e22c645e3 100644 --- a/requirements/emailController/addNonHgnEmailSubscription.md +++ b/requirements/emailController/addNonHgnEmailSubscription.md @@ -5,19 +5,40 @@ 1. ❌ **Returns error 400 if `email` field is missing from the request** - Ensures that the function checks for the presence of the `email` field in the request body and responds with a `400` status code if it's missing. -2. ❌ **Returns error 400 if the provided `email` already exists in the subscription list** +2. ❌ **Returns error 400 if the provided `email` is invalid** + - Verifies that the function validates email format using `isValidEmailAddress` and responds with a `400` status code for invalid emails. + +3. ❌ **Returns error 400 if the provided `email` already exists in the subscription list** - This case checks that the function responds with a `400` status code and a message indicating that the email is already subscribed. -3. ❌ **Returns error 500 if there is an internal error while checking the subscription list** - - Covers scenarios where there's an issue querying the `EmailSubscriptionList` collection for the provided email (e.g., database connection issues). +4. ❌ **Returns error 400 if the email is already an HGN user** + - Verifies that the function checks if the email belongs to an existing HGN user and responds with a `400` status code, directing them to use the HGN account profile page. + +5. ❌ **Returns error 500 if there is an internal error while checking the subscription list** + - Covers scenarios where there's an issue querying the `EmailSubcriptionList` collection for the provided email (e.g., database connection issues). -4. ❌ **Returns error 500 if there is an error sending the confirmation email** +6. ❌ **Returns error 500 if `FRONT_END_URL` cannot be determined from request** + - Verifies that the function handles cases where the frontend URL cannot be determined from request headers, config, or request information. + +7. ❌ **Returns error 500 if there is an error sending the confirmation email** - This case handles any issues that occur while calling the `emailSender` function, such as network errors or service unavailability. +8. ❌ **Returns error 400 if there's a duplicate key error (race condition)** + - Handles MongoDB duplicate key errors that might occur if the subscription is created simultaneously by multiple requests. + ## Positive Cases -1. ❌ **Returns status 200 when a new email is successfully subscribed** - - Ensures that the function successfully creates a JWT token, constructs the email, and sends the subscription confirmation email to the user. +1. ✅ **Returns status 200 when a new email is successfully subscribed** + - Ensures that the function successfully creates an unconfirmed subscription record, generates a JWT token, and sends the subscription confirmation email to the user. + +2. ✅ **Creates subscription with correct initial state** + - Verifies that the subscription is created with `isConfirmed: false`, `emailSubscriptions: true`, and proper normalization (lowercase email). + +3. ✅ **Successfully sends a confirmation email containing the correct link** + - Verifies that the generated JWT token is correctly included in the confirmation link, and the frontend URL is dynamically determined from the request origin. + +4. ✅ **Returns success even if confirmation email fails to send** + - Ensures that if the subscription is saved to the database but the confirmation email fails, the function still returns success (subscription is already saved). -2. ❌ **Successfully sends a confirmation email containing the correct link** - - Verifies that the generated JWT token is correctly included in the confirmation link sent to the user in the email body. +5. ❌ **Correctly normalizes email to lowercase** + - Ensures that email addresses are stored in lowercase format, matching the schema's lowercase enforcement. diff --git a/requirements/emailController/confirmNonHgnEmailSubscription.md b/requirements/emailController/confirmNonHgnEmailSubscription.md index d5e1367af..efd368f67 100644 --- a/requirements/emailController/confirmNonHgnEmailSubscription.md +++ b/requirements/emailController/confirmNonHgnEmailSubscription.md @@ -1,18 +1,29 @@ -# Confirm Non-HGN Email Subscription Function Tests +# Confirm Non-HGN Email Subscription Function ## Negative Cases -1. ✅ **Returns error 400 if `token` field is missing from the request** - - (Test: `should return 400 if token is not provided`) -2. ✅ **Returns error 401 if the provided `token` is invalid or expired** - - (Test: `should return 401 if token is invalid`) +1. ❌ **Returns error 400 if `token` field is missing from the request** + - Ensures that the function checks for the presence of the `token` field in the request body and responds with a `400` status code if it's missing. -3. ✅ **Returns error 400 if the decoded `token` does not contain a valid `email` field** - - (Test: `should return 400 if email is missing from payload`) +2. ❌ **Returns error 401 if the provided `token` is invalid or expired** + - Verifies that the function correctly handles invalid or expired JWT tokens and responds with a `401` status code. -4. ❌ **Returns error 500 if there is an internal error while saving the new email subscription** +3. ❌ **Returns error 400 if the decoded `token` does not contain a valid `email` field** + - Ensures that the function validates the token payload contains a valid email address and responds with a `400` status code if it doesn't. + +4. ❌ **Returns error 404 if subscription doesn't exist** + - Verifies that the function only confirms existing subscriptions. If no subscription exists for the email in the token, it should return a `404` status code with a message directing the user to subscribe first. + +5. ❌ **Returns error 500 if there is an internal error while updating the subscription** + - Covers scenarios where there's a database error while updating the subscription status. ## Positive Cases -1. ❌ **Returns status 200 when a new email is successfully subscribed** -2. ❌ **Returns status 200 if the email is already subscribed (duplicate email)** +1. ✅ **Returns status 200 when an existing unconfirmed subscription is successfully confirmed** + - Ensures that the function updates an existing unconfirmed subscription to confirmed status, sets `confirmedAt` timestamp, and enables `emailSubscriptions`. + +2. ✅ **Returns status 200 if the email subscription is already confirmed (idempotent)** + - Verifies that the function is idempotent - if a subscription is already confirmed, it returns success without attempting to update again. + +3. ❌ **Correctly handles email normalization (lowercase)** + - Ensures that email addresses are normalized to lowercase for consistent lookups, matching the schema's lowercase enforcement. diff --git a/requirements/emailController/processPendingAndStuckEmails.md b/requirements/emailController/processPendingAndStuckEmails.md new file mode 100644 index 000000000..ce81699ad --- /dev/null +++ b/requirements/emailController/processPendingAndStuckEmails.md @@ -0,0 +1,33 @@ +# Process Pending and Stuck Emails Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to process emails. + +3. ❌ **Returns error 500 if there is an internal error while processing** + - Covers scenarios where there are errors during the processing of pending and stuck emails (e.g., database errors, service failures). + +## Positive Cases + +1. ✅ **Returns status 200 when processing is triggered successfully** + - Ensures that the function triggers the email processor to handle pending and stuck emails and returns success. + +2. ✅ **Resets stuck emails (SENDING status) to PENDING** + - Verifies that emails in SENDING status are reset to PENDING so they can be reprocessed (typically after server restart). + +3. ✅ **Resets stuck batches (SENDING status) to PENDING** + - Ensures that EmailBatch items in SENDING status are reset to PENDING so they can be reprocessed. + +4. ✅ **Queues all PENDING emails for processing** + - Verifies that all emails in PENDING status are added to the processing queue for immediate processing. + +5. ✅ **Handles errors gracefully without throwing** + - Ensures that individual errors during processing (e.g., resetting a specific stuck email) are logged but don't prevent the overall process from completing. + +6. ❌ **Provides detailed logging for troubleshooting** + - Verifies that the function logs information about the number of stuck emails/batches found and processed. + diff --git a/requirements/emailController/removeNonHgnEmailSubscription.md b/requirements/emailController/removeNonHgnEmailSubscription.md index af793e2a9..dd9e92814 100644 --- a/requirements/emailController/removeNonHgnEmailSubscription.md +++ b/requirements/emailController/removeNonHgnEmailSubscription.md @@ -1,10 +1,29 @@ -# Remove Non-HGN Email Subscription Function Tests +# Remove Non-HGN Email Subscription Function ## Negative Cases + 1. ✅ **Returns error 400 if `email` field is missing from the request** - - (Test: `should return 400 if email is missing`) + - Ensures that the function checks for the presence of the `email` field in the request body and responds with a `400` status code if it's missing. + +2. ❌ **Returns error 400 if the provided `email` is invalid** + - Verifies that the function validates email format using `isValidEmailAddress` and responds with a `400` status code for invalid emails. -2. ❌ **Returns error 500 if there is an internal error while deleting the email subscription** +3. ❌ **Returns error 404 if the email subscription is not found** + - Verifies that the function handles cases where no subscription exists for the given email and responds with a `404` status code. + +4. ❌ **Returns error 500 if there is an internal error while deleting the email subscription** + - Covers scenarios where there's a database error while deleting the subscription (e.g., database connection issues). ## Positive Cases -1. ❌ **Returns status 200 when an email is successfully unsubscribed** + +1. ✅ **Returns status 200 when an email is successfully unsubscribed** + - Ensures that the function deletes the subscription record from the `EmailSubcriptionList` collection and returns success with a `200` status code. + +2. ✅ **Correctly normalizes email to lowercase for lookup** + - Verifies that the email is normalized to lowercase before querying/deleting, ensuring consistent matches with the schema's lowercase enforcement. + +3. ✅ **Uses direct email match (no regex needed)** + - Ensures that since the schema enforces lowercase emails, the function uses direct email matching instead of case-insensitive regex. + +4. ❌ **Handles concurrent unsubscribe requests gracefully** + - Ensures that if multiple unsubscribe requests are made simultaneously, the function handles race conditions appropriately. diff --git a/requirements/emailController/resendEmail.md b/requirements/emailController/resendEmail.md new file mode 100644 index 000000000..4bfee4de2 --- /dev/null +++ b/requirements/emailController/resendEmail.md @@ -0,0 +1,69 @@ +# Resend Email Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to resend emails. + +3. ❌ **Returns error 400 if `emailId` is missing or invalid** + - Ensures that the function validates `emailId` is a valid MongoDB ObjectId. + +4. ❌ **Returns error 404 if the original email is not found** + - Verifies that the function handles cases where the email with the provided `emailId` doesn't exist. + +5. ❌ **Returns error 400 if `recipientOption` is missing** + - Ensures that the `recipientOption` field is required in the request body. + +6. ❌ **Returns error 400 if `recipientOption` is invalid** + - Verifies that the `recipientOption` must be one of: `'all'`, `'specific'`, or `'same'`. + +7. ❌ **Returns error 400 if `specificRecipients` is required but missing for 'specific' option** + - Ensures that when `recipientOption` is `'specific'`, the `specificRecipients` array must be provided and non-empty. + +8. ❌ **Returns error 404 if no recipients found for 'same' option** + - Verifies that when `recipientOption` is `'same'`, the original email must have EmailBatch items with recipients. + +9. ❌ **Returns error 400 if recipient count exceeds maximum limit for 'specific' option** + - Ensures that when using `'specific'` option, the recipient limit (2000) is enforced. + +10. ❌ **Returns error 400 if no recipients are found** + - Verifies that after determining recipients, at least one recipient must be available. + +11. ❌ **Returns error 404 if requestor user is not found** + - Ensures that the function validates the requestor exists in the userProfile collection. + +12. ❌ **Returns error 500 if there is an internal error during email creation** + - Covers scenarios where there are database errors or service failures during email/batch creation. + +## Positive Cases + +1. ✅ **Returns status 200 when email is successfully resent with 'all' option** + - Ensures that when `recipientOption` is `'all'`, the function sends to all active HGN users and confirmed email subscribers. + +2. ✅ **Returns status 200 when email is successfully resent with 'specific' option** + - Verifies that when `recipientOption` is `'specific'`, the function sends to only the provided `specificRecipients` list. + +3. ✅ **Returns status 200 when email is successfully resent with 'same' option** + - Ensures that when `recipientOption` is `'same'`, the function extracts recipients from the original email's EmailBatch items and deduplicates them. + +4. ✅ **Creates new email copy with same subject and HTML content** + - Verifies that the function creates a new Email document with the same `subject` and `htmlContent` as the original, but with a new `createdBy` user. + +5. ✅ **Enforces recipient limit only for 'specific' option** + - Ensures that the maximum recipient limit is enforced only when `recipientOption` is `'specific'`, but skipped for `'all'` and `'same'` (broadcast scenarios). + +6. ✅ **Skips recipient limit for broadcast scenarios ('all' and 'same')** + - Verifies that when using `'all'` or `'same'` options, the recipient limit is not enforced. + +7. ✅ **Deduplicates recipients for 'same' option** + - Ensures that when using `'same'` option, duplicate email addresses are removed from the recipient list. + +8. ✅ **Creates email batches in a transaction** + - Ensures that the parent Email and all EmailBatch items are created atomically in a single transaction. + +9. ❌ **Handles transaction rollback on errors** + - Ensures that if any part of email/batch creation fails, the entire transaction is rolled back. + diff --git a/requirements/emailController/retryEmail.md b/requirements/emailController/retryEmail.md new file mode 100644 index 000000000..27012df40 --- /dev/null +++ b/requirements/emailController/retryEmail.md @@ -0,0 +1,51 @@ +# Retry Email Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to retry emails. + +3. ❌ **Returns error 400 if `emailId` parameter is missing or invalid** + - Ensures that the function validates `emailId` from `req.params` is a valid MongoDB ObjectId. + +4. ❌ **Returns error 404 if the email is not found** + - Verifies that the function handles cases where the email with the provided `emailId` doesn't exist. + +5. ❌ **Returns error 400 if email is not in a retryable status** + - Ensures that the function only allows retry for emails in `FAILED` or `PROCESSED` status. Returns `400` for other statuses. + +6. ❌ **Returns error 500 if there is an internal error while fetching failed batches** + - Covers scenarios where there are database errors while querying for failed EmailBatch items. + +7. ❌ **Returns error 500 if there is an internal error while resetting email status** + - Covers scenarios where there are database errors while updating the email status to PENDING. + +8. ❌ **Returns error 500 if there is an internal error while resetting batches** + - Covers scenarios where there are database errors while resetting individual EmailBatch items to PENDING. + +## Positive Cases + +1. ✅ **Returns status 200 when email is successfully retried with failed batches** + - Ensures that the function marks the parent Email as PENDING, resets all failed EmailBatch items to PENDING, queues the email for processing, and returns success with the count of failed items retried. + +2. ✅ **Returns status 200 when email has no failed batches** + - Verifies that if an email has no failed EmailBatch items, the function returns success with `failedItemsRetried: 0` without error. + +3. ✅ **Correctly resets only failed EmailBatch items** + - Ensures that only EmailBatch items with `FAILED` status are reset to PENDING for retry. + +4. ✅ **Marks parent email as PENDING** + - Verifies that the parent Email status is changed to PENDING, allowing it to be reprocessed. + +5. ✅ **Queues email for processing after reset** + - Ensures that after resetting the email and batches, the email is added to the processing queue. + +6. ✅ **Returns correct data in response** + - Verifies that the response includes `emailId` and `failedItemsRetried` count in the data field. + +7. ❌ **Handles concurrent retry requests gracefully** + - Ensures that if multiple retry requests are made simultaneously, the function handles race conditions appropriately. + diff --git a/requirements/emailController/sendEmail.md b/requirements/emailController/sendEmail.md index 7ca9a482c..4879a24a7 100644 --- a/requirements/emailController/sendEmail.md +++ b/requirements/emailController/sendEmail.md @@ -2,9 +2,43 @@ ## Negative Cases -1. ❌ **Returns error 400 if `to`, `subject`, or `html` fields are missing from the request** -2. ❌ **Returns error 500 if there is an internal error while sending the email** +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to send emails. + +3. ❌ **Returns error 400 if `to`, `subject`, or `html` fields are missing from the request** + - Ensures that all required fields (`to`, `subject`, `html`) are present in the request body. + +4. ❌ **Returns error 400 if email contains unreplaced template variables** + - Verifies that the function validates that all template variables in `subject` and `html` have been replaced before sending. + +5. ❌ **Returns error 404 if requestor user is not found** + - Ensures that the function validates the requestor exists in the userProfile collection. + +6. ❌ **Returns error 400 if recipient count exceeds maximum limit (2000)** + - Verifies that the function enforces the maximum recipients per request limit for specific recipient requests. + +7. ❌ **Returns error 400 if any recipient email is invalid** + - Ensures that all recipient email addresses are validated before creating batches. + +8. ❌ **Returns error 500 if there is an internal error during email creation** + - Covers scenarios where there are database errors or service failures during email/batch creation. ## Positive Cases -1. ✅ **Returns status 200 when email is successfully sent with `to`, `subject`, and `html` fields provided** +1. ✅ **Returns status 200 when email is successfully created with valid recipients** + - Ensures that the function creates the parent Email document and EmailBatch items in a transaction, queues the email for processing, and returns success. + +2. ✅ **Enforces recipient limit for specific recipient requests** + - Verifies that the maximum recipient limit (2000) is enforced when sending to specific recipients. + +3. ✅ **Creates email batches correctly** + - Ensures that recipients are properly normalized, validated, and chunked into EmailBatch items according to the configured batch size. + +4. ✅ **Validates all template variables are replaced** + - Verifies that the function checks both HTML content and subject for unreplaced template variables before allowing email creation. + +5. ❌ **Handles transaction rollback on errors** + - Ensures that if any part of email/batch creation fails, the entire transaction is rolled back. diff --git a/requirements/emailController/sendEmailToAll.md b/requirements/emailController/sendEmailToAll.md deleted file mode 100644 index 32a09fed6..000000000 --- a/requirements/emailController/sendEmailToAll.md +++ /dev/null @@ -1,26 +0,0 @@ -# Send Email to All Function - -## Negative Cases - -1. ❌ **Returns error 400 if `subject` or `html` fields are missing from the request** - - The request should be rejected if either the `subject` or `html` content is not provided in the request body. - -2. ❌ **Returns error 500 if there is an internal error while fetching users** - - This case covers scenarios where there's an error fetching users from the `userProfile` collection (e.g., database connection issues). - -3. ❌ **Returns error 500 if there is an internal error while fetching the subscription list** - - This case covers scenarios where there's an error fetching emails from the `EmailSubcriptionList` collection. - -4. ❌ **Returns error 500 if there is an error sending emails** - - This case handles any issues that occur while calling the `emailSender` function, such as network errors or service unavailability. - -## Positive Cases - -1. ❌ **Returns status 200 when emails are successfully sent to all active users** - - Ensures that the function sends emails correctly to all users meeting the criteria (`isActive` and `EmailSubcriptionList`). - -2. ❌ **Returns status 200 when emails are successfully sent to all users in the subscription list** - - Verifies that the function sends emails to all users in the `EmailSubcriptionList`, including the unsubscribe link in the email body. - -3. ❌ **Combines user and subscription list emails successfully** - - Ensures that the function correctly sends emails to both active users and the subscription list without issues. diff --git a/requirements/emailController/sendEmailToSubscribers.md b/requirements/emailController/sendEmailToSubscribers.md new file mode 100644 index 000000000..bcc2b8bbf --- /dev/null +++ b/requirements/emailController/sendEmailToSubscribers.md @@ -0,0 +1,50 @@ +# Send Email to All Subscribers Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to send emails to subscribers. + +3. ❌ **Returns error 400 if `subject` or `html` fields are missing from the request** + - The request should be rejected if either the `subject` or `html` content is not provided in the request body. + +4. ❌ **Returns error 400 if email contains unreplaced template variables** + - Verifies that the function validates that all template variables in `subject` and `html` have been replaced before sending. + +5. ❌ **Returns error 404 if requestor user is not found** + - Ensures that the function validates the requestor exists in the userProfile collection. + +6. ❌ **Returns error 400 if no recipients are found** + - Verifies that the function checks if there are any active HGN users or confirmed email subscribers before creating the email. + +7. ❌ **Returns error 500 if there is an internal error while fetching users** + - This case covers scenarios where there's an error fetching users from the `userProfile` collection (e.g., database connection issues). + +8. ❌ **Returns error 500 if there is an internal error while fetching the subscription list** + - This case covers scenarios where there's an error fetching emails from the `EmailSubcriptionList` collection. + +9. ❌ **Returns error 500 if there is an error creating email or batches** + - Covers scenarios where there are database errors or service failures during email/batch creation. + +## Positive Cases + +1. ✅ **Returns status 200 when emails are successfully created for all active users** + - Ensures that the function sends emails correctly to all users meeting the criteria (`isActive: true`, `emailSubscriptions: true`, non-empty `firstName`, non-null `email`). + +2. ✅ **Returns status 200 when emails are successfully created for all confirmed subscribers** + - Verifies that the function sends emails to all confirmed subscribers in the `EmailSubcriptionList` (with `isConfirmed: true` and `emailSubscriptions: true`). + +3. ✅ **Combines user and subscription list emails successfully** + - Ensures that the function correctly combines recipients from both active HGN users and confirmed email subscribers without duplicates. + +4. ✅ **Skips recipient limit for broadcast emails** + - Verifies that the maximum recipient limit is NOT enforced when broadcasting to all subscribers. + +5. ✅ **Creates email batches in a transaction** + - Ensures that the parent Email and all EmailBatch items are created atomically in a single transaction. + +6. ❌ **Handles transaction rollback on errors** + - Ensures that if any part of email/batch creation fails, the entire transaction is rolled back. diff --git a/requirements/emailController/updateEmailSubscription.md b/requirements/emailController/updateEmailSubscription.md index bcafa5a28..6f7b9fa15 100644 --- a/requirements/emailController/updateEmailSubscription.md +++ b/requirements/emailController/updateEmailSubscription.md @@ -2,19 +2,34 @@ ## Negative Cases -1. ❌ **Returns error 400 if `emailSubscriptions` field is missing from the request** +1. ❌ **Returns error 401 if `requestor.email` is missing from the request** + - Ensures that the function checks for the presence of `requestor.email` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 400 if `emailSubscriptions` field is missing from the request** - This ensures that the function checks for the presence of the `emailSubscriptions` field in the request body and responds with a `400` status code if it's missing. -2. ❌ **Returns error 400 if `email` field is missing from the requestor object** - - Ensures that the function requires an `email` field within the `requestor` object in the request body and returns `400` if it's absent. +3. ❌ **Returns error 400 if `emailSubscriptions` is not a boolean value** + - Verifies that the function validates that `emailSubscriptions` is a boolean type and returns `400` for invalid types. + +4. ❌ **Returns error 400 if the provided `email` is invalid** + - Ensures that the function validates the email format using `isValidEmailAddress` and responds with a `400` status code for invalid emails. -3. ❌ **Returns error 404 if the user with the provided `email` is not found** +5. ❌ **Returns error 404 if the user with the provided `email` is not found** - This checks that the function correctly handles cases where no user exists with the given `email` and responds with a `404` status code. -4. ✅ **Returns error 500 if there is an internal error while updating the user profile** +6. ❌ **Returns error 500 if there is an internal error while updating the user profile** - Covers scenarios where there's a database error while updating the user's email subscriptions. ## Positive Cases -1. ❌ **Returns status 200 and the updated user when email subscriptions are successfully updated** - - Ensures that the function updates the `emailSubscriptions` field for the user and returns the updated user document along with a `200` status code. +1. ✅ **Returns status 200 when email subscriptions are successfully updated** + - Ensures that the function updates the `emailSubscriptions` field for the user and returns success with a `200` status code. + +2. ✅ **Correctly normalizes email to lowercase for lookup** + - Verifies that the email is normalized to lowercase before querying the database, ensuring consistent lookups. + +3. ✅ **Updates user profile atomically** + - Ensures that the user profile update uses `findOneAndUpdate` to atomically update the subscription preference. + +4. ❌ **Handles concurrent update requests gracefully** + - Ensures that if multiple update requests are made simultaneously, the function handles race conditions appropriately. diff --git a/requirements/emailOutboxController/getEmailDetails.md b/requirements/emailOutboxController/getEmailDetails.md new file mode 100644 index 000000000..6400f4857 --- /dev/null +++ b/requirements/emailOutboxController/getEmailDetails.md @@ -0,0 +1,39 @@ +# Get Email Details (Outbox) Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to view email details. + +3. ❌ **Returns error 400 if email ID is invalid** + - Ensures that the function validates the email ID from `req.params.emailId` is a valid MongoDB ObjectId format. + +4. ❌ **Returns error 404 if email is not found** + - Verifies that the function handles cases where no email exists with the provided ID and responds with a `404` status code. + +5. ❌ **Returns error 500 if there is an internal error while fetching email details** + - Covers scenarios where there are database errors or service failures while fetching the email and its associated EmailBatch items. + +## Positive Cases + +1. ✅ **Returns status 200 with email and batch details** + - Ensures that the function successfully fetches the parent Email record and all associated EmailBatch items and returns them in the response. + +2. ✅ **Returns complete email information** + - Verifies that the response includes all email fields: `_id`, `subject`, `htmlContent`, `status`, `createdBy`, `createdAt`, `startedAt`, `completedAt`, `updatedAt`. + +3. ✅ **Returns all associated EmailBatch items** + - Ensures that all EmailBatch items associated with the email are included in the response, with all batch details (recipients, status, attempts, timestamps, etc.). + +4. ✅ **Returns email with populated creator information** + - Verifies that the `createdBy` field is populated with user profile information (firstName, lastName, email). + +5. ❌ **Handles emails with no batches gracefully** + - Verifies that if an email has no associated EmailBatch items, the function returns the email with an empty batches array without error. + +6. ❌ **Returns correct data structure** + - Ensures that the response follows the expected structure with email details and associated batches properly nested. + diff --git a/requirements/emailOutboxController/getEmails.md b/requirements/emailOutboxController/getEmails.md new file mode 100644 index 000000000..8121de7ab --- /dev/null +++ b/requirements/emailOutboxController/getEmails.md @@ -0,0 +1,30 @@ +# Get All Emails (Outbox) Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to view emails. + +3. ❌ **Returns error 500 if there is an internal error while fetching emails** + - Covers scenarios where there are database errors or service failures while fetching email records. + +## Positive Cases + +1. ✅ **Returns status 200 with all email records** + - Ensures that the function successfully fetches all Email (parent) records from the database and returns them in the response. + +2. ✅ **Returns emails ordered by creation date (descending)** + - Verifies that emails are returned sorted by `createdAt` in descending order (newest first). + +3. ✅ **Returns emails with populated creator information** + - Ensures that the `createdBy` field is populated with user profile information (firstName, lastName, email). + +4. ✅ **Returns complete email metadata** + - Verifies that the response includes all email fields: `_id`, `subject`, `htmlContent`, `status`, `createdBy`, `createdAt`, `startedAt`, `completedAt`, `updatedAt`. + +5. ❌ **Handles empty email list gracefully** + - Verifies that if no emails exist, the function returns an empty array without error. + diff --git a/requirements/emailTemplateController/createEmailTemplate.md b/requirements/emailTemplateController/createEmailTemplate.md new file mode 100644 index 000000000..ae6f41996 --- /dev/null +++ b/requirements/emailTemplateController/createEmailTemplate.md @@ -0,0 +1,51 @@ +# Create Email Template Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to create email templates. + +3. ❌ **Returns error 400 if required fields are missing** + - Ensures that required fields (`name`, `subject`, `html_content`) are present in the request body. + +4. ❌ **Returns error 400 if template name is invalid or empty** + - Verifies that the template name is a non-empty string and meets validation requirements. + +5. ❌ **Returns error 400 if template subject is invalid or empty** + - Ensures that the template subject is a non-empty string and meets validation requirements. + +6. ❌ **Returns error 400 if template HTML content is invalid or empty** + - Verifies that the template HTML content is a non-empty string and meets validation requirements. + +7. ❌ **Returns error 400 if template variables are invalid** + - Ensures that if `variables` are provided, they follow the correct structure and types as defined in `EMAIL_CONFIG.TEMPLATE_VARIABLE_TYPES`. + +8. ❌ **Returns error 409 if template name already exists** + - Verifies that the function checks for duplicate template names (case-insensitive) and responds with a `409` status code if a template with the same name already exists. + +9. ❌ **Returns error 500 if there is an internal error while creating the template** + - Covers scenarios where there are database errors or service failures while creating the email template. + +10. ❌ **Returns validation errors in response if template data is invalid** + - Ensures that if template validation fails, the response includes an `errors` array with specific validation error messages. + +## Positive Cases + +1. ✅ **Returns status 201 when email template is successfully created** + - Ensures that the function successfully creates a new email template and returns it with a `201` status code. + +2. ✅ **Creates template with correct creator information** + - Verifies that the `created_by` and `updated_by` fields are set to the requestor's user ID. + +3. ✅ **Stores template variables correctly** + - Ensures that if `variables` are provided, they are stored correctly with proper structure and validation. + +4. ✅ **Trims and normalizes template fields** + - Verifies that template `name` and `subject` are trimmed of whitespace before storage. + +5. ❌ **Returns created template with all fields** + - Ensures that the response includes the complete template object with all fields, timestamps, and creator information. + diff --git a/requirements/emailTemplateController/deleteEmailTemplate.md b/requirements/emailTemplateController/deleteEmailTemplate.md new file mode 100644 index 000000000..5159e1692 --- /dev/null +++ b/requirements/emailTemplateController/deleteEmailTemplate.md @@ -0,0 +1,33 @@ +# Delete Email Template Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to delete email templates. + +3. ❌ **Returns error 400 if template ID is invalid** + - Ensures that the function validates the template ID is a valid MongoDB ObjectId format. + +4. ❌ **Returns error 404 if template is not found** + - Verifies that the function handles cases where no template exists with the provided ID and responds with a `404` status code. + +5. ❌ **Returns error 500 if there is an internal error while deleting the template** + - Covers scenarios where there are database errors or service failures while deleting the email template. + +## Positive Cases + +1. ✅ **Returns status 200 when email template is successfully deleted** + - Ensures that the function successfully deletes the email template and returns a success message with a `200` status code. + +2. ✅ **Performs hard delete (permanently removes template)** + - Verifies that the template is permanently removed from the database, not just marked as deleted. + +3. ✅ **Records deleter information before deletion** + - Ensures that the `updated_by` field is set to the requestor's user ID before deletion (if applicable). + +4. ❌ **Handles deletion gracefully (no error if already deleted)** + - Verifies that if the template is already deleted or doesn't exist, the function handles it gracefully without error. + diff --git a/requirements/emailTemplateController/getAllEmailTemplates.md b/requirements/emailTemplateController/getAllEmailTemplates.md new file mode 100644 index 000000000..9d78fefa2 --- /dev/null +++ b/requirements/emailTemplateController/getAllEmailTemplates.md @@ -0,0 +1,33 @@ +# Get All Email Templates Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` or `user.userid` in the request body/user object and responds with a `401` status code if both are missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to view email templates. + +3. ❌ **Returns error 500 if there is an internal error while fetching templates** + - Covers scenarios where there are database errors or service failures while fetching email templates. + +## Positive Cases + +1. ✅ **Returns status 200 with all email templates** + - Ensures that the function successfully fetches all email templates from the database and returns them with populated creator/updater information. + +2. ✅ **Supports search functionality by template name** + - Verifies that the function filters templates by name when a `search` query parameter is provided (case-insensitive search). + +3. ✅ **Supports sorting by specified field** + - Ensures that templates can be sorted by any specified field via the `sortBy` query parameter, defaulting to `created_at` descending if not specified. + +4. ✅ **Supports optional content projection** + - Verifies that when `includeEmailContent` is set to `'true'`, the response includes `subject`, `html_content`, and `variables` fields. When not included, only basic metadata is returned. + +5. ✅ **Returns templates with populated creator and updater information** + - Ensures that `created_by` and `updated_by` fields are populated with user profile information (firstName, lastName, email). + +6. ❌ **Handles empty search results gracefully** + - Verifies that if no templates match the search criteria, the function returns an empty array without error. + diff --git a/requirements/emailTemplateController/getEmailTemplateById.md b/requirements/emailTemplateController/getEmailTemplateById.md new file mode 100644 index 000000000..f012fd067 --- /dev/null +++ b/requirements/emailTemplateController/getEmailTemplateById.md @@ -0,0 +1,30 @@ +# Get Email Template By ID Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` or `user.userid` in the request body/user object and responds with a `401` status code if both are missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to view email templates. + +3. ❌ **Returns error 400 if template ID is invalid** + - Ensures that the function validates the template ID is a valid MongoDB ObjectId format. + +4. ❌ **Returns error 404 if template is not found** + - Verifies that the function handles cases where no template exists with the provided ID and responds with a `404` status code. + +5. ❌ **Returns error 500 if there is an internal error while fetching the template** + - Covers scenarios where there are database errors or service failures while fetching the email template. + +## Positive Cases + +1. ✅ **Returns status 200 with the requested email template** + - Ensures that the function successfully fetches the email template with the provided ID and returns all template details. + +2. ✅ **Returns template with populated creator and updater information** + - Verifies that `created_by` and `updated_by` fields are populated with user profile information (firstName, lastName, email). + +3. ✅ **Returns complete template data including variables** + - Ensures that the response includes all template fields: `name`, `subject`, `html_content`, `variables`, `created_by`, `updated_by`, and timestamps. + diff --git a/requirements/emailTemplateController/previewTemplate.md b/requirements/emailTemplateController/previewTemplate.md new file mode 100644 index 000000000..ac96453b3 --- /dev/null +++ b/requirements/emailTemplateController/previewTemplate.md @@ -0,0 +1,45 @@ +# Preview Email Template Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` or `user.userid` in the request body/user object and responds with a `401` status code if both are missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to preview email templates. + +3. ❌ **Returns error 400 if template ID is invalid** + - Ensures that the function validates the template ID is a valid MongoDB ObjectId format. + +4. ❌ **Returns error 404 if template is not found** + - Verifies that the function handles cases where no template exists with the provided ID and responds with a `404` status code. + +5. ❌ **Returns error 400 if provided variables are invalid** + - Ensures that the function validates provided variables match the template's variable definitions in type and required fields. + +6. ❌ **Returns error 400 if required variables are missing** + - Verifies that all required variables for the template are provided in the request body. + +7. ❌ **Returns error 500 if there is an internal error while rendering the template** + - Covers scenarios where there are errors during template rendering (e.g., invalid template syntax, variable replacement errors). + +## Positive Cases + +1. ✅ **Returns status 200 with rendered template preview** + - Ensures that the function successfully renders the template with the provided variables and returns the rendered `subject` and `html_content`. + +2. ✅ **Replaces all template variables correctly** + - Verifies that all variables in the template are replaced with the provided values in both `subject` and `html_content`. + +3. ✅ **Validates variables before rendering** + - Ensures that the function validates all provided variables match the template's variable definitions before attempting to render. + +4. ✅ **Does not sanitize content for preview** + - Verifies that the preview is rendered without sanitization to allow full preview of the final email content. + +5. ✅ **Returns both subject and content in preview** + - Ensures that the response includes both the rendered `subject` and `html_content` (or `content`) in the preview object. + +6. ❌ **Handles missing optional variables gracefully** + - Verifies that if optional variables are not provided, they are handled appropriately (not replaced or replaced with empty strings). + diff --git a/requirements/emailTemplateController/updateEmailTemplate.md b/requirements/emailTemplateController/updateEmailTemplate.md new file mode 100644 index 000000000..9c60efda7 --- /dev/null +++ b/requirements/emailTemplateController/updateEmailTemplate.md @@ -0,0 +1,54 @@ +# Update Email Template Function + +## Negative Cases + +1. ❌ **Returns error 401 if `requestor` is missing from the request** + - Ensures that the function checks for the presence of `requestor.requestorId` in the request body and responds with a `401` status code if it's missing. + +2. ❌ **Returns error 403 if user doesn't have `sendEmails` permission** + - Verifies that the function checks user permissions and responds with a `403` status code if the user is not authorized to update email templates. + +3. ❌ **Returns error 400 if template ID is invalid** + - Ensures that the function validates the template ID is a valid MongoDB ObjectId format. + +4. ❌ **Returns error 404 if template is not found** + - Verifies that the function handles cases where no template exists with the provided ID and responds with a `404` status code. + +5. ❌ **Returns error 400 if template name is invalid or empty** + - Verifies that if `name` is provided in the update, it is a non-empty string and meets validation requirements. + +6. ❌ **Returns error 400 if template subject is invalid or empty** + - Ensures that if `subject` is provided in the update, it is a non-empty string and meets validation requirements. + +7. ❌ **Returns error 400 if template HTML content is invalid or empty** + - Verifies that if `html_content` is provided in the update, it is a non-empty string and meets validation requirements. + +8. ❌ **Returns error 400 if template variables are invalid** + - Ensures that if `variables` are provided, they follow the correct structure and types as defined in `EMAIL_CONFIG.TEMPLATE_VARIABLE_TYPES`. + +9. ❌ **Returns error 409 if template name already exists (when updating name)** + - Verifies that if the template name is being changed, the function checks for duplicate names (case-insensitive) and responds with a `409` status code if a template with the new name already exists. + +10. ❌ **Returns error 500 if there is an internal error while updating the template** + - Covers scenarios where there are database errors or service failures while updating the email template. + +11. ❌ **Returns validation errors in response if template data is invalid** + - Ensures that if template validation fails, the response includes an `errors` array with specific validation error messages. + +## Positive Cases + +1. ✅ **Returns status 200 when email template is successfully updated** + - Ensures that the function successfully updates the email template and returns the updated template with a `200` status code. + +2. ✅ **Updates template with correct updater information** + - Verifies that the `updated_by` field is set to the requestor's user ID. + +3. ✅ **Updates only provided fields (partial update support)** + - Ensures that only the fields provided in the request body are updated, leaving other fields unchanged. + +4. ✅ **Trims and normalizes updated template fields** + - Verifies that updated `name` and `subject` are trimmed of whitespace before storage. + +5. ❌ **Returns updated template with all fields** + - Ensures that the response includes the complete updated template object with all fields, timestamps, and creator/updater information. + diff --git a/src/config/emailConfig.js b/src/config/emailConfig.js new file mode 100644 index 000000000..5964fc843 --- /dev/null +++ b/src/config/emailConfig.js @@ -0,0 +1,60 @@ +/** + * Email Configuration + * Centralized configuration for email announcement system + */ + +const EMAIL_CONFIG = { + // Retry configuration + DEFAULT_MAX_RETRIES: 3, + INITIAL_RETRY_DELAY_MS: 1000, + + // Status enums + EMAIL_STATUSES: { + PENDING: 'PENDING', // Created, waiting to be processed + SENDING: 'SENDING', // Currently sending + SENT: 'SENT', // All emails successfully accepted by SMTP server + PROCESSED: 'PROCESSED', // Processing finished (mixed results) + FAILED: 'FAILED', // Failed to send + }, + + EMAIL_BATCH_STATUSES: { + PENDING: 'PENDING', // Created, waiting to be processed + SENDING: 'SENDING', // Currently sending + SENT: 'SENT', // Successfully delivered + FAILED: 'FAILED', // Delivery failed + }, + + EMAIL_TYPES: { + TO: 'TO', + CC: 'CC', + BCC: 'BCC', + }, + + // Centralized limits to keep model, services, and controllers consistent + LIMITS: { + MAX_RECIPIENTS_PER_REQUEST: 2000, // Must match EmailBatch.recipients validator + MAX_HTML_BYTES: 1 * 1024 * 1024, // 1MB - Reduced since base64 media files are blocked + SUBJECT_MAX_LENGTH: 200, // Standardized subject length limit + TEMPLATE_NAME_MAX_LENGTH: 50, // Template name maximum length + }, + + // Template variable types + TEMPLATE_VARIABLE_TYPES: ['text', 'number', 'image', 'url', 'textarea'], + + // Announcement service runtime knobs + ANNOUNCEMENTS: { + BATCH_SIZE: 100, // recipients per SMTP send batch + CONCURRENCY: 3, // concurrent SMTP batches processed simultaneously + BATCH_STAGGER_START_MS: 100, // Delay between starting batches within a concurrent chunk (staggered start for rate limiting) + DELAY_BETWEEN_CHUNKS_MS: 1000, // Delay after a chunk of batches completes before starting the next chunk + MAX_QUEUE_SIZE: 100, // Maximum emails in processing queue to prevent memory leak + }, + + // Email configuration + EMAIL: { + SENDER: process.env.ANNOUNCEMENT_EMAIL, + SENDER_NAME: process.env.ANNOUNCEMENT_EMAIL_SENDER_NAME, + }, +}; + +module.exports = { EMAIL_CONFIG }; diff --git a/src/controllers/emailController.js b/src/controllers/emailController.js index c71abf7e2..fcac88947 100644 --- a/src/controllers/emailController.js +++ b/src/controllers/emailController.js @@ -1,259 +1,931 @@ // emailController.js +const mongoose = require('mongoose'); const jwt = require('jsonwebtoken'); -const cheerio = require('cheerio'); const emailSender = require('../utilities/emailSender'); -const { hasPermission } = require('../utilities/permissions'); +const { EMAIL_CONFIG } = require('../config/emailConfig'); +const { isValidEmailAddress, normalizeRecipientsToArray } = require('../utilities/emailValidators'); +const EmailTemplateService = require('../services/announcements/emails/emailTemplateService'); const EmailSubcriptionList = require('../models/emailSubcriptionList'); const userProfile = require('../models/userProfile'); +const EmailBatchService = require('../services/announcements/emails/emailBatchService'); +const EmailService = require('../services/announcements/emails/emailService'); +const emailProcessor = require('../services/announcements/emails/emailProcessor'); +const { hasPermission } = require('../utilities/permissions'); +const { withTransaction } = require('../utilities/transactionHelper'); +const config = require('../config'); +// const logger = require('../startup/logger'); -const frontEndUrl = process.env.FRONT_END_URL || 'http://localhost:3000'; -const jwtSecret = process.env.JWT_SECRET || 'EmailSecret'; - -const handleContentToOC = (htmlContent) => - ` - - - - - - ${htmlContent} - - `; - -const handleContentToNonOC = (htmlContent, email) => - ` - - - - - - ${htmlContent} -

Thank you for subscribing to our email updates!

-

If you would like to unsubscribe, please click here

- - `; - -function extractImagesAndCreateAttachments(html) { - const $ = cheerio.load(html); - const attachments = []; - - $('img').each((i, img) => { - const src = $(img).attr('src'); - if (src.startsWith('data:image')) { - const base64Data = src.split(',')[1]; - const _cid = `image-${i}`; - attachments.push({ - filename: `image-${i}.png`, - content: Buffer.from(base64Data, 'base64'), - cid: _cid, - }); - $(img).attr('src', `cid:${_cid}`); - } - }); - return { - html: $.html(), - attachments, - }; -} +const jwtSecret = process.env.JWT_SECRET; +/** + * Create an announcement Email for provided recipients. + * - Validates permissions, subject/html, recipients, and template variables. + * - Creates parent Email and chunked EmailBatch items in a transaction. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ const sendEmail = async (req, res) => { + // Requestor is required for permission check + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ success: false, message: 'Missing requestor' }); + } + + // Permission check const canSendEmail = await hasPermission(req.body.requestor, 'sendEmails'); if (!canSendEmail) { - res.status(403).send('You are not authorized to send emails.'); - return; + return res + .status(403) + .json({ success: false, message: 'You are not authorized to send emails.' }); } + try { const { to, subject, html } = req.body; - // Validate required fields - if (!subject || !html || !to) { - const missingFields = []; - if (!subject) missingFields.push('Subject'); - if (!html) missingFields.push('HTML content'); - if (!to) missingFields.push('Recipient email'); + + // Validate subject and html are not empty after trim + if (!subject || typeof subject !== 'string' || !subject.trim()) { + return res + .status(400) + .json({ success: false, message: 'Subject is required and cannot be empty' }); + } + if (!html || typeof html !== 'string' || !html.trim()) { return res .status(400) - .send(`${missingFields.join(' and ')} ${missingFields.length > 1 ? 'are' : 'is'} required`); + .json({ success: false, message: 'HTML content is required and cannot be empty' }); } - await emailSender(to, subject, html) - .then(() => { - res.status(200).send(`Email sent successfully to ${to}`); - }) - .catch(() => { - res.status(500).send('Error sending email'); + // Validate that all template variables have been replaced (business rule) + const unmatchedVariablesHtml = EmailTemplateService.getUnreplacedVariables(html); + const unmatchedVariablesSubject = EmailTemplateService.getUnreplacedVariables(subject); + const unmatchedVariables = [ + ...new Set([...unmatchedVariablesHtml, ...unmatchedVariablesSubject]), + ]; + if (unmatchedVariables.length > 0) { + return res.status(400).json({ + success: false, + message: + 'Email contains unreplaced template variables. Please ensure all variables are replaced before sending.', + unmatchedVariables, }); + } + + // Get user + const user = await userProfile.findById(req.body.requestor.requestorId); + if (!user) { + return res.status(404).json({ success: false, message: 'Requestor not found' }); + } + + // Normalize and deduplicate recipients (case-insensitive) + const recipientsArray = normalizeRecipientsToArray(to); + const uniqueRecipients = [ + ...new Set(recipientsArray.map((email) => email.toLowerCase().trim())), + ]; + const recipientObjects = uniqueRecipients.map((emailAddr) => ({ email: emailAddr })); + + // Create email and batches in transaction (validation happens in services) + // Queue email INSIDE transaction to ensure rollback on queue failure + const _email = await withTransaction(async (session) => { + // Create parent Email (validates subject, htmlContent, createdBy) + const createdEmail = await EmailService.createEmail( + { + subject, + htmlContent: html, + createdBy: user._id, + }, + session, + ); + + // Create EmailBatch items with all recipients (validates recipients, counts, email format) + // Enforce recipient limit for specific recipient requests + await EmailBatchService.createEmailBatches( + createdEmail._id, + recipientObjects, + { + emailType: EMAIL_CONFIG.EMAIL_TYPES.BCC, + enforceRecipientLimit: true, // Enforce limit for specific recipients + }, + session, + ); + + // Queue email BEFORE committing transaction + // If queue is full or queueing fails, transaction will rollback + const queued = emailProcessor.queueEmail(createdEmail._id); + if (!queued) { + const error = new Error( + 'Email queue is currently full. Please try again in a few moments or contact support if this persists.', + ); + error.statusCode = 503; // Service Unavailable + throw error; + } + + return createdEmail; + }); + + return res.status(200).json({ + success: true, + message: `Email queued for processing (${uniqueRecipients.length} recipient(s))`, + }); } catch (error) { - return res.status(500).send('Error sending email'); + // logger.logException(error, 'Error creating email'); + const statusCode = error.statusCode || 500; + const response = { + success: false, + message: error.message || 'Error creating email', + }; + // Include invalidRecipients if present (from service validation) + if (error.invalidRecipients) { + response.invalidRecipients = error.invalidRecipients; + } + return res.status(statusCode).json(response); } }; -const sendEmailToAll = async (req, res) => { - const canSendEmailToAll = await hasPermission(req.body.requestor, 'sendEmailToAll'); - if (!canSendEmailToAll) { - res.status(403).send('You are not authorized to send emails to all.'); - return; +/** + * Broadcast an announcement Email to all active HGN users and confirmed subscribers. + * - Validates permissions and content; creates Email and batches in a transaction. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const sendEmailToSubscribers = async (req, res) => { + // Requestor is required for permission check + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ success: false, message: 'Missing requestor' }); + } + + // Permission check - sendEmailToSubscribers requires sendEmails + const cansendEmailToSubscribers = await hasPermission(req.body.requestor, 'sendEmails'); + if (!cansendEmailToSubscribers) { + return res + .status(403) + .json({ success: false, message: 'You are not authorized to send emails to subscribers.' }); } + try { const { subject, html } = req.body; - if (!subject || !html) { - return res.status(400).send('Subject and HTML content are required'); + + // Validate subject and html are not empty after trim + if (!subject || typeof subject !== 'string' || !subject.trim()) { + return res + .status(400) + .json({ success: false, message: 'Subject is required and cannot be empty' }); + } + if (!html || typeof html !== 'string' || !html.trim()) { + return res + .status(400) + .json({ success: false, message: 'HTML content is required and cannot be empty' }); } - const { html: processedHtml, attachments } = extractImagesAndCreateAttachments(html); + // Validate that all template variables have been replaced (business rule) + const unmatchedVariablesHtml = EmailTemplateService.getUnreplacedVariables(html); + const unmatchedVariablesSubject = EmailTemplateService.getUnreplacedVariables(subject); + const unmatchedVariables = [ + ...new Set([...unmatchedVariablesHtml, ...unmatchedVariablesSubject]), + ]; + if (unmatchedVariables.length > 0) { + return res.status(400).json({ + success: false, + message: + 'Email contains unreplaced template variables. Please ensure all variables are replaced before sending.', + unmatchedVariables, + }); + } + + // Get user + const user = await userProfile.findById(req.body.requestor.requestorId); + if (!user) { + return res.status(404).json({ success: false, message: 'User not found' }); + } + // Get ALL recipients (HGN users + email subscribers) const users = await userProfile.find({ - firstName: '', + firstName: { $ne: '' }, email: { $ne: null }, isActive: true, emailSubscriptions: true, }); - if (users.length === 0) { - return res.status(404).send('No users found'); + + const emailSubscribers = await EmailSubcriptionList.find({ + email: { $exists: true, $nin: [null, ''] }, + isConfirmed: true, + emailSubscriptions: true, + }); + + // Collect all recipients and deduplicate (case-insensitive) + const allRecipientEmails = [ + ...users.map((hgnUser) => hgnUser.email), + ...emailSubscribers.map((subscriber) => subscriber.email), + ]; + + const uniqueRecipients = [ + ...new Set(allRecipientEmails.map((email) => email.toLowerCase().trim())), + ]; + const recipientObjects = uniqueRecipients.map((emailAddr) => ({ email: emailAddr })); + + if (uniqueRecipients.length === 0) { + return res.status(400).json({ success: false, message: 'No recipients found' }); } - const recipientEmails = users.map((user) => user.email); - console.log('# sendEmailToAll to', recipientEmails.join(',')); - if (recipientEmails.length === 0) { - throw new Error('No recipients defined'); + // Create email and batches in transaction (validation happens in services) + // Queue email INSIDE transaction to ensure rollback on queue failure + const _email = await withTransaction(async (session) => { + // Create parent Email (validates subject, htmlContent, createdBy) + const createdEmail = await EmailService.createEmail( + { + subject, + htmlContent: html, + createdBy: user._id, + }, + session, + ); + + // Create EmailBatch items with all recipients (validates recipients, counts, email format) + // Skip recipient limit for broadcast to all subscribers + await EmailBatchService.createEmailBatches( + createdEmail._id, + recipientObjects, + { + emailType: EMAIL_CONFIG.EMAIL_TYPES.BCC, + enforceRecipientLimit: false, // Skip limit for broadcast + }, + session, + ); + + // Queue email BEFORE committing transaction + const queued = emailProcessor.queueEmail(createdEmail._id); + if (!queued) { + const error = new Error( + 'Email queue is currently full. Please try again in a few moments or contact support if this persists.', + ); + error.statusCode = 503; + throw error; + } + + return createdEmail; + }); + + return res.status(200).json({ + success: true, + message: `Broadcast email queued for processing (${uniqueRecipients.length} recipient(s))`, + }); + } catch (error) { + // logger.logException(error, 'Error creating broadcast email'); + const statusCode = error.statusCode || 500; + const response = { + success: false, + message: error.message || 'Error creating broadcast email', + }; + // Include invalidRecipients if present (from service validation) + if (error.invalidRecipients) { + response.invalidRecipients = error.invalidRecipients; } - const emailContentToOCmembers = handleContentToOC(processedHtml); - await Promise.all( - recipientEmails.map((email) => - emailSender(email, subject, emailContentToOCmembers, attachments), - ), - ); - const emailSubscribers = await EmailSubcriptionList.find({ email: { $exists: true, $ne: '' } }); - console.log('# sendEmailToAll emailSubscribers', emailSubscribers.length); + return res.status(statusCode).json(response); + } +}; + +/** + * Resend a previously created Email to a selected audience. + * - Options: 'all' (users+subscribers), 'specific' (list), 'same' (original recipients). + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const resendEmail = async (req, res) => { + // Requestor is required for permission check + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ success: false, message: 'Missing requestor' }); + } + + // Permission check - resending requires sendEmails permission + const canSendEmail = await hasPermission(req.body.requestor, 'sendEmails'); + if (!canSendEmail) { + return res + .status(403) + .json({ success: false, message: 'You are not authorized to resend emails.' }); + } + + try { + const { emailId, recipientOption, specificRecipients } = req.body; + + // Validate emailId + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + return res.status(400).json({ success: false, message: 'Invalid emailId' }); + } + + // Get the original email (service throws error if not found) + const originalEmail = await EmailService.getEmailById(emailId, null, true); + + // Validate recipient option + if (!recipientOption) { + return res.status(400).json({ success: false, message: 'Recipient option is required' }); + } + + const validRecipientOptions = ['all', 'specific', 'same']; + if (!validRecipientOptions.includes(recipientOption)) { + return res.status(400).json({ + success: false, + message: `Invalid recipient option. Must be one of: ${validRecipientOptions.join(', ')}`, + }); + } + + // Get requestor user + const user = await userProfile.findById(req.body.requestor.requestorId); + if (!user) { + return res.status(400).json({ success: false, message: 'Requestor not found' }); + } + + let allRecipients = []; + + // Determine recipients based on option + if (recipientOption === 'all') { + // Get ALL recipients (HGN users + email subscribers) + const users = await userProfile.find({ + firstName: { $ne: '' }, + email: { $ne: null }, + isActive: true, + emailSubscriptions: true, + }); + + const emailSubscribers = await EmailSubcriptionList.find({ + email: { $exists: true, $nin: [null, ''] }, + isConfirmed: true, + emailSubscriptions: true, + }); + + allRecipients = [ + ...users.map((hgnUser) => hgnUser.email), + ...emailSubscribers.map((subscriber) => subscriber.email), + ].map((email) => ({ email })); + } else if (recipientOption === 'specific') { + // Use provided specific recipients + if ( + !specificRecipients || + !Array.isArray(specificRecipients) || + specificRecipients.length === 0 + ) { + return res.status(400).json({ + success: false, + message: 'specificRecipients array is required for specific option', + }); + } + + // Normalize recipients (validation happens in service) + const recipientsArray = normalizeRecipientsToArray(specificRecipients); + allRecipients = recipientsArray.map((email) => ({ email: email.toLowerCase().trim() })); + } else if (recipientOption === 'same') { + // Get recipients from original email's EmailBatch items + const emailBatchItems = await EmailBatchService.getBatchesForEmail(emailId); + if (!emailBatchItems || emailBatchItems.length === 0) { + return res + .status(404) + .json({ success: false, message: 'No recipients found in original email' }); + } + + // Extract all recipients from all EmailBatch items + const batchRecipients = emailBatchItems + .filter((batch) => batch.recipients && Array.isArray(batch.recipients)) + .flatMap((batch) => batch.recipients); + allRecipients.push(...batchRecipients); + } + + // Deduplicate all recipients (case-insensitive) + const allRecipientEmails = allRecipients.map((r) => r.email).filter(Boolean); + const uniqueRecipients = [ + ...new Set(allRecipientEmails.map((email) => email.toLowerCase().trim())), + ]; + const recipientObjects = uniqueRecipients.map((emailAddr) => ({ email: emailAddr })); + + if (uniqueRecipients.length === 0) { + return res.status(400).json({ success: false, message: 'No recipients found' }); + } + + // Create email and batches in transaction + // Queue email INSIDE transaction to ensure rollback on queue failure + const newEmail = await withTransaction(async (session) => { + // Create new Email (copy) - validation happens in service + const createdEmail = await EmailService.createEmail( + { + subject: originalEmail.subject, + htmlContent: originalEmail.htmlContent, + createdBy: user._id, + }, + session, + ); + + // Create EmailBatch items + // Enforce limit only for 'specific' recipient option, skip for 'all' and 'same' (broadcast scenarios) + const shouldEnforceLimit = recipientOption === 'specific'; + await EmailBatchService.createEmailBatches( + createdEmail._id, + recipientObjects, + { + emailType: EMAIL_CONFIG.EMAIL_TYPES.BCC, + enforceRecipientLimit: shouldEnforceLimit, + }, + session, + ); + + // Queue email BEFORE committing transaction + const queued = emailProcessor.queueEmail(createdEmail._id); + if (!queued) { + const error = new Error( + 'Email queue is currently full. Please try again in a few moments or contact support if this persists.', + ); + error.statusCode = 503; + throw error; + } + + return createdEmail; + }); + + return res.status(200).json({ + success: true, + message: `Email queued for resend (${uniqueRecipients.length} recipient(s))`, + data: { + emailId: newEmail._id, + recipientCount: uniqueRecipients.length, + }, + }); + } catch (error) { + // logger.logException(error, 'Error resending email'); + const statusCode = error.statusCode || 500; + const response = { + success: false, + message: error.message || 'Error resending email', + }; + // Include invalidRecipients if present (from service validation) + if (error.invalidRecipients) { + response.invalidRecipients = error.invalidRecipients; + } + return res.status(statusCode).json(response); + } +}; + +/** + * Retry a parent Email by resetting all FAILED EmailBatch items to PENDING. + * - Processes the email immediately asynchronously. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const retryEmail = async (req, res) => { + try { + const { emailId } = req.params; + + // Requestor is required for permission check + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ success: false, message: 'Missing requestor' }); + } + + // Validate emailId is a valid ObjectId + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + return res.status(400).json({ + success: false, + message: 'Invalid Email ID', + }); + } + + // Permission check - retrying emails requires sendEmails permission + const canRetryEmail = await hasPermission(req.body.requestor, 'sendEmails'); + if (!canRetryEmail) { + return res + .status(403) + .json({ success: false, message: 'You are not authorized to retry emails.' }); + } + + // Get the Email (service throws error if not found) + const email = await EmailService.getEmailById(emailId, null, true); + + // Only allow retry for emails in final states (FAILED or PROCESSED) + const allowedRetryStatuses = [ + EMAIL_CONFIG.EMAIL_STATUSES.FAILED, + EMAIL_CONFIG.EMAIL_STATUSES.PROCESSED, + ]; + + if (!allowedRetryStatuses.includes(email.status)) { + return res.status(400).json({ + success: false, + message: `Email must be in FAILED or PROCESSED status to retry. Current status: ${email.status}`, + }); + } + + // Get all FAILED EmailBatch items (service validates emailId) + const failedItems = await EmailBatchService.getFailedBatchesForEmail(emailId); + + if (failedItems.length === 0) { + // logger.logInfo(`Email ${emailId} has no failed EmailBatch items to retry`); + return res.status(200).json({ + success: true, + message: 'No failed EmailBatch items to retry', + data: { + emailId: email._id, + failedItemsRetried: 0, + }, + }); + } + + // logger.logInfo(`Retrying ${failedItems.length} failed EmailBatch items: ${emailId}`); + + // Mark parent Email as PENDING for retry + await EmailService.markEmailPending(emailId); + + // Reset each failed item to PENDING await Promise.all( - emailSubscribers.map(({ email }) => { - const emailContentToNonOCmembers = handleContentToNonOC(processedHtml, email); - return emailSender(email, subject, emailContentToNonOCmembers, attachments); + failedItems.map(async (item) => { + await EmailBatchService.resetEmailBatchForRetry(item._id); }), ); - return res.status(200).send('Email sent successfully'); + + // logger.logInfo( + // `Successfully reset Email ${emailId} and ${failedItems.length} failed EmailBatch items to PENDING for retry`, + // ); + + // Add email to queue for processing (non-blocking, sequential processing) + const queued = emailProcessor.queueEmail(emailId); + if (!queued) { + return res.status(503).json({ + success: false, + message: + 'Email queue is currently full. Your email has been reset to PENDING and will be processed automatically when the server restarts, or you can use the "Process Pending Emails" button to retry manually.', + }); + } + + res.status(200).json({ + success: true, + message: `Successfully reset ${failedItems.length} failed EmailBatch items for retry`, + data: { + emailId: email._id, + failedItemsRetried: failedItems.length, + }, + }); } catch (error) { - console.error('Error sending email:', error); - return res.status(500).send('Error sending email'); + // logger.logException(error, 'Error retrying Email'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: error.message || 'Error retrying Email', + }); } }; +/** + * Manually trigger processing of pending and stuck emails. + * - Resets stuck emails (SENDING status) to PENDING + * - Resets stuck batches (SENDING status) to PENDING + * - Queues all PENDING emails for processing + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const processPendingAndStuckEmails = async (req, res) => { + try { + // Requestor is required for permission check + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ success: false, message: 'Missing requestor' }); + } + + // Permission check - processing stuck emails requires sendEmails permission + const canProcessEmails = await hasPermission(req.body.requestor, 'sendEmails'); + if (!canProcessEmails) { + return res + .status(403) + .json({ success: false, message: 'You are not authorized to process emails.' }); + } + + // Trigger processing and get statistics + const stats = await emailProcessor.processPendingAndStuckEmails(); + + // Build user-friendly message + const parts = []; + if (stats.stuckEmailsReset > 0) { + parts.push(`${stats.stuckEmailsReset} stuck email(s) reset`); + } + if (stats.stuckBatchesReset > 0) { + parts.push(`${stats.stuckBatchesReset} stuck batch(es) reset`); + } + if (stats.runtimeStuckBatchesReset > 0) { + parts.push(`${stats.runtimeStuckBatchesReset} timeout batch(es) reset`); + } + if (stats.pendingEmailsQueued > 0) { + parts.push(`${stats.pendingEmailsQueued} pending email(s) queued`); + } + + const message = + parts.length > 0 + ? `Recovery complete: ${parts.join(', ')}` + : 'No stuck or pending emails found - all clear!'; + + return res.status(200).json({ + success: true, + message, + data: stats, + }); + } catch (error) { + // logger.logException(error, 'Error processing pending and stuck emails'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: error.message || 'Error processing pending and stuck emails', + }); + } +}; + +/** + * Update the current user's emailSubscriptions preference. + * - Normalizes email to lowercase for consistent lookups. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ const updateEmailSubscriptions = async (req, res) => { try { + if (!req?.body?.requestor?.email) { + return res.status(401).json({ success: false, message: 'Missing requestor email' }); + } + const { emailSubscriptions } = req.body; + if (typeof emailSubscriptions !== 'boolean') { + return res + .status(400) + .json({ success: false, message: 'emailSubscriptions must be a boolean value' }); + } + const { email } = req.body.requestor; + if (!isValidEmailAddress(email)) { + return res.status(400).json({ success: false, message: 'Invalid email address' }); + } + + // Normalize email for consistent lookup + const normalizedEmail = email.trim().toLowerCase(); + const user = await userProfile.findOneAndUpdate( - { email }, + { email: normalizedEmail }, { emailSubscriptions }, { new: true }, ); - return res.status(200).send(user); + + if (!user) { + return res.status(404).json({ success: false, message: 'User not found' }); + } + + return res + .status(200) + .json({ success: true, message: 'Email subscription updated successfully' }); } catch (error) { - console.error('Error updating email subscriptions:', error); - return res.status(500).send('Error updating email subscriptions'); + // logger.logException(error, 'Error updating email subscriptions'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: 'Error updating email subscriptions', + }); } }; +/** + * Add a non-HGN user's email to the subscription list and send confirmation. + * - Rejects if already an HGN user or already subscribed. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ const addNonHgnEmailSubscription = async (req, res) => { try { const { email } = req.body; - if (!email) { - return res.status(400).send('Email is required'); + if (!email || typeof email !== 'string') { + return res.status(400).json({ success: false, message: 'Email is required' }); } - const emailList = await EmailSubcriptionList.find({ email: { $eq: email } }); - if (emailList.length > 0) { - return res.status(400).send('Email already exists'); + // Normalize and validate email + const normalizedEmail = email.trim().toLowerCase(); + if (!isValidEmailAddress(normalizedEmail)) { + return res.status(400).json({ success: false, message: 'Invalid email address' }); + } + + // Check if email already exists (direct match since schema enforces lowercase) + const existingSubscription = await EmailSubcriptionList.findOne({ + email: normalizedEmail, + }); + + if (existingSubscription) { + return res.status(400).json({ success: false, message: 'Email already subscribed' }); + } + + // check if this email is already in the HGN user list + const hgnUser = await userProfile.findOne({ email: normalizedEmail }); + if (hgnUser) { + return res.status(400).json({ + success: false, + message: 'Please use the HGN account profile page to subscribe to email updates.', + }); } - // Save to DB immediately - const newEmailList = new EmailSubcriptionList({ email }); + // Save to DB immediately with confirmation pending + const newEmailList = new EmailSubcriptionList({ + email: normalizedEmail, + isConfirmed: false, + emailSubscriptions: true, + }); await newEmailList.save(); - // Optional: Still send confirmation email - const payload = { email }; - const token = jwt.sign(payload, jwtSecret, { expiresIn: 360 }); + // Send confirmation email + if (!jwtSecret) { + return res.status(500).json({ + success: false, + message: 'Server configuration error. JWT_SECRET is not set.', + }); + } + const payload = { email: normalizedEmail }; + const token = jwt.sign(payload, jwtSecret, { expiresIn: '24h' }); + + // Get frontend URL from request origin + const getFrontendUrl = () => { + // Try to get from request origin header first + const origin = req.get('origin') || req.get('referer'); + if (origin) { + try { + const url = new URL(origin); + return `${url.protocol}//${url.host}`; + } catch (error) { + // logger.logException(error, 'Error parsing request origin'); + } + } + // Fallback to config or construct from request + if (config.FRONT_END_URL) { + return config.FRONT_END_URL; + } + // Last resort: construct from request + const protocol = req.protocol || 'https'; + const host = req.get('host'); + if (host) { + return `${protocol}://${host}`; + } + return null; + }; + + const frontendUrl = getFrontendUrl(); + if (!frontendUrl) { + // logger.logException( + // new Error('Unable to determine frontend URL from request'), + // 'Configuration error', + // ); + return res + .status(500) + .json({ success: false, message: 'Server Error. Please contact support.' }); + } + const emailContent = ` - - - - -

Thank you for subscribing to our email updates!

-

Click here to confirm your email

- - +

Thank you for subscribing to our email updates!

+

Click here to confirm your email

`; - emailSender(email, 'HGN Email Subscription', emailContent); - return res.status(200).send('Email subscribed successfully'); + try { + await emailSender( + normalizedEmail, + 'HGN Email Subscription', + emailContent, + null, + null, + null, + null, + { type: 'subscription_confirmation' }, + ); + return res.status(200).json({ + success: true, + message: 'Email subscribed successfully. Please check your inbox to confirm.', + }); + } catch (emailError) { + // logger.logException(emailError, 'Error sending confirmation email'); + // Still return success since the subscription was saved to DB + return res.status(200).json({ + success: true, + message: + 'Email subscribed successfully. Confirmation email failed to send. Please contact support.', + }); + } } catch (error) { - console.error('Error adding email subscription:', error); - res.status(500).send('Error adding email subscription'); + // logger.logException(error, 'Error adding email subscription'); + if (error.code === 11000) { + return res.status(400).json({ success: false, message: 'Email already subscribed' }); + } + return res.status(500).json({ success: false, message: 'Error adding email subscription' }); } }; +/** + * Confirm a non-HGN email subscription using a signed token. + * - Only confirms existing unconfirmed subscriptions. + * - Returns error if subscription doesn't exist (user must subscribe first). + * @param {import('express').Request} req + * @param {import('express').Response} res + */ const confirmNonHgnEmailSubscription = async (req, res) => { try { const { token } = req.body; - if (!token) { - return res.status(400).send('Invalid token'); + if (!token || typeof token !== 'string') { + return res.status(400).json({ success: false, message: 'Token is required' }); + } + + if (!jwtSecret) { + return res.status(500).json({ + success: false, + message: 'Server configuration error. JWT_SECRET is not set.', + }); } + let payload = {}; try { payload = jwt.verify(token, jwtSecret); } catch (err) { - // console.log(err); - return res.status(401).json({ errors: [{ msg: 'Token is not valid' }] }); + return res.status(401).json({ success: false, message: 'Invalid or expired token' }); } + const { email } = payload; - if (!email) { - return res.status(400).send('Invalid token'); + if (!email || !isValidEmailAddress(email)) { + return res.status(400).json({ success: false, message: 'Invalid token payload' }); } - try { - const newEmailList = new EmailSubcriptionList({ email }); - await newEmailList.save(); - } catch (error) { - if (error.code === 11000) { - return res.status(200).send('Email already exists'); - } + + // Normalize email (schema enforces lowercase, but normalize here for consistency) + const normalizedEmail = email.trim().toLowerCase(); + + // Find existing subscription (direct match since schema enforces lowercase) + const existingSubscription = await EmailSubcriptionList.findOne({ + email: normalizedEmail, + }); + + if (!existingSubscription) { + return res.status(404).json({ + success: false, + message: 'Subscription not found. Please subscribe first using the subscription form.', + }); + } + + // If already confirmed, return success (idempotent) + if (existingSubscription.isConfirmed) { + return res.status(200).json({ + success: true, + message: 'Email subscription already confirmed', + }); } - // console.log('email', email); - return res.status(200).send('Email subsribed successfully'); + + // Update subscription to confirmed + existingSubscription.isConfirmed = true; + existingSubscription.confirmedAt = new Date(); + existingSubscription.emailSubscriptions = true; + await existingSubscription.save(); + + return res + .status(200) + .json({ success: true, message: 'Email subscription confirmed successfully' }); } catch (error) { - console.error('Error updating email subscriptions:', error); - return res.status(500).send('Error updating email subscriptions'); + // logger.logException(error, 'Error confirming email subscription'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: error.message || 'Error confirming email subscription', + }); } }; +/** + * Remove a non-HGN email from the subscription list (unsubscribe). + * @param {import('express').Request} req + * @param {import('express').Response} res + */ const removeNonHgnEmailSubscription = async (req, res) => { try { const { email } = req.body; // Validate input - if (!email) { - return res.status(400).send('Email is required'); + if (!email || typeof email !== 'string') { + return res.status(400).json({ success: false, message: 'Email is required' }); } - // Try to delete the email + // Normalize email + const normalizedEmail = email.trim().toLowerCase(); + if (!isValidEmailAddress(normalizedEmail)) { + return res.status(400).json({ success: false, message: 'Invalid email address' }); + } + + // Try to delete the email subscription (direct match since schema enforces lowercase) const deletedEntry = await EmailSubcriptionList.findOneAndDelete({ - email: { $eq: email }, + email: normalizedEmail, }); // If not found, respond accordingly if (!deletedEntry) { - return res.status(404).send('Email not found or already unsubscribed'); + return res + .status(404) + .json({ success: false, message: 'Email not found or already unsubscribed' }); } - return res.status(200).send('Email unsubscribed successfully'); + return res.status(200).json({ success: true, message: 'Email unsubscribed successfully' }); } catch (error) { - return res.status(500).send('Server error while unsubscribing'); + // logger.logException(error, 'Error removing email subscription'); + return res.status(500).json({ success: false, message: 'Error removing email subscription' }); } }; module.exports = { sendEmail, - sendEmailToAll, + sendEmailToSubscribers, + resendEmail, updateEmailSubscriptions, addNonHgnEmailSubscription, removeNonHgnEmailSubscription, confirmNonHgnEmailSubscription, + retryEmail, + processPendingAndStuckEmails, }; diff --git a/src/controllers/emailController.spec.js b/src/controllers/emailController.spec.js index e4a48ab87..43c503a80 100644 --- a/src/controllers/emailController.spec.js +++ b/src/controllers/emailController.spec.js @@ -11,7 +11,7 @@ jest.mock('../utilities/emailSender'); const makeSut = () => { const { sendEmail, - sendEmailToAll, + sendEmailToSubscribers, updateEmailSubscriptions, addNonHgnEmailSubscription, removeNonHgnEmailSubscription, @@ -19,7 +19,7 @@ const makeSut = () => { } = emailController; return { sendEmail, - sendEmailToAll, + sendEmailToSubscribers, updateEmailSubscriptions, addNonHgnEmailSubscription, removeNonHgnEmailSubscription, @@ -75,16 +75,20 @@ describe('updateEmailSubscriptions function', () => { const updateReq = { body: { - emailSubscriptions: ['subscription1', 'subscription2'], + emailSubscriptions: true, requestor: { email: 'test@example.com', }, }, }; - const response = await updateEmailSubscriptions(updateReq, mockRes); + await updateEmailSubscriptions(updateReq, mockRes); - assertResMock(500, 'Error updating email subscriptions', response, mockRes); + expect(mockRes.status).toHaveBeenCalledWith(500); + expect(mockRes.json).toHaveBeenCalledWith({ + success: false, + message: 'Error updating email subscriptions', + }); }); }); @@ -101,9 +105,13 @@ describe('confirmNonHgnEmailSubscription function', () => { const { confirmNonHgnEmailSubscription } = makeSut(); const emptyReq = { body: {} }; - const response = await confirmNonHgnEmailSubscription(emptyReq, mockRes); + await confirmNonHgnEmailSubscription(emptyReq, mockRes); - assertResMock(400, 'Invalid token', response, mockRes); + expect(mockRes.status).toHaveBeenCalledWith(400); + expect(mockRes.json).toHaveBeenCalledWith({ + success: false, + message: 'Token is required', + }); }); test('should return 401 if token is invalid', async () => { @@ -118,7 +126,8 @@ describe('confirmNonHgnEmailSubscription function', () => { expect(mockRes.status).toHaveBeenCalledWith(401); expect(mockRes.json).toHaveBeenCalledWith({ - errors: [{ msg: 'Token is not valid' }], + success: false, + message: 'Invalid or expired token', }); }); @@ -129,9 +138,13 @@ describe('confirmNonHgnEmailSubscription function', () => { // Mocking jwt.verify to return a payload without email jwt.verify.mockReturnValue({}); - const response = await confirmNonHgnEmailSubscription(validTokenReq, mockRes); + await confirmNonHgnEmailSubscription(validTokenReq, mockRes); - assertResMock(400, 'Invalid token', response, mockRes); + expect(mockRes.status).toHaveBeenCalledWith(400); + expect(mockRes.json).toHaveBeenCalledWith({ + success: false, + message: 'Invalid token payload', + }); }); }); @@ -144,8 +157,12 @@ describe('removeNonHgnEmailSubscription function', () => { const { removeNonHgnEmailSubscription } = makeSut(); const noEmailReq = { body: {} }; - const response = await removeNonHgnEmailSubscription(noEmailReq, mockRes); + await removeNonHgnEmailSubscription(noEmailReq, mockRes); - assertResMock(400, 'Email is required', response, mockRes); + expect(mockRes.status).toHaveBeenCalledWith(400); + expect(mockRes.json).toHaveBeenCalledWith({ + success: false, + message: 'Email is required', + }); }); }); diff --git a/src/controllers/emailOutboxController.js b/src/controllers/emailOutboxController.js new file mode 100644 index 000000000..771bdf5ec --- /dev/null +++ b/src/controllers/emailOutboxController.js @@ -0,0 +1,84 @@ +const EmailBatchService = require('../services/announcements/emails/emailBatchService'); +const EmailService = require('../services/announcements/emails/emailService'); +const { hasPermission } = require('../utilities/permissions'); +// const logger = require('../startup/logger'); + +/** + * Get all announcement Email records (parent documents) - Outbox view. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const getEmails = async (req, res) => { + try { + // Requestor validation + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ success: false, message: 'Missing requestor' }); + } + + // Permission check - viewing emails requires sendEmails permission + const canViewEmails = await hasPermission(req.body.requestor, 'sendEmails'); + if (!canViewEmails) { + return res + .status(403) + .json({ success: false, message: 'You are not authorized to view emails.' }); + } + + const emails = await EmailService.getAllEmails(); + + res.status(200).json({ + success: true, + data: emails, + }); + } catch (error) { + // logger.logException(error, 'Error getting emails'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: error.message || 'Error getting emails', + }); + } +}; + +/** + * Get a parent Email and its associated EmailBatch items - Outbox detail view. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const getEmailDetails = async (req, res) => { + try { + // Requestor validation + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ success: false, message: 'Missing requestor' }); + } + + // Permission check - viewing email details requires sendEmails permission + const canViewEmails = await hasPermission(req.body.requestor, 'sendEmails'); + if (!canViewEmails) { + return res + .status(403) + .json({ success: false, message: 'You are not authorized to view email details.' }); + } + + const { emailId } = req.params; // emailId is now the ObjectId of parent Email + + // Service validates emailId and throws error if not found + const result = await EmailBatchService.getEmailWithBatches(emailId); + + res.status(200).json({ + success: true, + data: result, + }); + } catch (error) { + // logger.logException(error, 'Error getting Email details with EmailBatch items'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: error.message || 'Error getting email details', + }); + } +}; + +module.exports = { + getEmails, + getEmailDetails, +}; diff --git a/src/controllers/emailTemplateController.js b/src/controllers/emailTemplateController.js new file mode 100644 index 000000000..edc189e67 --- /dev/null +++ b/src/controllers/emailTemplateController.js @@ -0,0 +1,366 @@ +/** + * Email Template Controller - Handles HTTP requests for email template operations + */ + +const EmailTemplateService = require('../services/announcements/emails/emailTemplateService'); +const { hasPermission } = require('../utilities/permissions'); +// const logger = require('../startup/logger'); + +/** + * Get all email templates (with basic search/sort and optional content projection). + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const getAllEmailTemplates = async (req, res) => { + try { + // Permission check - use sendEmails permission to view templates + if (!req?.body?.requestor?.requestorId && !req?.user?.userid) { + return res.status(401).json({ + success: false, + message: 'Missing requestor', + }); + } + + const requestor = req.body.requestor || req.user; + const canViewTemplates = await hasPermission(requestor, 'sendEmails'); + if (!canViewTemplates) { + return res.status(403).json({ + success: false, + message: 'You are not authorized to view email templates.', + }); + } + + const { search, sortBy, sortOrder, includeEmailContent } = req.query; + + const query = {}; + const sort = {}; + + // Add search functionality + if (search && search.trim()) { + query.$or = [{ name: { $regex: search.trim(), $options: 'i' } }]; + } + + // Build sort object + if (sortBy) { + // Use sortOrder if provided (asc = 1, desc = -1), otherwise default to ascending + const order = sortOrder === 'desc' ? -1 : 1; + sort[sortBy] = order; + } else { + sort.created_at = -1; + } + + // Build projection + let projection = '_id name created_by updated_by created_at updated_at'; + if (includeEmailContent === 'true') { + projection += ' subject html_content variables'; + } + + const templates = await EmailTemplateService.getAllTemplates(query, { + sort, + projection, + populate: true, + }); + + res.status(200).json({ + success: true, + templates, + }); + } catch (error) { + // logger.logException(error, 'Error fetching email templates'); + const statusCode = error.statusCode || 500; + const response = { + success: false, + message: error.message || 'Error fetching email templates', + }; + if (error.errors && Array.isArray(error.errors)) { + response.errors = error.errors; + } + return res.status(statusCode).json(response); + } +}; + +/** + * Get a single email template by ID. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const getEmailTemplateById = async (req, res) => { + try { + // Permission check - use sendEmails permission to view templates + if (!req?.body?.requestor?.requestorId && !req?.user?.userid) { + return res.status(401).json({ + success: false, + message: 'Missing requestor', + }); + } + + const requestor = req.body.requestor || req.user; + const canViewTemplates = await hasPermission(requestor, 'sendEmails'); + if (!canViewTemplates) { + return res.status(403).json({ + success: false, + message: 'You are not authorized to view email templates.', + }); + } + + const { id } = req.params; + + // Service validates ID and throws error with statusCode if not found + const template = await EmailTemplateService.getTemplateById(id, { + populate: true, + }); + + res.status(200).json({ + success: true, + template, + }); + } catch (error) { + // logger.logException(error, 'Error fetching email template'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: error.message || 'Error fetching email template', + }); + } +}; + +/** + * Create a new email template. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const createEmailTemplate = async (req, res) => { + try { + // Requestor is required for permission check + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ + success: false, + message: 'Missing requestor', + }); + } + + // Permission check - use sendEmails permission to create templates + const canCreateTemplate = await hasPermission(req.body.requestor, 'sendEmails'); + if (!canCreateTemplate) { + return res.status(403).json({ + success: false, + message: 'You are not authorized to create email templates.', + }); + } + + const { name, subject, html_content: htmlContent, variables } = req.body; + const userId = req.body.requestor.requestorId; + + const templateData = { + name, + subject, + html_content: htmlContent, + variables, + }; + + const template = await EmailTemplateService.createTemplate(templateData, userId); + + res.status(201).json({ + success: true, + message: 'Email template created successfully', + template, + }); + } catch (error) { + // logger.logException(error, 'Error creating email template'); + const statusCode = error.statusCode || 500; + const response = { + success: false, + message: error.message || 'Error creating email template', + }; + // Always include detailed errors array if available + if (error.errors && Array.isArray(error.errors) && error.errors.length > 0) { + response.errors = error.errors; + // If message is vague, enhance it with error count + if ( + response.message === 'Invalid template data' || + response.message === 'Error creating email template' + ) { + response.message = `Validation failed: ${error.errors.length} error(s) found`; + } + } + return res.status(statusCode).json(response); + } +}; + +/** + * Update an existing email template by ID. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const updateEmailTemplate = async (req, res) => { + try { + // Requestor is required for permission check + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ + success: false, + message: 'Missing requestor', + }); + } + + // Permission check - use sendEmails permission to update templates + const canUpdateTemplate = await hasPermission(req.body.requestor, 'sendEmails'); + if (!canUpdateTemplate) { + return res.status(403).json({ + success: false, + message: 'You are not authorized to update email templates.', + }); + } + + const { id } = req.params; + const { name, subject, html_content: htmlContent, variables } = req.body; + const userId = req.body.requestor.requestorId; + + const templateData = { + name, + subject, + html_content: htmlContent, + variables, + }; + + const template = await EmailTemplateService.updateTemplate(id, templateData, userId); + + res.status(200).json({ + success: true, + message: 'Email template updated successfully', + template, + }); + } catch (error) { + // logger.logException(error, 'Error updating email template'); + const statusCode = error.statusCode || 500; + const response = { + success: false, + message: error.message || 'Error updating email template', + }; + // Always include detailed errors array if available + if (error.errors && Array.isArray(error.errors) && error.errors.length > 0) { + response.errors = error.errors; + // If message is vague, enhance it with error count + if ( + response.message === 'Invalid template data' || + response.message === 'Error updating email template' + ) { + response.message = `Validation failed: ${error.errors.length} error(s) found`; + } + } + return res.status(statusCode).json(response); + } +}; + +/** + * Delete an email template by ID (hard delete). + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const deleteEmailTemplate = async (req, res) => { + try { + // Requestor is required for permission check + if (!req?.body?.requestor?.requestorId) { + return res.status(401).json({ + success: false, + message: 'Missing requestor', + }); + } + + // Permission check - use sendEmails permission to delete templates + const canDeleteTemplate = await hasPermission(req.body.requestor, 'sendEmails'); + if (!canDeleteTemplate) { + return res.status(403).json({ + success: false, + message: 'You are not authorized to delete email templates.', + }); + } + + const { id } = req.params; + + await EmailTemplateService.deleteTemplate(id); + + res.status(200).json({ + success: true, + message: 'Email template deleted successfully', + }); + } catch (error) { + // logger.logException(error, 'Error deleting email template'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: error.message || 'Error deleting email template', + }); + } +}; + +/** + * Preview template with variable values. + * @param {import('express').Request} req + * @param {import('express').Response} res + */ +const previewTemplate = async (req, res) => { + try { + // Permission check + if (!req?.body?.requestor?.requestorId && !req?.user?.userid) { + return res.status(401).json({ + success: false, + message: 'Missing requestor', + }); + } + + const requestor = req.body.requestor || req.user; + const canViewTemplates = await hasPermission(requestor, 'sendEmails'); + if (!canViewTemplates) { + return res.status(403).json({ + success: false, + message: 'You are not authorized to preview email templates.', + }); + } + + const { id } = req.params; + const { variables = {} } = req.body; + + // Service validates ID and throws error with statusCode if not found + const template = await EmailTemplateService.getTemplateById(id, { + populate: false, + }); + + // Validate variables + const validation = EmailTemplateService.validateVariables(template, variables); + if (!validation.isValid) { + return res.status(400).json({ + success: false, + message: 'Invalid variables', + errors: validation.errors, + missing: validation.missing, + }); + } + + // Render template + const rendered = EmailTemplateService.renderTemplate(template, variables, { + sanitize: false, // Don't sanitize for preview + strict: false, + }); + + res.status(200).json({ + success: true, + preview: rendered, + }); + } catch (error) { + // logger.logException(error, 'Error previewing email template'); + const statusCode = error.statusCode || 500; + return res.status(statusCode).json({ + success: false, + message: error.message || 'Error previewing email template', + }); + } +}; + +module.exports = { + getAllEmailTemplates, + getEmailTemplateById, + createEmailTemplate, + updateEmailTemplate, + deleteEmailTemplate, + previewTemplate, +}; diff --git a/src/models/email.js b/src/models/email.js new file mode 100644 index 000000000..63d9300e9 --- /dev/null +++ b/src/models/email.js @@ -0,0 +1,53 @@ +const mongoose = require('mongoose'); +const { EMAIL_CONFIG } = require('../config/emailConfig'); + +/** + * Email (parent) model for announcement sending lifecycle. + * - Stores subject/html and status transitions (PENDING → SENDING → SENT/PROCESSED/FAILED). + * - References creator and tracks timing fields for auditing. + */ +const { Schema } = mongoose; + +const EmailSchema = new Schema({ + subject: { + type: String, + required: [true, 'Subject is required'], + }, + htmlContent: { + type: String, + required: [true, 'HTML content is required'], + }, + status: { + type: String, + enum: Object.values(EMAIL_CONFIG.EMAIL_STATUSES), + default: EMAIL_CONFIG.EMAIL_STATUSES.PENDING, + index: true, + }, + createdBy: { + type: Schema.Types.ObjectId, + ref: 'userProfile', + required: [true, 'createdBy is required'], + }, + createdAt: { type: Date, default: () => new Date(), index: true }, + startedAt: { + type: Date, + }, + completedAt: { + type: Date, + }, + updatedAt: { type: Date, default: () => new Date() }, +}); + +// Update timestamps and validate basic constraints +EmailSchema.pre('save', function (next) { + this.updatedAt = new Date(); + next(); +}); + +// Add indexes for better performance +EmailSchema.index({ status: 1, createdAt: 1 }); +EmailSchema.index({ createdBy: 1, createdAt: -1 }); +EmailSchema.index({ startedAt: 1 }); +EmailSchema.index({ completedAt: 1 }); + +module.exports = mongoose.model('Email', EmailSchema, 'emails'); diff --git a/src/models/emailBatch.js b/src/models/emailBatch.js new file mode 100644 index 000000000..abb2326ba --- /dev/null +++ b/src/models/emailBatch.js @@ -0,0 +1,109 @@ +const mongoose = require('mongoose'); +const { EMAIL_CONFIG } = require('../config/emailConfig'); + +/** + * EmailBatch (child) model representing one SMTP send to a group of recipients. + * - Tracks recipients, emailType, status, attempt counters and error snapshots. + */ +const { Schema } = mongoose; + +const EmailBatchSchema = new Schema({ + // Email reference + emailId: { + type: Schema.Types.ObjectId, + ref: 'Email', + required: [true, 'emailId is required'], + index: true, + }, + + // Multiple recipients in one batch item (emails only) + recipients: { + type: [ + { + _id: false, // Prevent MongoDB from generating _id for each recipient + email: { + type: String, + required: [true, 'Email is required'], + }, + }, + ], + required: [true, 'Recipients array is required'], + }, + + // Email type for the batch item (uses config enum) + emailType: { + type: String, + enum: Object.values(EMAIL_CONFIG.EMAIL_TYPES), + default: EMAIL_CONFIG.EMAIL_TYPES.BCC, // Use BCC for multiple recipients + required: [true, 'Email type is required'], + }, + + // Status tracking (for the entire batch item) - uses config enum + status: { + type: String, + enum: Object.values(EMAIL_CONFIG.EMAIL_BATCH_STATUSES), + default: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.PENDING, + index: true, + required: [true, 'Status is required'], + }, + + attempts: { + type: Number, + default: 0, + }, + lastAttemptedAt: { + type: Date, + }, + sentAt: { + type: Date, + }, + failedAt: { + type: Date, + }, + + lastError: { + type: String, + }, + lastErrorAt: { + type: Date, + }, + errorCode: { + type: String, + }, + + sendResponse: { + type: Schema.Types.Mixed, + default: null, + }, + + createdAt: { type: Date, default: () => new Date(), index: true }, + updatedAt: { type: Date, default: () => new Date() }, +}); + +// Update timestamps and validate basic constraints +EmailBatchSchema.pre('save', function (next) { + this.updatedAt = new Date(); + + // Validate status consistency with timestamps + if (this.status === EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENT && !this.sentAt) { + this.sentAt = new Date(); + } + if (this.status === EMAIL_CONFIG.EMAIL_BATCH_STATUSES.FAILED && !this.failedAt) { + this.failedAt = new Date(); + } + + next(); +}); + +// Add indexes for better performance +EmailBatchSchema.index({ emailId: 1, status: 1 }); // For batch queries by status +EmailBatchSchema.index({ status: 1, createdAt: 1 }); // For status-based queries +EmailBatchSchema.index({ emailId: 1, createdAt: -1 }); // For batch history +EmailBatchSchema.index({ lastAttemptedAt: 1 }); // For retry logic +EmailBatchSchema.index({ attempts: 1, status: 1 }); // For retry queries +EmailBatchSchema.index({ errorCode: 1 }); // For error queries +EmailBatchSchema.index({ failedAt: -1 }); // For failed batch queries +EmailBatchSchema.index({ status: 1, failedAt: -1 }); // Compound index for failed batches +EmailBatchSchema.index({ emailId: 1, status: 1, createdAt: -1 }); // Compound index for email batches by status + +module.exports = mongoose.model('EmailBatch', EmailBatchSchema, 'emailBatches'); diff --git a/src/models/emailSubcriptionList.js b/src/models/emailSubcriptionList.js index 6fbc0804a..e07bb3a33 100644 --- a/src/models/emailSubcriptionList.js +++ b/src/models/emailSubcriptionList.js @@ -1,14 +1,41 @@ /* eslint-disable quotes */ -const mongoose = require("mongoose"); +const mongoose = require('mongoose'); const { Schema } = mongoose; -const emailSubscriptionSchema = new Schema({ - email: { type: String, required: true, unique: true }, +const emailSubscriptionListSchema = new Schema({ + email: { + type: String, + required: [true, 'Email is required'], + unique: true, + lowercase: true, + trim: true, + index: true, + }, emailSubscriptions: { type: Boolean, default: true, }, + isConfirmed: { + type: Boolean, + default: false, + index: true, + }, + subscribedAt: { + type: Date, + default: () => new Date(), + }, + confirmedAt: { + type: Date, + default: null, + }, }); -module.exports = mongoose.model("emailSubscriptions", emailSubscriptionSchema, "emailSubscriptions"); +// Compound index for common queries (isConfirmed + emailSubscriptions) +emailSubscriptionListSchema.index({ isConfirmed: 1, emailSubscriptions: 1 }); + +module.exports = mongoose.model( + 'emailSubscriptions', + emailSubscriptionListSchema, + 'emailSubscriptions', +); diff --git a/src/models/emailTemplate.js b/src/models/emailTemplate.js new file mode 100644 index 000000000..929febc98 --- /dev/null +++ b/src/models/emailTemplate.js @@ -0,0 +1,100 @@ +/** + * EmailTemplate model for reusable announcement email content. + * - Stores template name, subject, HTML content, and declared variables + * - Tracks creator/updater and timestamps for auditing and sorting + * - Includes helpful indexes and text search for fast lookup + */ +const mongoose = require('mongoose'); +const { EMAIL_CONFIG } = require('../config/emailConfig'); + +const emailTemplateSchema = new mongoose.Schema( + { + name: { + type: String, + required: true, + trim: true, + }, + subject: { + type: String, + required: true, + trim: true, + }, + html_content: { + type: String, + required: true, + }, + variables: [ + { + name: { + type: String, + required: true, + trim: true, + }, + type: { + type: String, + enum: EMAIL_CONFIG.TEMPLATE_VARIABLE_TYPES, + }, + }, + ], + created_by: { + type: mongoose.Schema.Types.ObjectId, + ref: 'userProfile', + }, + updated_by: { + type: mongoose.Schema.Types.ObjectId, + ref: 'userProfile', + }, + }, + { + timestamps: { + createdAt: 'created_at', + updatedAt: 'updated_at', + }, + }, +); + +// Unique index on name (case-insensitive) +emailTemplateSchema.index({ name: 1 }, { unique: true }); + +// Indexes for better search performance +emailTemplateSchema.index({ created_at: -1 }); +emailTemplateSchema.index({ updated_at: -1 }); +emailTemplateSchema.index({ created_by: 1 }); +emailTemplateSchema.index({ updated_by: 1 }); + +// Text index for full-text search +emailTemplateSchema.index({ + name: 'text', + subject: 'text', +}); + +// Compound indexes for common queries +emailTemplateSchema.index({ created_by: 1, created_at: -1 }); +emailTemplateSchema.index({ name: 1, created_at: -1 }); + +// Virtual for camelCase compatibility (for API responses) +emailTemplateSchema.virtual('htmlContent').get(function () { + return this.html_content; +}); + +emailTemplateSchema.virtual('createdBy').get(function () { + return this.created_by; +}); + +emailTemplateSchema.virtual('updatedBy').get(function () { + return this.updated_by; +}); + +emailTemplateSchema.virtual('createdAt').get(function () { + return this.created_at; +}); + +emailTemplateSchema.virtual('updatedAt').get(function () { + return this.updated_at; +}); + +// Ensure virtuals are included in JSON output +emailTemplateSchema.set('toJSON', { virtuals: true }); +emailTemplateSchema.set('toObject', { virtuals: true }); + +module.exports = mongoose.model('EmailTemplate', emailTemplateSchema); diff --git a/src/routes/emailOutboxRouter.js b/src/routes/emailOutboxRouter.js new file mode 100644 index 000000000..4ad1c0964 --- /dev/null +++ b/src/routes/emailOutboxRouter.js @@ -0,0 +1,10 @@ +const express = require('express'); + +const router = express.Router(); + +const emailOutboxController = require('../controllers/emailOutboxController'); + +router.get('/email-outbox', emailOutboxController.getEmails); +router.get('/email-outbox/:emailId', emailOutboxController.getEmailDetails); + +module.exports = router; diff --git a/src/routes/emailRouter.js b/src/routes/emailRouter.js index 8c520009b..e47d25630 100644 --- a/src/routes/emailRouter.js +++ b/src/routes/emailRouter.js @@ -1,29 +1,29 @@ const express = require('express'); const { sendEmail, - sendEmailToAll, + sendEmailToSubscribers, + resendEmail, updateEmailSubscriptions, addNonHgnEmailSubscription, removeNonHgnEmailSubscription, confirmNonHgnEmailSubscription, + retryEmail, + processPendingAndStuckEmails, } = require('../controllers/emailController'); const routes = function () { const emailRouter = express.Router(); - emailRouter.route('/send-emails') - .post(sendEmail); - emailRouter.route('/broadcast-emails') - .post(sendEmailToAll); + emailRouter.route('/send-emails').post(sendEmail); + emailRouter.route('/broadcast-emails').post(sendEmailToSubscribers); + emailRouter.route('/resend-email').post(resendEmail); + emailRouter.route('/retry-email/:emailId').post(retryEmail); + emailRouter.route('/process-pending-and-stuck-emails').post(processPendingAndStuckEmails); - emailRouter.route('/update-email-subscriptions') - .post(updateEmailSubscriptions); - emailRouter.route('/add-non-hgn-email-subscription') - .post(addNonHgnEmailSubscription); - emailRouter.route('/confirm-non-hgn-email-subscription') - .post(confirmNonHgnEmailSubscription); - emailRouter.route('/remove-non-hgn-email-subscription') - .post(removeNonHgnEmailSubscription); + emailRouter.route('/update-email-subscriptions').post(updateEmailSubscriptions); + emailRouter.route('/add-non-hgn-email-subscription').post(addNonHgnEmailSubscription); + emailRouter.route('/confirm-non-hgn-email-subscription').post(confirmNonHgnEmailSubscription); + emailRouter.route('/remove-non-hgn-email-subscription').post(removeNonHgnEmailSubscription); return emailRouter; }; diff --git a/src/routes/emailTemplateRouter.js b/src/routes/emailTemplateRouter.js new file mode 100644 index 000000000..01a0dc674 --- /dev/null +++ b/src/routes/emailTemplateRouter.js @@ -0,0 +1,14 @@ +const express = require('express'); +const emailTemplateController = require('../controllers/emailTemplateController'); + +const router = express.Router(); + +// Email template routes +router.get('/email-templates', emailTemplateController.getAllEmailTemplates); +router.get('/email-templates/:id', emailTemplateController.getEmailTemplateById); +router.post('/email-templates', emailTemplateController.createEmailTemplate); +router.put('/email-templates/:id', emailTemplateController.updateEmailTemplate); +router.delete('/email-templates/:id', emailTemplateController.deleteEmailTemplate); +router.post('/email-templates/:id/preview', emailTemplateController.previewTemplate); + +module.exports = router; diff --git a/src/routes/templateRoutes.js b/src/routes/templateRoutes.js new file mode 100644 index 000000000..0ee712653 --- /dev/null +++ b/src/routes/templateRoutes.js @@ -0,0 +1,24 @@ +/** + * Template Routes - API endpoints for template management + */ + +const express = require('express'); + +const router = express.Router(); +const templateController = require('../controllers/templateController'); + +// Template statistics (must come before /:id routes) +router.get('/stats', templateController.getTemplateStats); + +// Template CRUD operations +router.get('/', templateController.getAllTemplates); +router.get('/:id', templateController.getTemplateById); +router.post('/', templateController.createTemplate); +router.put('/:id', templateController.updateTemplate); +router.delete('/:id', templateController.deleteTemplate); + +// Template rendering and sending +router.post('/:id/render', templateController.renderTemplate); +router.post('/:id/send', templateController.sendTemplateEmail); + +module.exports = router; diff --git a/src/server.js b/src/server.js index c803aa67e..b86ac0ba3 100644 --- a/src/server.js +++ b/src/server.js @@ -1,15 +1,25 @@ /* eslint-disable quotes */ require('dotenv').config(); const http = require('http'); +const mongoose = require('mongoose'); require('./jobs/dailyMessageEmailNotification'); const { app, logger } = require('./app'); const TimerWebsockets = require('./websockets').default; const MessagingWebSocket = require('./websockets/lbMessaging/messagingSocket').default; +const emailProcessor = require('./services/announcements/emails/emailProcessor'); require('./startup/db')(); require('./cronjobs/userProfileJobs')(); require('./cronjobs/pullRequestReviewJobs')(); require('./jobs/analyticsAggregation').scheduleDaily(); require('./cronjobs/bidWinnerJobs')(); + +// Process pending and stuck emails on startup (only after DB is connected) +mongoose.connection.once('connected', () => { + emailProcessor.processPendingAndStuckEmails().catch((error) => { + logger.logException(error, 'Error processing pending emails on startup'); + }); +}); + const websocketRouter = require('./websockets/webSocketRouter'); const port = process.env.PORT || 4500; diff --git a/src/services/announcements/emails/emailBatchService.js b/src/services/announcements/emails/emailBatchService.js new file mode 100644 index 000000000..451fe65cf --- /dev/null +++ b/src/services/announcements/emails/emailBatchService.js @@ -0,0 +1,566 @@ +/** + * Email Batch Service - Manages EmailBatch items (child records) + * Focus: Creating and managing EmailBatch items that reference parent Email records + */ + +const mongoose = require('mongoose'); +const EmailBatch = require('../../../models/emailBatch'); +const Email = require('../../../models/email'); +const { EMAIL_CONFIG } = require('../../../config/emailConfig'); +const { + normalizeRecipientsToObjects, + isValidEmailAddress, +} = require('../../../utilities/emailValidators'); +// const logger = require('../../../startup/logger'); + +class EmailBatchService { + /** + * Create EmailBatch items for a parent Email. + * - Validates parent Email ID, normalizes recipients and chunks by configured size. + * - Returns inserted EmailBatch documents. + * @param {string|ObjectId} emailId - Parent Email ObjectId. + * @param {Array<{email: string}|string>} recipients - Recipients (auto-normalized). + * @param {{batchSize?: number, emailType?: 'TO'|'CC'|'BCC'}} config - Optional overrides. + * @param {import('mongoose').ClientSession|null} session - Optional transaction session. + * @returns {Promise} Created EmailBatch items. + */ + static async createEmailBatches(emailId, recipients, config = {}, session = null) { + // emailId is now the ObjectId directly - validate it + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + const error = new Error(`Email not found with id: ${emailId}`); + error.statusCode = 404; + throw error; + } + + const batchSize = config.batchSize || EMAIL_CONFIG.ANNOUNCEMENTS.BATCH_SIZE; + const emailType = config.emailType || EMAIL_CONFIG.EMAIL_TYPES.BCC; + + // Normalize recipients to { email } + const normalizedRecipients = normalizeRecipientsToObjects(recipients); + if (normalizedRecipients.length === 0) { + const error = new Error('At least one recipient is required'); + error.statusCode = 400; + throw error; + } + + // Validate email format for all recipients FIRST + const invalidRecipients = normalizedRecipients.filter( + (recipient) => !isValidEmailAddress(recipient.email), + ); + if (invalidRecipients.length > 0) { + const error = new Error('One or more recipient emails are invalid'); + error.statusCode = 400; + error.invalidRecipients = invalidRecipients.map((r) => r.email); + throw error; + } + + // Filter to only valid recipients + const validRecipients = normalizedRecipients.filter((recipient) => + isValidEmailAddress(recipient.email), + ); + + // Check if we have any valid recipients AFTER filtering + if (validRecipients.length === 0) { + const error = new Error('No valid recipients after validation'); + error.statusCode = 400; + throw error; + } + + // Validate recipient count limit (only enforce when enforceRecipientLimit is true) + // Default to true to enforce limit for specific recipient requests + const enforceRecipientLimit = config.enforceRecipientLimit !== false; + if ( + enforceRecipientLimit && + validRecipients.length > EMAIL_CONFIG.LIMITS.MAX_RECIPIENTS_PER_REQUEST + ) { + const error = new Error( + `A maximum of ${EMAIL_CONFIG.LIMITS.MAX_RECIPIENTS_PER_REQUEST} recipients are allowed per request`, + ); + error.statusCode = 400; + throw error; + } + + // Chunk recipients into EmailBatch items + const emailBatchItems = []; + + for (let i = 0; i < validRecipients.length; i += batchSize) { + const recipientChunk = validRecipients.slice(i, i + batchSize); + + const emailBatchItem = { + emailId, // emailId is now the ObjectId directly + recipients: recipientChunk.map((recipient) => ({ email: recipient.email })), + emailType, + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.PENDING, + }; + + emailBatchItems.push(emailBatchItem); + } + + // Insert with session if provided for transaction support + let inserted; + try { + inserted = await EmailBatch.insertMany(emailBatchItems, { session }); + } catch (dbError) { + // Handle MongoDB errors + if (dbError.name === 'ValidationError') { + const error = new Error(`Validation error: ${dbError.message}`); + error.statusCode = 400; + throw error; + } + if (dbError.code === 11000) { + const error = new Error('Duplicate key error'); + error.statusCode = 409; + throw error; + } + // Re-throw with status code for other database errors + dbError.statusCode = 500; + throw dbError; + } + + // logger.logInfo( + // `Created ${emailBatchItems.length} EmailBatch items for Email ${emailId} with ${normalizedRecipients.length} total recipients`, + // ); + + return inserted; + } + + /** + * Get Email with its EmailBatch items and essential metadata for UI. + * @param {string|ObjectId} emailId - Parent Email ObjectId. + * @returns {Promise<{email: Object, batches: Array}>} + * @throws {Error} If email not found + */ + static async getEmailWithBatches(emailId) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + const error = new Error('Valid email ID is required'); + error.statusCode = 400; + throw error; + } + + // Get email with createdBy populated using lean for consistency + const email = await Email.findById(emailId) + .populate('createdBy', 'firstName lastName email') + .lean(); + + if (!email) { + const error = new Error(`Email ${emailId} not found`); + error.statusCode = 404; + throw error; + } + + const emailBatches = await this.getBatchesForEmail(emailId); + + // Transform EmailBatch items + const transformedBatches = emailBatches.map((batch) => ({ + _id: batch._id, + emailId: batch.emailId, + recipients: batch.recipients || null, + status: batch.status, + attempts: batch.attempts || null, + lastAttemptedAt: batch.lastAttemptedAt, + sentAt: batch.sentAt, + failedAt: batch.failedAt, + lastError: batch.lastError, + lastErrorAt: batch.lastErrorAt, + errorCode: batch.errorCode, + sendResponse: batch.sendResponse || null, + emailType: batch.emailType, + createdAt: batch.createdAt, + updatedAt: batch.updatedAt, + })); + + return { + email, + batches: transformedBatches, + }; + } + + /** + * Fetch EmailBatch items for a parent Email. + * @param {string|ObjectId} emailId - Parent Email ObjectId. + * @returns {Promise} Sorted ascending by createdAt. + */ + static async getBatchesForEmail(emailId) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + const error = new Error('Valid email ID is required'); + error.statusCode = 400; + throw error; + } + return EmailBatch.find({ emailId }).sort({ createdAt: 1 }); + } + + /** + * Get PENDING EmailBatch items for a parent Email. + * Used by email processor for processing. + * @param {string|ObjectId} emailId - Parent Email ObjectId. + * @returns {Promise} Sorted ascending by createdAt. + */ + static async getPendingBatchesForEmail(emailId) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + const error = new Error('Valid email ID is required'); + error.statusCode = 400; + throw error; + } + return EmailBatch.find({ + emailId, + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.PENDING, + }).sort({ createdAt: 1 }); + } + + /** + * Get EmailBatch by ID. + * @param {string|ObjectId} batchId - EmailBatch ObjectId. + * @returns {Promise} EmailBatch document or null if not found. + */ + static async getBatchById(batchId) { + if (!batchId || !mongoose.Types.ObjectId.isValid(batchId)) { + const error = new Error('Valid email batch ID is required'); + error.statusCode = 400; + throw error; + } + return EmailBatch.findById(batchId); + } + + /** + * Get failed EmailBatch items for a parent Email. + * @param {string|ObjectId} emailId - Parent Email ObjectId. + * @returns {Promise} Array of failed EmailBatch items. + */ + static async getFailedBatchesForEmail(emailId) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + const error = new Error('Valid email ID is required'); + error.statusCode = 400; + throw error; + } + return EmailBatch.find({ + emailId, + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.FAILED, + }).sort({ createdAt: 1 }); + } + + /** + * Get all stuck EmailBatch items (SENDING status). + * On server restart, any batch in SENDING status is considered stuck because + * the processing was interrupted. We reset ALL SENDING batches because the + * server restart means they're no longer being processed. + * @returns {Promise} Array of EmailBatch items with SENDING status that are stuck. + */ + static async getStuckBatches() { + // On server restart: Reset ALL batches in SENDING status (they're all stuck) + return EmailBatch.find({ + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENDING, + }).sort({ lastAttemptedAt: 1 }); // Process oldest first + } + + /** + * Reset an EmailBatch item for retry, clearing attempts and error fields. + * Uses atomic update to prevent race conditions. + * @param {string|ObjectId} emailBatchId - Batch ObjectId. + * @returns {Promise} Updated document. + * @throws {Error} If batch not found + */ + static async resetEmailBatchForRetry(emailBatchId) { + if (!emailBatchId || !mongoose.Types.ObjectId.isValid(emailBatchId)) { + const error = new Error('Valid email batch ID is required'); + error.statusCode = 400; + throw error; + } + + const now = new Date(); + const updated = await EmailBatch.findByIdAndUpdate( + emailBatchId, + { + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.PENDING, + attempts: 0, + lastError: null, + lastErrorAt: null, + errorCode: null, + failedAt: null, + lastAttemptedAt: null, + updatedAt: now, + }, + { new: true }, + ); + + if (!updated) { + const error = new Error(`EmailBatch ${emailBatchId} not found`); + error.statusCode = 404; + throw error; + } + + return updated; + } + + /** + * Mark a batch item as SENDING, increment attempts, and set lastAttemptedAt. + * Uses atomic update with condition to prevent race conditions. + * @param {string|ObjectId} emailBatchId - Batch ObjectId. + * @returns {Promise} Updated batch document. + * @throws {Error} If batch not found or not in PENDING status + */ + static async markEmailBatchSending(emailBatchId) { + const now = new Date(); + const updated = await EmailBatch.findOneAndUpdate( + { + _id: emailBatchId, + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.PENDING, + }, + { + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENDING, + $inc: { attempts: 1 }, + lastAttemptedAt: now, + }, + { new: true }, + ); + + if (!updated) { + const error = new Error(`EmailBatch ${emailBatchId} not found or not in PENDING status`); + error.statusCode = 404; + throw error; + } + + return updated; + } + + /** + * Mark a batch item as SENT and set sentAt timestamp. + * Uses atomic update with status check to prevent race conditions. + * @param {string|ObjectId} emailBatchId - Batch ObjectId. + * @param {{attemptCount?: number, sendResponse?: Object}} options - Optional attempt count and send response to store. + * @returns {Promise} Updated batch document. + * @throws {Error} If batch not found or not in SENDING status + */ + static async markEmailBatchSent(emailBatchId, options = {}) { + if (!emailBatchId || !mongoose.Types.ObjectId.isValid(emailBatchId)) { + const error = new Error('Valid email batch ID is required'); + error.statusCode = 400; + throw error; + } + + const now = new Date(); + const updateFields = { + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENT, + sentAt: now, + updatedAt: now, + }; + + // Update attempts count if provided (to reflect actual retry attempts) + if (options.attemptCount && options.attemptCount > 0) { + updateFields.attempts = options.attemptCount; + } + + // Store send response if provided (contains messageId, accepted, rejected, etc.) + if (options.sendResponse) { + updateFields.sendResponse = options.sendResponse; + } + + // Atomic update: only update if batch is in SENDING status (prevents race conditions) + const updated = await EmailBatch.findOneAndUpdate( + { + _id: emailBatchId, + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENDING, + }, + updateFields, + { new: true }, + ); + + if (!updated) { + // Batch not found or not in SENDING status (might already be SENT/FAILED) + // Check current status to provide better error message + const currentBatch = await EmailBatch.findById(emailBatchId); + if (!currentBatch) { + const error = new Error(`EmailBatch ${emailBatchId} not found`); + error.statusCode = 404; + throw error; + } + // Batch exists but not in SENDING status - log and return current batch (idempotent) + // logger.logInfo( + // `EmailBatch ${emailBatchId} is not in SENDING status (current: ${currentBatch.status}), skipping mark as SENT`, + // ); + return currentBatch; + } + + return updated; + } + + /** + * Mark a batch item as FAILED and snapshot the error info. + * Uses atomic update with status check to prevent race conditions. + * @param {string|ObjectId} emailBatchId - Batch ObjectId. + * @param {{errorCode?: string, errorMessage?: string, attemptCount?: number}} param1 - Error details and attempt count. + * @returns {Promise} Updated batch document. + * @throws {Error} If batch not found + */ + static async markEmailBatchFailed(emailBatchId, { errorCode, errorMessage, attemptCount }) { + if (!emailBatchId || !mongoose.Types.ObjectId.isValid(emailBatchId)) { + const error = new Error('Valid email batch ID is required'); + error.statusCode = 400; + throw error; + } + + const now = new Date(); + const updateFields = { + status: EMAIL_CONFIG.EMAIL_BATCH_STATUSES.FAILED, + failedAt: now, + lastError: errorMessage?.slice(0, 500) || null, + lastErrorAt: now, + errorCode: errorCode?.toString().slice(0, 1000) || null, + updatedAt: now, + }; + + // Update attempts count if provided (to reflect actual retry attempts) + if (attemptCount && attemptCount > 0) { + updateFields.attempts = attemptCount; + } + + // Atomic update: only update if batch is in SENDING status (prevents race conditions) + // Allow updating from PENDING as well (in case of early failures) + const updated = await EmailBatch.findOneAndUpdate( + { + _id: emailBatchId, + status: { + $in: [ + EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENDING, + EMAIL_CONFIG.EMAIL_BATCH_STATUSES.PENDING, + ], + }, + }, + updateFields, + { new: true }, + ); + + if (!updated) { + // Batch not found or already in final state (SENT/FAILED) + // Check current status to provide better error message + const currentBatch = await EmailBatch.findById(emailBatchId); + if (!currentBatch) { + const error = new Error(`EmailBatch ${emailBatchId} not found`); + error.statusCode = 404; + throw error; + } + // Batch exists but already in final state - log and return current batch (idempotent) + // logger.logInfo( + // `EmailBatch ${emailBatchId} is already in final state (current: ${currentBatch.status}), skipping mark as FAILED`, + // ); + return currentBatch; + } + + return updated; + } + + /** + * Determine the parent Email status from child EmailBatch statuses. + * Rules: + * - All SENT => SENT (all batches succeeded) + * - All FAILED => FAILED (all batches failed) + * - Mixed (some SENT and some FAILED, but ALL batches completed) => PROCESSED + * - Otherwise (PENDING or SENDING batches) => SENDING (still in progress) + * @param {string|ObjectId} emailId - Parent Email ObjectId. + * @returns {Promise} Derived status constant from EMAIL_CONFIG.EMAIL_STATUSES. + */ + static async determineEmailStatus(emailId) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + const error = new Error('Valid email ID is required'); + error.statusCode = 400; + throw error; + } + + // Get all batches and count statuses in memory (simpler and faster for small-medium counts) + const batches = await this.getBatchesForEmail(emailId); + + // Handle edge case: no batches + if (batches.length === 0) { + // logger.logInfo(`Email ${emailId} has no batches, returning FAILED status`); + return EMAIL_CONFIG.EMAIL_STATUSES.FAILED; + } + + const statusMap = batches.reduce((acc, batch) => { + acc[batch.status] = (acc[batch.status] || 0) + 1; + return acc; + }, {}); + + const pending = statusMap[EMAIL_CONFIG.EMAIL_BATCH_STATUSES.PENDING] || 0; + const sending = statusMap[EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENDING] || 0; + const sent = statusMap[EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENT] || 0; + const failed = statusMap[EMAIL_CONFIG.EMAIL_BATCH_STATUSES.FAILED] || 0; + + // Still processing (pending or sending batches) = keep SENDING status + // This check must come FIRST to avoid returning final states when still processing + if (pending > 0 || sending > 0) { + return EMAIL_CONFIG.EMAIL_STATUSES.SENDING; + } + + // All batches completed - determine final status + // All sent = SENT (all batches succeeded) + if (sent > 0 && failed === 0) { + return EMAIL_CONFIG.EMAIL_STATUSES.SENT; + } + + // All failed = FAILED (all batches failed) + if (failed > 0 && sent === 0) { + return EMAIL_CONFIG.EMAIL_STATUSES.FAILED; + } + + // Mixed results (some sent, some failed) = PROCESSED + // All batches have completed (no pending or sending), but results are mixed + if (sent > 0 && failed > 0) { + return EMAIL_CONFIG.EMAIL_STATUSES.PROCESSED; + } + + // Fallback: Should not reach here, but keep SENDING status + return EMAIL_CONFIG.EMAIL_STATUSES.SENDING; + } + + /** + * Synchronize parent Email status based on child EmailBatch statuses. + * - Determines status from batches and updates parent Email + * - Sets completedAt timestamp only for final states (SENT, FAILED, PROCESSED) + * - This ensures Email.status stays in sync when batches are updated + * - Uses atomic update to prevent race conditions + * @param {string|ObjectId} emailId - Parent Email ObjectId. + * @returns {Promise} Updated Email document. + */ + static async syncParentEmailStatus(emailId) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + const error = new Error('Valid email ID is required'); + error.statusCode = 400; + throw error; + } + + // Determine the correct status from batches + const derivedStatus = await this.determineEmailStatus(emailId); + + // Check if this is a final state that should set completedAt + const finalStates = [ + EMAIL_CONFIG.EMAIL_STATUSES.SENT, + EMAIL_CONFIG.EMAIL_STATUSES.FAILED, + EMAIL_CONFIG.EMAIL_STATUSES.PROCESSED, + ]; + const isFinalState = finalStates.includes(derivedStatus); + + const now = new Date(); + const updateFields = { + status: derivedStatus, + updatedAt: now, + }; + + // Only set completedAt for final states + if (isFinalState) { + updateFields.completedAt = now; + } + + // Update Email status atomically + // Note: We don't check current status because we're recomputing from batches (source of truth) + const email = await Email.findByIdAndUpdate(emailId, updateFields, { new: true }); + + if (!email) { + // Email not found - log but don't throw (might have been deleted) + // logger.logInfo(`Email ${emailId} not found when syncing status`); + return null; + } + + return email; + } +} + +module.exports = EmailBatchService; diff --git a/src/services/announcements/emails/emailProcessor.js b/src/services/announcements/emails/emailProcessor.js new file mode 100644 index 000000000..30691b5c6 --- /dev/null +++ b/src/services/announcements/emails/emailProcessor.js @@ -0,0 +1,600 @@ +const mongoose = require('mongoose'); +const { EMAIL_CONFIG } = require('../../../config/emailConfig'); +const EmailService = require('./emailService'); +const EmailBatchService = require('./emailBatchService'); +const emailSendingService = require('./emailSendingService'); +// const logger = require('../../../startup/logger'); + +class EmailProcessor { + /** + * Initialize processor runtime configuration. + * - Tracks currently processing parent Email IDs to avoid duplicate work. + * - Loads retry settings from EMAIL_CONFIG to coordinate with sending service. + * - Maintains in-memory queue for sequential email processing. + */ + constructor() { + this.processingBatches = new Set(); + this.maxRetries = EMAIL_CONFIG.DEFAULT_MAX_RETRIES; + this.retryDelay = EMAIL_CONFIG.INITIAL_RETRY_DELAY_MS; + this.emailQueue = []; // In-memory queue for emails to process + this.isProcessingQueue = false; // Flag to prevent multiple queue processors + this.currentlyProcessingEmailId = null; // Track which email is currently being processed + this.maxQueueSize = EMAIL_CONFIG.ANNOUNCEMENTS.MAX_QUEUE_SIZE || 100; // Max queue size to prevent memory leak + } + + /** + * Add an email to the processing queue. + * - Adds email to in-memory queue if not already queued or processing + * - Starts queue processor if not already running and DB is connected + * - Returns immediately (non-blocking) + * - Uses setImmediate pattern for asynchronous queue processing + * @param {string|ObjectId} emailId - The ObjectId of the parent Email. + * @returns {void} + */ + queueEmail(emailId) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + // logger.logException(new Error('Invalid emailId'), 'EmailProcessor.queueEmail'); + return false; // Return false to indicate failure + } + + const emailIdStr = emailId.toString(); + + // Atomic check and add: Skip if already in queue or currently processing + // Use includes check first for early return + if ( + this.emailQueue.includes(emailIdStr) || + this.currentlyProcessingEmailId === emailIdStr || + this.processingBatches.has(emailIdStr) + ) { + // logger.logInfo(`Email ${emailIdStr} is already queued or being processed, skipping`); + return true; // Already queued, consider this success + } + + // Check queue size to prevent memory leak - REJECT instead of dropping old emails + if (this.emailQueue.length > this.maxQueueSize) { + // logger.logException( + // new Error(`Email queue is full (${this.maxQueueSize}). Rejecting new email ${emailIdStr}.`), + // 'EmailProcessor.queueEmail - Queue overflow', + // ); + return false; // Return false to signal queue is full + } + + // Add to queue (atomic operation - push is atomic in single-threaded JS) + this.emailQueue.push(emailIdStr); + // logger.logInfo(`Email ${emailIdStr} added to queue. Queue length: ${this.emailQueue.length}`); + + // Start queue processor if not already running + // Check flag and start processor synchronously to prevent race condition + if (!this.isProcessingQueue) { + setImmediate(() => { + // eslint-disable-next-line no-unused-vars + this.processQueue().catch((error) => { + // logger.logException(error, 'Error in queue processor'); + // Reset flag so queue can restart on next email addition + this.isProcessingQueue = false; + }); + }); + } + + return true; // Successfully queued + } + + /** + * Process the email queue sequentially. + * - Processes one email at a time + * - Once an email is done, processes the next one + * - Continues until queue is empty + * - Database connection is ensured at startup (server.js waits for DB connection) + * - Uses atomic check-and-set pattern to prevent race conditions + * @returns {Promise} + */ + async processQueue() { + // Atomic check-and-set using synchronous operations + // This must be done synchronously before ANY async operations to prevent race conditions + // In Node.js event loop, synchronous code is atomic + if (this.isProcessingQueue) { + return; // Already processing + } + + // Set flag IMMEDIATELY and synchronously to prevent race conditions + this.isProcessingQueue = true; + // logger.logInfo('Email queue processor started'); + + try { + // eslint-disable-next-line no-constant-condition + while (true) { + // Get next email from queue (FIFO) + const emailId = this.emailQueue.shift(); + if (!emailId) { + break; + } + + this.currentlyProcessingEmailId = emailId; + // logger.logInfo( + // `Processing email ${emailId} from queue. Remaining: ${this.emailQueue.length}`, + // ); + + try { + // Process the email (this processes all its batches) + // Sequential processing is required - await in loop is necessary + // eslint-disable-next-line no-await-in-loop + await this.processEmail(emailId); + // logger.logInfo(`Email ${emailId} processing completed`); + } catch (error) { + // logger.logException(error, `Error processing email ${emailId} from queue`); + } finally { + this.currentlyProcessingEmailId = null; + } + + // Small delay before processing next email to avoid overwhelming the system + if (this.emailQueue.length > 0) { + // eslint-disable-next-line no-await-in-loop + await EmailProcessor.sleep(100); + } + } + } finally { + this.isProcessingQueue = false; + // logger.logInfo('Email queue processor stopped'); + } + } + + /** + * Process a single parent Email by sending all of its pending EmailBatch items. + * - Processes all batches for the email sequentially (with concurrency within batches) + * - Idempotent with respect to concurrent calls (skips if already processing) + * - Simple flow: PENDING → SENDING → SENT/FAILED/PROCESSED + * @param {string|ObjectId} emailId - The ObjectId of the parent Email. + * @returns {Promise} Final email status: SENT | FAILED | PROCESSED | SENDING + * @throws {Error} When emailId is invalid or the Email is not found. + */ + async processEmail(emailId) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + throw new Error('emailId is required and must be a valid ObjectId'); + } + + const emailIdStr = emailId.toString(); + + // Atomic check-and-add to prevent race condition: prevent concurrent processing of the same email + // Check first, then add atomically (in single-threaded JS, this is safe between async operations) + if (this.processingBatches.has(emailIdStr)) { + // logger.logInfo(`Email ${emailIdStr} is already being processed, skipping`); + return EMAIL_CONFIG.EMAIL_STATUSES.SENDING; + } + + // Add to processing set BEFORE any async operations to prevent race conditions + // This ensures no other processEmail call can start while this one is initializing + this.processingBatches.add(emailIdStr); + + try { + // Get email - don't throw if not found, handle gracefully in processor + const email = await EmailService.getEmailById(emailId); + if (!email) { + throw new Error(`Email not found with id: ${emailId}`); + } + + // Skip if already in final state + if ( + email.status === EMAIL_CONFIG.EMAIL_STATUSES.SENT || + email.status === EMAIL_CONFIG.EMAIL_STATUSES.FAILED || + email.status === EMAIL_CONFIG.EMAIL_STATUSES.PROCESSED + ) { + // logger.logInfo(`Email ${emailId} is already in final state: ${email.status}`); + return email.status; + } + + // Mark email as SENDING (atomic update from PENDING to SENDING) + // If already SENDING, this will fail and we'll skip processing + try { + await EmailService.markEmailStarted(emailId); + } catch (startError) { + // If marking as started fails, email is likely already being processed + const currentEmail = await EmailService.getEmailById(emailId); + if (currentEmail && currentEmail.status === EMAIL_CONFIG.EMAIL_STATUSES.SENDING) { + // logger.logInfo(`Email ${emailIdStr} is already being processed, skipping`); + this.processingBatches.delete(emailIdStr); + return EMAIL_CONFIG.EMAIL_STATUSES.SENDING; + } + // Re-throw if it's a different error + throw startError; + } + + // Process all PENDING EmailBatch items for this email + await this.processEmailBatches(email); + + // Sync parent Email status based on all batch statuses + // Auto-sync from individual batch updates may have already updated status, + // but this ensures final status and completedAt timestamp are set correctly + const updatedEmail = await EmailBatchService.syncParentEmailStatus(email._id); + const finalStatus = updatedEmail ? updatedEmail.status : EMAIL_CONFIG.EMAIL_STATUSES.FAILED; + + // logger.logInfo(`Email ${emailIdStr} processed with status: ${finalStatus}`); + return finalStatus; + } catch (error) { + // logger.logException(error, `Error processing Email ${emailIdStr}`); + + // Reset any batches that were marked as SENDING back to PENDING + // This prevents batches from being stuck in SENDING status + try { + const batches = await EmailBatchService.getBatchesForEmail(emailIdStr); + const sendingBatches = batches.filter( + (batch) => batch.status === EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENDING, + ); + await Promise.allSettled( + sendingBatches.map(async (batch) => { + try { + await EmailBatchService.resetEmailBatchForRetry(batch._id); + // logger.logInfo( + // `Reset batch ${batch._id} from SENDING to PENDING due to email processing error`, + // ); + } catch (resetError) { + // logger.logException(resetError, `Error resetting batch ${batch._id} to PENDING`); + } + }), + ); + } catch (resetError) { + // logger.logException(resetError, `Error resetting batches for email ${emailIdStr}`); + } + + // Sync parent Email status based on actual batch states (not just mark as FAILED) + // This ensures status accurately reflects batches that may have succeeded before the error + try { + const updatedEmail = await EmailBatchService.syncParentEmailStatus(emailIdStr); + const finalStatus = updatedEmail ? updatedEmail.status : EMAIL_CONFIG.EMAIL_STATUSES.FAILED; + return finalStatus; + } catch (updateError) { + // If sync fails, fall back to marking as FAILED + // logger.logException(updateError, 'Error syncing Email status after error'); + try { + await EmailService.markEmailCompleted(emailIdStr, EMAIL_CONFIG.EMAIL_STATUSES.FAILED); + } catch (markError) { + // logger.logException(markError, 'Error updating Email status to failed'); + } + return EMAIL_CONFIG.EMAIL_STATUSES.FAILED; + } + } finally { + this.processingBatches.delete(emailIdStr); + } + } + + /** + * Process all PENDING EmailBatch items for a given parent Email. + * - Only processes PENDING batches (simple and straightforward) + * - Sends batches with limited concurrency + * - Each request is independent - processes only batches for this email + * @param {Object} email - The parent Email mongoose document. + * @returns {Promise} + */ + async processEmailBatches(email) { + // Get only PENDING batches for this email (service validates emailId) + const pendingBatches = await EmailBatchService.getPendingBatchesForEmail(email._id); + + if (pendingBatches.length === 0) { + // logger.logInfo(`No PENDING EmailBatch items found for Email ${email._id}`); + return; + } + + // logger.logInfo( + // `Processing ${pendingBatches.length} PENDING EmailBatch items for Email ${email._id}`, + // ); + + // Process items with concurrency limit + const concurrency = EMAIL_CONFIG.ANNOUNCEMENTS.CONCURRENCY || 3; + const delayBetweenChunks = EMAIL_CONFIG.ANNOUNCEMENTS.DELAY_BETWEEN_CHUNKS_MS || 1000; + const batchStaggerStart = EMAIL_CONFIG.ANNOUNCEMENTS.BATCH_STAGGER_START_MS || 0; + const results = []; + + // Process batches in chunks with concurrency control + // eslint-disable-next-line no-await-in-loop + for (let i = 0; i < pendingBatches.length; i += concurrency) { + const batchChunk = pendingBatches.slice(i, i + concurrency); + + // Process batches with optional staggered start delays within the chunk + // This staggers when each batch in the chunk starts processing (helps with rate limiting) + const batchPromises = batchChunk.map((item, index) => { + if (batchStaggerStart > 0 && index > 0) { + // Stagger the start: batch 1 starts immediately, batch 2 after staggerDelay, batch 3 after 2*staggerDelay, etc. + return EmailProcessor.sleep(batchStaggerStart * index).then(() => + this.processEmailBatch(item, email), + ); + } + // First batch in chunk starts immediately (no stagger) + return this.processEmailBatch(item, email); + }); + + // Wait for all batches in this chunk to complete + // eslint-disable-next-line no-await-in-loop + const batchResults = await Promise.allSettled(batchPromises); + results.push(...batchResults); + + // Add delay after this chunk completes before starting the next chunk + // This provides consistent pacing to prevent hitting Gmail rate limits + if (delayBetweenChunks > 0 && i + concurrency < pendingBatches.length) { + // eslint-disable-next-line no-await-in-loop + await EmailProcessor.sleep(delayBetweenChunks); + } + } + + // Log summary of processing + // const succeeded = results.filter((r) => r.status === 'fulfilled').length; + // const failed = results.filter((r) => r.status === 'rejected').length; + + // logger.logInfo( + // `Completed processing ${pendingBatches.length} EmailBatch items for Email ${email._id}: ${succeeded} succeeded, ${failed} failed`, + // ); + } + + /** + * Send one EmailBatch item (one SMTP send for a group of recipients). + * - Marks batch as SENDING (atomic update from PENDING) + * - Sends email with retry logic + * - Marks as SENT on success or FAILED on failure + * @param {Object} item - The EmailBatch mongoose document (should be PENDING). + * @param {Object} email - The parent Email mongoose document. + * @returns {Promise} + * @throws {Error} Bubbles final failure so callers can classify in allSettled results. + */ + async processEmailBatch(item, email) { + if (!item || !item._id) { + throw new Error('Invalid EmailBatch item'); + } + if (!email || !email._id) { + throw new Error('Invalid Email parent'); + } + + const recipientEmails = (item.recipients || []) + .map((r) => r?.email) + .filter((e) => e && typeof e === 'string'); + + if (recipientEmails.length === 0) { + // logger.logException( + // new Error('No valid recipients found'), + // `EmailBatch item ${item._id} has no valid recipients`, + // ); + await EmailBatchService.markEmailBatchFailed(item._id, { + errorCode: 'NO_RECIPIENTS', + errorMessage: 'No valid recipients found', + }); + return; + } + + // Mark as SENDING (atomic update from PENDING to SENDING) + // If this fails, batch was already processed by another thread - skip it + let updatedItem; + try { + updatedItem = await EmailBatchService.markEmailBatchSending(item._id); + } catch (markError) { + // Batch was likely already processed - check status and skip if so + // Use service method for consistency (service validates batchId) + try { + const currentBatch = await EmailBatchService.getBatchById(item._id); + if (currentBatch) { + if ( + currentBatch.status === EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENT || + currentBatch.status === EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENDING + ) { + // logger.logInfo( + // `EmailBatch ${item._id} is already ${currentBatch.status}, skipping duplicate processing`, + // ); + return; // Skip this batch + } + } + } catch (batchError) { + // If batch not found or invalid ID, log and re-throw original error + // logger.logException(batchError, `Error checking EmailBatch ${item._id} status`); + } + // Re-throw if it's a different error + throw markError; + } + + // Build mail options with sender name + const senderName = EMAIL_CONFIG.EMAIL.SENDER_NAME; + const senderEmail = EMAIL_CONFIG.EMAIL.SENDER; + const fromField = senderName ? `${senderName} <${senderEmail}>` : senderEmail; + + const mailOptions = { + from: fromField, + subject: email.subject, + html: email.htmlContent, + }; + + if (item.emailType === EMAIL_CONFIG.EMAIL_TYPES.BCC) { + mailOptions.to = EMAIL_CONFIG.EMAIL.SENDER; + mailOptions.bcc = recipientEmails.join(','); + } else { + mailOptions.to = recipientEmails.join(','); + } + + // Delegate retry/backoff to the sending service + const sendResult = await emailSendingService.sendWithRetry( + mailOptions, + this.maxRetries, + this.retryDelay, + ); + + if (sendResult.success) { + const actualAttemptCount = sendResult.attemptCount || updatedItem?.attempts || 1; + await EmailBatchService.markEmailBatchSent(item._id, { + attemptCount: actualAttemptCount, // Persist the actual number of attempts made + sendResponse: sendResult.response, // Store the full response from email API + }); + // logger.logInfo( + // `EmailBatch item ${item._id} sent successfully to ${recipientEmails.length} recipients (attempts ${actualAttemptCount})`, + // ); + return; + } + + // Final failure after retries + const finalError = sendResult.error || new Error('Failed to send email'); + // Extract error code, or use error name if code is missing, or default to 'SEND_FAILED' + const errorCode = finalError.code || finalError.name || 'SEND_FAILED'; + const errorMessage = finalError.message || 'Failed to send email'; + const actualAttemptCount = sendResult.attemptCount || 1; // Use actual attempt count from retry logic + + await EmailBatchService.markEmailBatchFailed(item._id, { + errorCode, + errorMessage, + attemptCount: actualAttemptCount, // Persist the actual number of attempts made + }); + + // logger.logInfo( + // `Permanently failed to send EmailBatch item ${item._id} to ${recipientEmails.length} recipients after ${actualAttemptCount} attempts`, + // ); + // Throw to ensure Promise.allSettled records this item as failed + throw finalError; + } + + /** + * Sleep utility to await a given duration. + * @param {number} ms - Milliseconds to wait. + * @returns {Promise} + */ + static sleep(ms) { + return new Promise((resolve) => { + setTimeout(resolve, ms); + }); + } + + /** + * Get lightweight processor status for diagnostics/telemetry. + * @returns {{isRunning: boolean, processingBatches: string[], maxRetries: number, queueLength: number, currentlyProcessing: string|null, isProcessingQueue: boolean}} + */ + getStatus() { + return { + isRunning: true, + processingBatches: Array.from(this.processingBatches), + maxRetries: this.maxRetries, + queueLength: this.emailQueue.length, + currentlyProcessing: this.currentlyProcessingEmailId, + isProcessingQueue: this.isProcessingQueue, + }; + } + + /** + * Reset batches that are stuck in SENDING status during runtime. + * - Identifies batches that have been in SENDING status for more than 20 minutes + * - Resets them to PENDING so they can be retried + * - Called by cron job to handle runtime failures (not just startup) + * @returns {Promise} Number of batches reset + */ + static async resetStuckRuntimeBatches() { + try { + const STUCK_TIMEOUT_MS = 20 * 60 * 1000; // 20 minutes + const timeoutThreshold = new Date(Date.now() - STUCK_TIMEOUT_MS); + + // Find batches stuck in SENDING status for more than 15 minutes + const EmailBatch = require('../../../models/emailBatch'); + const emailConfig = require('../../../config/emailConfig').EMAIL_CONFIG; + + const stuckBatches = await EmailBatch.find({ + status: emailConfig.EMAIL_BATCH_STATUSES.SENDING, + lastAttemptedAt: { $lt: timeoutThreshold }, + }); + + if (stuckBatches.length === 0) { + return 0; + } + + // logger.logInfo( + // `Found ${stuckBatches.length} batches stuck in SENDING status for >15 minutes, resetting...`, + // ); + + let resetCount = 0; + await Promise.allSettled( + stuckBatches.map(async (batch) => { + try { + await EmailBatchService.resetEmailBatchForRetry(batch._id); + resetCount += 1; + // logger.logInfo(`Reset runtime stuck batch ${batch._id} to PENDING`); + } catch (error) { + // logger.logException(error, `Error resetting runtime stuck batch ${batch._id}`); + } + }), + ); + + return resetCount; + } catch (error) { + // logger.logException(error, 'Error in resetStuckRuntimeBatches'); + return 0; + } + } + + /** + * Process pending and stuck emails on system startup OR via cron job. + * - Called after database connection is established (server.js) + * - Called periodically by cron job (every 10 minutes) + * - Resets stuck emails (SENDING status) to PENDING + * - Resets stuck batches (SENDING status) to PENDING + * - Resets runtime stuck batches (SENDING > 20 minutes) + * - Queues all PENDING emails for processing + * @returns {Promise} + */ + async processPendingAndStuckEmails() { + // logger.logInfo('Starting processing of pending and stuck emails...'); + + // Step 1: Reset stuck emails to PENDING + const stuckEmails = await EmailService.getStuckEmails(); + if (stuckEmails.length > 0) { + // logger.logInfo(`Found ${stuckEmails.length} stuck emails, resetting to PENDING...`); + await Promise.allSettled( + stuckEmails.map(async (email) => { + try { + await EmailService.resetStuckEmail(email._id); + // logger.logInfo(`Reset stuck email ${email._id} to PENDING`); + } catch (error) { + // logger.logException(error, `Error resetting stuck email ${email._id}`); + } + }), + ); + } + + // Step 2: Reset stuck batches to PENDING (from server restart) + const stuckBatches = await EmailBatchService.getStuckBatches(); + if (stuckBatches.length > 0) { + // logger.logInfo(`Found ${stuckBatches.length} stuck batches, resetting to PENDING...`); + await Promise.allSettled( + stuckBatches.map(async (batch) => { + try { + await EmailBatchService.resetEmailBatchForRetry(batch._id); + // logger.logInfo(`Reset stuck batch ${batch._id} to PENDING`); + } catch (error) { + // logger.logException(error, `Error resetting stuck batch ${batch._id}`); + } + }), + ); + } + + // Step 3: Reset runtime stuck batches (SENDING > 20 minutes) + const runtimeResetCount = await EmailProcessor.resetStuckRuntimeBatches(); + if (runtimeResetCount > 0) { + // logger.logInfo(`Reset ${runtimeResetCount} runtime stuck batches`); + } + + // Step 4: Queue all PENDING emails for processing + const pendingEmails = await EmailService.getPendingEmails(); + if (pendingEmails.length > 0) { + // logger.logInfo(`Found ${pendingEmails.length} pending emails, adding to queue...`); + // Queue all emails (non-blocking, sequential processing) + pendingEmails.forEach((email) => { + this.queueEmail(email._id); + }); + // logger.logInfo(`Queued ${pendingEmails.length} pending emails for processing`); + } else { + // logger.logInfo('No pending emails found'); + } + + // logger.logInfo('Processing of pending and stuck emails completed'); + + // Return statistics for user feedback + return { + stuckEmailsReset: stuckEmails.length, + stuckBatchesReset: stuckBatches.length, + runtimeStuckBatchesReset: runtimeResetCount, + pendingEmailsQueued: pendingEmails.length, + }; + } +} + +// Create singleton instance +const emailProcessor = new EmailProcessor(); + +module.exports = emailProcessor; diff --git a/src/services/announcements/emails/emailSendingService.js b/src/services/announcements/emails/emailSendingService.js new file mode 100644 index 000000000..6cba06b39 --- /dev/null +++ b/src/services/announcements/emails/emailSendingService.js @@ -0,0 +1,266 @@ +/** + * Email Sending Service + * Handles sending emails via Gmail API using OAuth2 authentication + * Provides validation, retry logic, and comprehensive error handling + */ + +const nodemailer = require('nodemailer'); +const { google } = require('googleapis'); +// const logger = require('../../../startup/logger'); + +class EmailSendingService { + /** + * Initialize Gmail OAuth2 transport configuration and validate required env vars. + * Uses lazy initialization - only initializes when first used (not at module load). + * Throws during initialization if configuration is incomplete to fail fast. + */ + constructor() { + this._initialized = false; + this.config = null; + this.OAuth2Client = null; + this.transporter = null; + } + + /** + * Initialize the service if not already initialized. + * Lazy initialization allows tests to run without email config. + * @private + */ + _initialize() { + if (this._initialized) { + return; + } + + this.config = { + email: process.env.ANNOUNCEMENT_EMAIL, + clientId: process.env.ANNOUNCEMENT_EMAIL_CLIENT_ID, + clientSecret: process.env.ANNOUNCEMENT_EMAIL_CLIENT_SECRET, + redirectUri: process.env.ANNOUNCEMENT_EMAIL_CLIENT_REDIRECT_URI, + refreshToken: process.env.ANNOUNCEMENT_EMAIL_REFRESH_TOKEN, + }; + + // Validate configuration + const required = ['email', 'clientId', 'clientSecret', 'refreshToken', 'redirectUri']; + const missing = required.filter((k) => !this.config[k]); + if (missing.length) { + throw new Error(`Email config incomplete. Missing: ${missing.join(', ')}`); + } + + this.OAuth2Client = new google.auth.OAuth2( + this.config.clientId, + this.config.clientSecret, + this.config.redirectUri, + ); + this.OAuth2Client.setCredentials({ refresh_token: this.config.refreshToken }); + + // Create the email transporter + this.transporter = nodemailer.createTransport({ + service: 'gmail', + auth: { + type: 'OAuth2', + user: this.config.email, + clientId: this.config.clientId, + clientSecret: this.config.clientSecret, + }, + }); + + this._initialized = true; + } + + /** + * Get OAuth access token (refreshes on each call). + * Similar to emailSender.js pattern - refreshes token for each send to avoid stale tokens. + * @returns {Promise} Access token + * @throws {Error} If token refresh fails + */ + async getAccessToken() { + this._initialize(); + const accessTokenResp = await this.OAuth2Client.getAccessToken(); + let token; + + if (accessTokenResp && typeof accessTokenResp === 'object' && accessTokenResp.token) { + token = accessTokenResp.token; + } else if (typeof accessTokenResp === 'string') { + token = accessTokenResp; + } else { + throw new Error('Invalid access token response format'); + } + + if (!token) { + throw new Error('NO_OAUTH_ACCESS_TOKEN: Failed to obtain access token'); + } + + return token; + } + + /** + * Send email with enhanced announcement tracking. + * - Validates recipients, subject, and service configuration. + * - Fetches OAuth2 access token and attaches OAuth credentials to the request. + * - Returns a structured result instead of throwing to simplify callers. + * @param {Object} mailOptions - Nodemailer-compatible options (to|bcc, subject, html, from?). + * @returns {Promise<{success: boolean, response?: Object, error?: Error}>} + */ + async sendEmail(mailOptions) { + this._initialize(); + // Validation + if (!mailOptions) { + const error = new Error('INVALID_MAIL_OPTIONS: mailOptions is required'); + // logger.logException(error, 'EmailSendingService.sendEmail validation failed'); + return { success: false, error }; + } + + if (!mailOptions.to && !mailOptions.bcc) { + const error = new Error('INVALID_RECIPIENTS: At least one recipient (to or bcc) is required'); + // logger.logException(error, 'EmailSendingService.sendEmail validation failed'); + return { success: false, error }; + } + + // Validate subject and htmlContent + if (!mailOptions.subject || mailOptions.subject.trim() === '') { + const error = new Error('INVALID_SUBJECT: Subject is required and cannot be empty'); + // logger.logException(error, 'EmailSendingService.sendEmail validation failed'); + return { success: false, error }; + } + + if (!this.config.email || !this.config.clientId || !this.config.clientSecret) { + const error = new Error('INVALID_CONFIG: Email configuration is incomplete'); + // logger.logException(error, 'EmailSendingService.sendEmail configuration check failed'); + return { success: false, error }; + } + + try { + // Get access token (refreshes on each send to avoid stale tokens) + let token; + try { + token = await this.getAccessToken(); + } catch (tokenError) { + const error = new Error(`OAUTH_TOKEN_ERROR: ${tokenError.message}`); + // logger.logException(error, 'EmailSendingService.sendEmail OAuth token refresh failed'); + return { success: false, error }; + } + + // Configure OAuth2 + mailOptions.auth = { + type: 'OAuth2', + user: this.config.email, + clientId: this.config.clientId, + clientSecret: this.config.clientSecret, + refreshToken: this.config.refreshToken, + accessToken: token, + }; + + // Send email + const result = await this.transporter.sendMail(mailOptions); + + // Enhanced logging for announcements + // logger.logInfo( + // `Announcement email sent to: ${mailOptions.to || mailOptions.bcc || 'unknown'}`, + // result, + // ); + + return { success: true, response: result }; + } catch (error) { + // logger.logException( + // error, + // `Error sending announcement email to: ${mailOptions.to || mailOptions.bcc || 'unknown'}`, + // ); + return { success: false, error }; + } + } + + /** + * Send email with retry logic and announcement-specific handling. + * - Executes exponential backoff between attempts: initialDelayMs * 2^(attempt-1). + * - Never throws; returns final success/failure and attemptCount for auditing. + * @param {Object} mailOptions - Nodemailer-compatible mail options. + * @param {number} retries - Total attempts (>=1). + * @param {number} initialDelayMs - Initial backoff delay in ms. + * @returns {Promise<{success: boolean, response?: Object, error?: Error, attemptCount: number}>} + */ + async sendWithRetry(mailOptions, retries = 3, initialDelayMs = 1000) { + // Validation + if (!mailOptions) { + const error = new Error('INVALID_MAIL_OPTIONS: mailOptions is required'); + // logger.logException(error, 'EmailSendingService.sendWithRetry validation failed'); + return { success: false, error, attemptCount: 0 }; + } + + if (!Number.isInteger(retries) || retries < 1) { + const error = new Error('INVALID_RETRIES: retries must be a positive integer'); + // logger.logException(error, 'EmailSendingService.sendWithRetry validation failed'); + return { success: false, error, attemptCount: 0 }; + } + + let attemptCount = 0; + + /* eslint-disable no-await-in-loop */ + for (let attempt = 1; attempt <= retries; attempt += 1) { + attemptCount += 1; + + try { + const result = await this.sendEmail(mailOptions); + + if (result.success) { + // Store Gmail response for audit logging + mailOptions.gmailResponse = result.response; + // logger.logInfo( + // `Email sent successfully on attempt ${attempt} to: ${mailOptions.to || mailOptions.bcc || 'unknown'}`, + // ); + return { success: true, response: result.response, attemptCount }; + } + // result.success is false - log and try again or return + const error = result.error || new Error('Unknown error from sendEmail'); + // logger.logException( + // error, + // `Announcement send attempt ${attempt} failed to: ${mailOptions.to || mailOptions.bcc || '(empty)'}`, + // ); + + // If this is the last attempt, return failure info + if (attempt >= retries) { + return { success: false, error, attemptCount }; + } + } catch (err) { + // Unexpected error (shouldn't happen since sendEmail now returns {success, error}) + // logger.logException( + // err, + // `Unexpected error in announcement send attempt ${attempt} to: ${mailOptions.to || mailOptions.bcc || '(empty)'}`, + // ); + + // If this is the last attempt, return failure info + if (attempt >= retries) { + return { success: false, error: err, attemptCount }; + } + } + + // Exponential backoff before retry (2^n: 1x, 2x, 4x, 8x, ...) + if (attempt < retries) { + const delay = initialDelayMs * 2 ** (attempt - 1); + await EmailSendingService.sleep(delay); + } + } + /* eslint-enable no-await-in-loop */ + + return { + success: false, + error: new Error('MAX_RETRIES_EXCEEDED: All retry attempts failed'), + attemptCount, + }; + } + + /** + * Sleep utility for backoff timing. + * @param {number} ms - Milliseconds to wait. + * @returns {Promise} + */ + static sleep(ms) { + return new Promise((resolve) => { + setTimeout(resolve, ms); + }); + } +} + +// Create singleton instance +const emailSendingService = new EmailSendingService(); + +module.exports = emailSendingService; diff --git a/src/services/announcements/emails/emailService.js b/src/services/announcements/emails/emailService.js new file mode 100644 index 000000000..d0d8db691 --- /dev/null +++ b/src/services/announcements/emails/emailService.js @@ -0,0 +1,367 @@ +const mongoose = require('mongoose'); +const Email = require('../../../models/email'); +const EmailBatch = require('../../../models/emailBatch'); +const { EMAIL_CONFIG } = require('../../../config/emailConfig'); +const { ensureHtmlWithinLimit } = require('../../../utilities/emailValidators'); + +class EmailService { + /** + * Create a parent Email document for announcements. + * Validates and trims large text fields and supports optional transaction sessions. + * @param {{subject: string, htmlContent: string, createdBy: string|ObjectId}} param0 + * @param {import('mongoose').ClientSession|null} session + * @returns {Promise} Created Email document. + * @throws {Error} If validation fails + */ + static async createEmail({ subject, htmlContent, createdBy }, session = null) { + // Validate required fields + if (!subject || typeof subject !== 'string' || !subject.trim()) { + const error = new Error('Subject is required'); + error.statusCode = 400; + throw error; + } + + if (!htmlContent || typeof htmlContent !== 'string' || !htmlContent.trim()) { + const error = new Error('HTML content is required'); + error.statusCode = 400; + throw error; + } + + if (!createdBy || !mongoose.Types.ObjectId.isValid(createdBy)) { + const error = new Error('Valid createdBy is required'); + error.statusCode = 400; + throw error; + } + + // Validate subject length + const trimmedSubject = subject.trim(); + if (trimmedSubject.length > EMAIL_CONFIG.LIMITS.SUBJECT_MAX_LENGTH) { + const error = new Error( + `Subject cannot exceed ${EMAIL_CONFIG.LIMITS.SUBJECT_MAX_LENGTH} characters`, + ); + error.statusCode = 400; + throw error; + } + + // Validate HTML content size + if (!ensureHtmlWithinLimit(htmlContent)) { + const error = new Error( + `HTML content exceeds ${EMAIL_CONFIG.LIMITS.MAX_HTML_BYTES / (1024 * 1024)}MB limit`, + ); + error.statusCode = 413; + throw error; + } + + const normalizedHtml = htmlContent.trim(); + + const emailData = { + subject: trimmedSubject, + htmlContent: normalizedHtml, + createdBy, + }; + + const email = new Email(emailData); + + // Save with session if provided for transaction support + try { + await email.save({ session }); + } catch (dbError) { + // Handle MongoDB errors + if (dbError.name === 'ValidationError') { + const error = new Error(`Validation error: ${dbError.message}`); + error.statusCode = 400; + throw error; + } + if (dbError.code === 11000) { + const error = new Error('Duplicate key error'); + error.statusCode = 409; + throw error; + } + // Re-throw with status code for other database errors + dbError.statusCode = 500; + throw dbError; + } + + return email; + } + + /** + * Fetch a parent Email by ObjectId. + * @param {string|ObjectId} id + * @param {import('mongoose').ClientSession|null} session + * @param {boolean} throwIfNotFound - If true, throw error with statusCode 404 if not found. Default: false (returns null). + * @param {boolean} populateCreatedBy - If true, populate createdBy field. Default: false. + * @returns {Promise} + * @throws {Error} If throwIfNotFound is true and email is not found + */ + static async getEmailById( + id, + session = null, + throwIfNotFound = false, + populateCreatedBy = false, + ) { + if (!id || !mongoose.Types.ObjectId.isValid(id)) { + if (throwIfNotFound) { + const error = new Error('Valid email ID is required'); + error.statusCode = 400; + throw error; + } + return null; + } + let query = Email.findById(id); + if (session) { + query = query.session(session); + } + if (populateCreatedBy) { + query = query.populate('createdBy', 'firstName lastName email'); + } + const email = await query; + if (!email && throwIfNotFound) { + const error = new Error(`Email ${id} not found`); + error.statusCode = 404; + throw error; + } + return email; + } + + /** + * Update Email status with validation against configured enum. + * @param {string|ObjectId} emailId + * @param {string} status - One of EMAIL_CONFIG.EMAIL_STATUSES.* + * @returns {Promise} Updated Email document. + * @throws {Error} If email not found or invalid status + */ + static async updateEmailStatus(emailId, status) { + if (!emailId || !mongoose.Types.ObjectId.isValid(emailId)) { + const error = new Error('Valid email ID is required'); + error.statusCode = 400; + throw error; + } + if (!Object.values(EMAIL_CONFIG.EMAIL_STATUSES).includes(status)) { + const error = new Error('Invalid email status'); + error.statusCode = 400; + throw error; + } + const email = await Email.findByIdAndUpdate( + emailId, + { status, updatedAt: new Date() }, + { new: true }, + ); + if (!email) { + const error = new Error(`Email ${emailId} not found`); + error.statusCode = 404; + throw error; + } + return email; + } + + /** + * Mark Email as SENDING and set startedAt. + * Uses atomic update with condition to prevent race conditions. + * @param {string|ObjectId} emailId + * @returns {Promise} Updated Email document. + * @throws {Error} If email not found or not in PENDING status + */ + static async markEmailStarted(emailId) { + const now = new Date(); + const email = await Email.findOneAndUpdate( + { + _id: emailId, + status: EMAIL_CONFIG.EMAIL_STATUSES.PENDING, + }, + { + status: EMAIL_CONFIG.EMAIL_STATUSES.SENDING, + startedAt: now, + updatedAt: now, + }, + { new: true }, + ); + + if (!email) { + const error = new Error(`Email ${emailId} not found or not in PENDING status`); + error.statusCode = 404; + throw error; + } + + return email; + } + + /** + * Mark Email as completed with final status, setting completedAt. + * Falls back to SENT if an invalid finalStatus is passed. + * @param {string|ObjectId} emailId + * @param {string} finalStatus + * @returns {Promise} Updated Email document. + * @throws {Error} If email not found + */ + static async markEmailCompleted(emailId, finalStatus) { + const now = new Date(); + const statusToSet = Object.values(EMAIL_CONFIG.EMAIL_STATUSES).includes(finalStatus) + ? finalStatus + : EMAIL_CONFIG.EMAIL_STATUSES.SENT; + + const email = await Email.findByIdAndUpdate( + emailId, + { + status: statusToSet, + completedAt: now, + updatedAt: now, + }, + { new: true }, + ); + if (!email) { + const error = new Error(`Email ${emailId} not found`); + error.statusCode = 404; + throw error; + } + return email; + } + + /** + * Mark an Email as PENDING for retry and clear timing fields. + * @param {string|ObjectId} emailId + * @returns {Promise} Updated Email document. + * @throws {Error} If email not found + */ + static async markEmailPending(emailId) { + const now = new Date(); + const email = await Email.findByIdAndUpdate( + emailId, + { + status: EMAIL_CONFIG.EMAIL_STATUSES.PENDING, + startedAt: null, + completedAt: null, + updatedAt: now, + }, + { new: true }, + ); + if (!email) { + const error = new Error(`Email ${emailId} not found`); + error.statusCode = 404; + throw error; + } + return email; + } + + /** + * Get all Emails ordered by creation date descending. + * Includes aggregated recipient counts from EmailBatch items. + * @returns {Promise} Array of Email objects (lean, with createdBy populated and recipient counts). + */ + static async getAllEmails() { + const emails = await Email.find() + .sort({ createdAt: -1 }) + .populate('createdBy', 'firstName lastName email') + .lean(); + + // Aggregate recipient counts and batch counts from EmailBatch for each email + const emailsWithCounts = await Promise.all( + emails.map(async (email) => { + // Get total recipients count (sum of all recipients across all batches) + const totalRecipientsAggregation = await EmailBatch.aggregate([ + { $match: { emailId: email._id } }, + { + $group: { + _id: null, + totalRecipients: { + $sum: { $size: { $ifNull: ['$recipients', []] } }, + }, + }, + }, + ]); + + // Count batches by status (for sentEmails and failedEmails) + const batchCountsByStatus = await EmailBatch.aggregate([ + { $match: { emailId: email._id } }, + { + $group: { + _id: '$status', + batchCount: { $sum: 1 }, + }, + }, + ]); + + // Calculate totals + const totalEmails = + totalRecipientsAggregation.length > 0 + ? totalRecipientsAggregation[0].totalRecipients || 0 + : 0; + + let sentEmails = 0; // Batch count + let failedEmails = 0; // Batch count + + batchCountsByStatus.forEach((result) => { + if (result._id === EMAIL_CONFIG.EMAIL_BATCH_STATUSES.SENT) { + sentEmails = result.batchCount || 0; + } else if (result._id === EMAIL_CONFIG.EMAIL_BATCH_STATUSES.FAILED) { + failedEmails = result.batchCount || 0; + } + }); + + return { + ...email, + totalEmails, // Total recipients + sentEmails, // Count of SENT batches + failedEmails, // Count of FAILED batches + }; + }), + ); + + return emailsWithCounts; + } + + /** + * Get all PENDING emails that need to be processed. + * @returns {Promise} Array of Email objects with PENDING status. + */ + static async getPendingEmails() { + return Email.find({ + status: EMAIL_CONFIG.EMAIL_STATUSES.PENDING, + }) + .sort({ createdAt: 1 }) // Process oldest first + .lean(); + } + + /** + * Get all STUCK emails (SENDING status). + * On server restart, any email in SENDING status is considered stuck because + * the processing was interrupted. We reset ALL SENDING emails because the + * server restart means they're no longer being processed. + * @returns {Promise} Array of Email objects with SENDING status that are stuck. + */ + static async getStuckEmails() { + // On server restart: Reset ALL emails in SENDING status (they're all stuck) + return Email.find({ + status: EMAIL_CONFIG.EMAIL_STATUSES.SENDING, + }) + .sort({ startedAt: 1 }) // Process oldest first + .lean(); + } + + /** + * Reset stuck email to PENDING status so it can be reprocessed. + * @param {string|ObjectId} emailId + * @returns {Promise} Updated Email document. + * @throws {Error} If email not found + */ + static async resetStuckEmail(emailId) { + const now = new Date(); + const email = await Email.findByIdAndUpdate( + emailId, + { + status: EMAIL_CONFIG.EMAIL_STATUSES.PENDING, + startedAt: null, // Clear startedAt so it can be reprocessed + updatedAt: now, + }, + { new: true }, + ); + if (!email) { + const error = new Error(`Email ${emailId} not found`); + error.statusCode = 404; + throw error; + } + return email; + } +} + +module.exports = EmailService; diff --git a/src/services/announcements/emails/emailTemplateService.js b/src/services/announcements/emails/emailTemplateService.js new file mode 100644 index 000000000..8b2f641db --- /dev/null +++ b/src/services/announcements/emails/emailTemplateService.js @@ -0,0 +1,739 @@ +/** + * Email Template Service - Manages EmailTemplate operations + * Provides business logic for template CRUD operations, validation, and rendering + */ + +const mongoose = require('mongoose'); +const sanitizeHtmlLib = require('sanitize-html'); +const EmailTemplate = require('../../../models/emailTemplate'); +const { EMAIL_CONFIG } = require('../../../config/emailConfig'); +const { ensureHtmlWithinLimit } = require('../../../utilities/emailValidators'); +// const logger = require('../../../startup/logger'); + +class EmailTemplateService { + /** + * Validate template variables. + * - Ensures non-empty unique names and validates allowed types. + * - Allowed types are defined in EMAIL_CONFIG.TEMPLATE_VARIABLE_TYPES + * @param {Array<{name: string, type?: string}>} variables - Variable definitions + * @returns {{isValid: boolean, errors?: string[]}} + */ + static validateTemplateVariables(variables) { + if (!variables || !Array.isArray(variables)) { + return { isValid: true, errors: [] }; + } + + const errors = []; + const variableNames = new Set(); + + variables.forEach((variable, index) => { + if (!variable.name || typeof variable.name !== 'string' || !variable.name.trim()) { + errors.push(`Variable ${index + 1}: name is required and must be a non-empty string`); + } else { + const varName = variable.name.trim(); + // Validate variable name format (alphanumeric and underscore only) + if (!/^[a-zA-Z0-9_]+$/.test(varName)) { + errors.push( + `Variable ${index + 1}: name must contain only alphanumeric characters and underscores`, + ); + } + // Check for duplicates (case-insensitive) + if (variableNames.has(varName.toLowerCase())) { + errors.push(`Variable ${index + 1}: duplicate variable name '${varName}'`); + } + variableNames.add(varName.toLowerCase()); + } + + // Validate type - type is required and must be one of: text, number, image + if (!variable.type) { + errors.push(`Variable ${index + 1}: type is required`); + } else if (!EMAIL_CONFIG.TEMPLATE_VARIABLE_TYPES.includes(variable.type)) { + errors.push( + `Variable ${index + 1}: type '${variable.type}' is invalid. Type must be one of: ${EMAIL_CONFIG.TEMPLATE_VARIABLE_TYPES.join(', ')}`, + ); + } + }); + + return { + isValid: errors.length === 0, + errors, + }; + } + + /** + * Validate template content (HTML and subject) against defined variables. + * - Flags undefined placeholders and unused defined variables. + * @param {Array<{name: string}>} templateVariables + * @param {string} htmlContent + * @param {string} subject + * @returns {{isValid: boolean, errors: string[]}} + */ + static validateTemplateVariableUsage(templateVariables, htmlContent, subject) { + const errors = []; + + if (!templateVariables || templateVariables.length === 0) { + return { isValid: true, errors: [] }; + } + + // Extract variable placeholders from content (format: {{variableName}}) + const variablePlaceholderRegex = /\{\{(\w+)\}\}/g; + const usedVariables = new Set(); + const foundPlaceholders = []; + + // Check HTML content + if (htmlContent) { + let match = variablePlaceholderRegex.exec(htmlContent); + while (match !== null) { + const varName = match[1]; + foundPlaceholders.push(varName); + usedVariables.add(varName); + match = variablePlaceholderRegex.exec(htmlContent); + } + } + + // Reset regex for subject + variablePlaceholderRegex.lastIndex = 0; + + // Check subject + if (subject) { + let match = variablePlaceholderRegex.exec(subject); + while (match !== null) { + const varName = match[1]; + foundPlaceholders.push(varName); + usedVariables.add(varName); + match = variablePlaceholderRegex.exec(subject); + } + } + + // Check for undefined variable placeholders in content + const definedVariableNames = templateVariables.map((v) => v.name); + foundPlaceholders.forEach((placeholder) => { + if (!definedVariableNames.includes(placeholder)) { + errors.push( + `Variable placeholder '{{${placeholder}}}' is used in content but not defined in template variables`, + ); + } + }); + + // Check for defined variables that are not used in content (treated as errors) + templateVariables.forEach((variable) => { + if (!usedVariables.has(variable.name)) { + errors.push(`Variable '{{${variable.name}}}' is defined but not used in template content`); + } + }); + + return { + isValid: errors.length === 0, + errors, + }; + } + + /** + * Validate template data (name, subject, HTML, variables). + * @param {Object} templateData - Template data to validate + * @returns {{isValid: boolean, errors: string[]}} + */ + static validateTemplateData(templateData) { + const errors = []; + const { name, subject, html_content: htmlContent, variables } = templateData; + + // Validate name + if (!name || typeof name !== 'string' || !name.trim()) { + errors.push('Template name is required'); + } else { + const trimmedName = name.trim(); + if (trimmedName.length > EMAIL_CONFIG.LIMITS.TEMPLATE_NAME_MAX_LENGTH) { + errors.push( + `Template name cannot exceed ${EMAIL_CONFIG.LIMITS.TEMPLATE_NAME_MAX_LENGTH} characters`, + ); + } + } + + // Validate subject + if (!subject || typeof subject !== 'string' || !subject.trim()) { + errors.push('Template subject is required'); + } else { + const trimmedSubject = subject.trim(); + if (trimmedSubject.length > EMAIL_CONFIG.LIMITS.SUBJECT_MAX_LENGTH) { + errors.push(`Subject cannot exceed ${EMAIL_CONFIG.LIMITS.SUBJECT_MAX_LENGTH} characters`); + } + } + + // Validate HTML content + if (!htmlContent || typeof htmlContent !== 'string' || !htmlContent.trim()) { + errors.push('Template HTML content is required'); + } else if (!ensureHtmlWithinLimit(htmlContent)) { + errors.push( + `HTML content exceeds ${EMAIL_CONFIG.LIMITS.MAX_HTML_BYTES / (1024 * 1024)}MB limit`, + ); + } + + // Validate variables + if (variables && variables.length > 0) { + const variableValidation = this.validateTemplateVariables(variables); + if (!variableValidation.isValid) { + errors.push(...variableValidation.errors); + } + + // Validate variable usage in content + const variableUsageValidation = this.validateTemplateVariableUsage( + variables, + htmlContent, + subject, + ); + if (!variableUsageValidation.isValid) { + errors.push(...variableUsageValidation.errors); + } + } else { + // If no variables are defined, check for any variable placeholders in content + const variablePlaceholderRegex = /\{\{(\w+)\}\}/g; + const foundInHtml = variablePlaceholderRegex.test(htmlContent); + variablePlaceholderRegex.lastIndex = 0; + const foundInSubject = variablePlaceholderRegex.test(subject); + + if (foundInHtml || foundInSubject) { + errors.push( + 'Template content contains variable placeholders ({{variableName}}) but no variables are defined. Please define variables or remove placeholders from content.', + ); + } + } + + return { + isValid: errors.length === 0, + errors, + }; + } + + /** + * Create a new email template. + * @param {Object} templateData - Template data + * @param {string|ObjectId} userId - User ID creating the template + * @returns {Promise} Created template + * @throws {Error} If validation fails or template already exists + */ + static async createTemplate(templateData, userId) { + const { name, subject, html_content: htmlContent, variables } = templateData; + + // Validate template data + const validation = this.validateTemplateData(templateData); + if (!validation.isValid) { + // Create descriptive error message from validation errors + const errorCount = validation.errors.length; + const errorMessage = + errorCount === 1 ? validation.errors[0] : `Validation failed: ${errorCount} error(s) found`; + + const error = new Error(errorMessage); + error.errors = validation.errors; + error.statusCode = 400; + throw error; + } + + // Validate userId + if (userId && !mongoose.Types.ObjectId.isValid(userId)) { + const error = new Error('Invalid user ID'); + error.statusCode = 400; + throw error; + } + + const trimmedName = name.trim(); + const trimmedSubject = subject.trim(); + + // Check if template with the same name already exists (case-insensitive) + const existingTemplate = await this.templateNameExists(trimmedName, null); + + if (existingTemplate) { + const error = new Error('Email template with this name already exists'); + error.statusCode = 409; + throw error; + } + + // Create new email template + const template = new EmailTemplate({ + name: trimmedName, + subject: trimmedSubject, + html_content: htmlContent.trim(), + variables: variables || [], + created_by: userId, + updated_by: userId, + }); + try { + await template.save(); + } catch (dbError) { + // Handle MongoDB errors + if (dbError.name === 'ValidationError') { + const error = new Error(`Validation error: ${dbError.message}`); + error.statusCode = 400; + throw error; + } + if (dbError.code === 11000) { + const error = new Error('Email template with this name already exists'); + error.statusCode = 409; + throw error; + } + // Re-throw with status code for other database errors + dbError.statusCode = 500; + throw dbError; + } + + // Populate created_by and updated_by fields + await template.populate('created_by', 'firstName lastName email'); + await template.populate('updated_by', 'firstName lastName email'); + + // logger.logInfo(`Email template created: ${template.name} by user ${userId}`); + + return template; + } + + /** + * Get template by ID. + * @param {string|ObjectId} id - Template ID + * @param {Object} options - Query options (populate) + * @returns {Promise} Template + * @throws {Error} If template not found or invalid ID + */ + static async getTemplateById(id, options = {}) { + if (!id || !mongoose.Types.ObjectId.isValid(id)) { + const error = new Error('Invalid template ID'); + error.statusCode = 400; + throw error; + } + + const { populate = true } = options; + + let template = EmailTemplate.findById(id); + + if (populate) { + template = template + .populate('created_by', 'firstName lastName email') + .populate('updated_by', 'firstName lastName email'); + } + + const result = await template; + if (!result) { + const error = new Error('Email template not found'); + error.statusCode = 404; + throw error; + } + + return result; + } + + /** + * Get all templates with optional filtering and sorting. + * @param {Object} query - MongoDB query + * @param {Object} options - Query options (sort, projection, populate) + * @returns {Promise} Array of templates + */ + static async getAllTemplates(query = {}, options = {}) { + const { sort = { created_at: -1 }, projection = null, populate = true } = options; + + let queryBuilder = EmailTemplate.find(query); + + if (projection) { + queryBuilder = queryBuilder.select(projection); + } + + if (sort) { + queryBuilder = queryBuilder.sort(sort); + } + + if (populate) { + queryBuilder = queryBuilder + .populate('created_by', 'firstName lastName') + .populate('updated_by', 'firstName lastName'); + } + + return queryBuilder.lean(); + } + + /** + * Update an existing template. + * @param {string|ObjectId} id - Template ID + * @param {Object} templateData - Updated template data + * @param {string|ObjectId} userId - User ID updating the template + * @returns {Promise} Updated template + * @throws {Error} If validation fails or template not found + */ + static async updateTemplate(id, templateData, userId) { + if (!id || !mongoose.Types.ObjectId.isValid(id)) { + const error = new Error('Invalid template ID'); + error.statusCode = 400; + throw error; + } + + // Validate template data + const validation = this.validateTemplateData(templateData); + if (!validation.isValid) { + // Create descriptive error message from validation errors + const errorCount = validation.errors.length; + const errorMessage = + errorCount === 1 ? validation.errors[0] : `Validation failed: ${errorCount} error(s) found`; + + const error = new Error(errorMessage); + error.errors = validation.errors; + error.statusCode = 400; + throw error; + } + + // Validate userId + if (userId && !mongoose.Types.ObjectId.isValid(userId)) { + const error = new Error('Invalid user ID'); + error.statusCode = 400; + throw error; + } + + // Get current template + const currentTemplate = await EmailTemplate.findById(id); + + if (!currentTemplate) { + const error = new Error('Email template not found'); + error.statusCode = 404; + throw error; + } + + const { name, subject, html_content: htmlContent, variables } = templateData; + const trimmedName = name.trim(); + const trimmedSubject = subject.trim(); + + // Only check for duplicate names if the name is actually changing (case-insensitive) + if (currentTemplate.name.toLowerCase() !== trimmedName.toLowerCase()) { + const existingTemplate = await EmailTemplate.findOne({ + name: { $regex: new RegExp(`^${trimmedName}$`, 'i') }, + _id: { $ne: id }, + }); + + if (existingTemplate) { + const error = new Error('Another email template with this name already exists'); + error.statusCode = 409; + throw error; + } + } + + // Update template + const updateData = { + name: trimmedName, + subject: trimmedSubject, + html_content: htmlContent.trim(), + variables: variables || [], + updated_by: userId, + }; + + let template; + try { + template = await EmailTemplate.findByIdAndUpdate(id, updateData, { + new: true, + runValidators: true, + }) + .populate('created_by', 'firstName lastName email') + .populate('updated_by', 'firstName lastName email'); + } catch (dbError) { + // Handle MongoDB errors + if (dbError.name === 'ValidationError') { + const error = new Error(`Validation error: ${dbError.message}`); + error.statusCode = 400; + throw error; + } + if (dbError.code === 11000) { + const error = new Error('Another email template with this name already exists'); + error.statusCode = 409; + throw error; + } + // Re-throw with status code for other database errors + dbError.statusCode = 500; + throw dbError; + } + + if (!template) { + const error = new Error('Email template not found'); + error.statusCode = 404; + throw error; + } + + // logger.logInfo(`Email template updated: ${template.name} by user ${userId}`); + + return template; + } + + /** + * Delete a template (hard delete). + * @param {string|ObjectId} id - Template ID + * @returns {Promise} Deleted template + * @throws {Error} If template not found + */ + static async deleteTemplate(id) { + if (!id || !mongoose.Types.ObjectId.isValid(id)) { + const error = new Error('Invalid template ID'); + error.statusCode = 400; + throw error; + } + + const template = await EmailTemplate.findById(id); + + if (!template) { + const error = new Error('Email template not found'); + error.statusCode = 404; + throw error; + } + + // Hard delete + await EmailTemplate.findByIdAndDelete(id); + + // logger.logInfo(`Email template deleted: ${template.name} by user ${userId}`); + + return template; + } + + /** + * Check if template name exists (case-insensitive). + * @param {string} name - Template name + * @param {string|ObjectId} excludeId - Template ID to exclude from check + * @returns {Promise} True if name exists + */ + static async templateNameExists(name, excludeId = null) { + const query = { + name: { $regex: new RegExp(`^${name}$`, 'i') }, + }; + + if (excludeId) { + query._id = { $ne: excludeId }; + } + + const existing = await EmailTemplate.findOne(query); + return !!existing; + } + + /** + * Render template with variable values. + * Replaces {{variableName}} placeholders with actual values. + * @param {Object} template - Template object with subject and html_content + * @param {Object} variables - Object mapping variable names to values + * @param {Object} options - Rendering options (sanitize, strict) + * @returns {{subject: string, htmlContent: string}} Rendered template + */ + static renderTemplate(template, variables = {}, options = {}) { + const { sanitize = true, strict = false } = options; + + if (!template) { + const error = new Error('Template is required'); + error.statusCode = 400; + throw error; + } + + let subject = template.subject || ''; + let htmlContent = template.html_content || template.htmlContent || ''; + + // Get template variables + const templateVariables = template.variables || []; + + // Replace variables in subject and HTML + templateVariables.forEach((variable) => { + if (!variable || !variable.name) return; + + const varName = variable.name; + const value = variables[varName]; + + // In strict mode, throw error if variable is missing + if (strict && value === undefined) { + const error = new Error(`Missing required variable: ${varName}`); + error.statusCode = 400; + throw error; + } + + // Skip if value is not provided + if (value === undefined || value === null) { + return; + } + + // Handle image variables + let processedValue = value; + if (variable.type === 'image') { + // Use extracted image if available + const extractedKey = `${varName}_extracted`; + if (variables[extractedKey]) { + processedValue = variables[extractedKey]; + } else if (typeof value === 'string') { + // Try to extract image URL from value + const imageMatch = + value.match(/src=["']([^"']+)["']/i) || value.match(/https?:\/\/[^\s]+/i); + if (imageMatch) { + processedValue = imageMatch[1] || imageMatch[0]; + } + } + } + + // Replace all occurrences of {{variableName}} + const regex = new RegExp(`\\{\\{${varName}\\}\\}`, 'g'); + subject = subject.replace(regex, String(processedValue)); + htmlContent = htmlContent.replace(regex, String(processedValue)); + }); + + // Sanitize HTML if requested + if (sanitize) { + htmlContent = this.sanitizeHtml(htmlContent); + } + + return { + subject: subject.trim(), + htmlContent: htmlContent.trim(), + }; + } + + /** + * Validate that all required variables are provided. + * @param {Object} template - Template object + * @param {Object} variables - Variable values + * @returns {{isValid: boolean, errors: string[], missing: string[]}} Validation result + */ + static validateVariables(template, variables = {}) { + const errors = []; + const missing = []; + const templateVariables = template.variables || []; + + // Check for missing variables + templateVariables.forEach((variable) => { + if (!variable || !variable.name) return; + + const varName = variable.name; + if ( + !(varName in variables) || + variables[varName] === undefined || + variables[varName] === null + ) { + missing.push(varName); + errors.push(`Missing required variable: ${varName}`); + } + }); + + // Check for unused variables + const templateVariableNames = new Set(templateVariables.map((v) => v.name)); + Object.keys(variables).forEach((varName) => { + if (!templateVariableNames.has(varName) && !varName.endsWith('_extracted')) { + errors.push(`Unknown variable: ${varName}`); + } + }); + + return { + isValid: errors.length === 0, + errors, + missing, + }; + } + + /** + * Check if template has unreplaced variables. + * @param {string} content - Content to check (subject or HTML) + * @returns {string[]} Array of unreplaced variable names + */ + static getUnreplacedVariables(content) { + if (!content || typeof content !== 'string') { + return []; + } + + const variablePlaceholderRegex = /\{\{(\w+)\}\}/g; + const unreplaced = []; + let match = variablePlaceholderRegex.exec(content); + + while (match !== null) { + const varName = match[1]; + if (!unreplaced.includes(varName)) { + unreplaced.push(varName); + } + match = variablePlaceholderRegex.exec(content); + } + + return unreplaced; + } + + /** + * Sanitize HTML content to prevent XSS attacks. + * @param {string} html - HTML content to sanitize + * @param {Object} options - Sanitization options + * @returns {string} Sanitized HTML + */ + static sanitizeHtml(html, options = {}) { + if (!html || typeof html !== 'string') { + return ''; + } + + const defaultOptions = { + allowedTags: [ + 'p', + 'br', + 'strong', + 'em', + 'b', + 'i', + 'u', + 'a', + 'ul', + 'ol', + 'li', + 'h1', + 'h2', + 'h3', + 'h4', + 'h5', + 'h6', + 'div', + 'span', + 'img', + 'table', + 'thead', + 'tbody', + 'tr', + 'td', + 'th', + 'blockquote', + 'hr', + ], + allowedAttributes: { + a: ['href', 'title', 'target', 'rel'], + img: ['src', 'alt', 'title'], + '*': ['style', 'class'], + }, + allowedSchemes: ['http', 'https', 'mailto'], + allowedSchemesByTag: { + img: ['http', 'https', 'data'], + }, + ...options, + }; + + return sanitizeHtmlLib(html, defaultOptions); + } + + /** + * Extract variables from template content. + * @param {Object} template - Template object + * @returns {string[]} Array of variable names found in content + */ + static extractVariablesFromContent(template) { + const variables = new Set(); + const variablePlaceholderRegex = /\{\{(\w+)\}\}/g; + + // Check subject + if (template.subject) { + let match = variablePlaceholderRegex.exec(template.subject); + while (match !== null) { + variables.add(match[1]); + match = variablePlaceholderRegex.exec(template.subject); + } + } + + // Reset regex + variablePlaceholderRegex.lastIndex = 0; + + // Check HTML content + const htmlContent = template.html_content || template.htmlContent || ''; + if (htmlContent) { + let match = variablePlaceholderRegex.exec(htmlContent); + while (match !== null) { + variables.add(match[1]); + match = variablePlaceholderRegex.exec(htmlContent); + } + } + + return Array.from(variables); + } +} + +module.exports = EmailTemplateService; diff --git a/src/startup/routes.js b/src/startup/routes.js index 71526e7a1..18da7aa25 100644 --- a/src/startup/routes.js +++ b/src/startup/routes.js @@ -170,6 +170,7 @@ const rolePresetRouter = require('../routes/rolePresetRouter')(rolePreset); const ownerMessageRouter = require('../routes/ownerMessageRouter')(ownerMessage); const emailRouter = require('../routes/emailRouter')(); +const emailOutboxRouter = require('../routes/emailOutboxRouter'); const reasonRouter = require('../routes/reasonRouter')(reason, userProfile); const mouseoverTextRouter = require('../routes/mouseoverTextRouter')(mouseoverText); @@ -312,6 +313,7 @@ const supplierPerformanceRouter = require('../routes/summaryDashboard/supplierPe const registrationRouter = require('../routes/registrationRouter')(registration); const templateRouter = require('../routes/templateRouter'); +const emailTemplateRouter = require('../routes/emailTemplateRouter'); const projectMaterialRouter = require('../routes/projectMaterialroutes'); @@ -390,6 +392,7 @@ module.exports = function (app) { app.use('/api', informationRouter); app.use('/api', mouseoverTextRouter); app.use('/api', permissionChangeLogRouter); + app.use('/api', emailOutboxRouter); app.use('/api', emailRouter); app.use('/api', isEmailExistsRouter); app.use('/api', faqRouter); @@ -423,6 +426,7 @@ module.exports = function (app) { app.use('/api/costs', costsRouter); app.use('/api', hoursPledgedRoutes); app.use('/api', templateRouter); + app.use('/api', emailTemplateRouter); app.use('/api/help-categories', helpCategoryRouter); app.use('/api', tagRouter); diff --git a/src/test/createTestPermissions.js b/src/test/createTestPermissions.js index 8b32993bc..efa923a96 100644 --- a/src/test/createTestPermissions.js +++ b/src/test/createTestPermissions.js @@ -204,7 +204,6 @@ const permissionsRoles = [ 'deleteTimeEntry', 'postTimeEntry', 'sendEmails', - 'sendEmailToAll', 'updatePassword', 'resetPassword', 'getUserProfiles', diff --git a/src/utilities/createInitialPermissions.js b/src/utilities/createInitialPermissions.js index 6fa4f3925..ab29f185d 100644 --- a/src/utilities/createInitialPermissions.js +++ b/src/utilities/createInitialPermissions.js @@ -243,7 +243,6 @@ const permissionsRoles = [ 'deleteTimeEntry', 'postTimeEntry', 'sendEmails', - 'sendEmailToAll', 'updatePassword', 'resetPassword', 'getUserProfiles', diff --git a/src/utilities/emailValidators.js b/src/utilities/emailValidators.js new file mode 100644 index 000000000..2193221bf --- /dev/null +++ b/src/utilities/emailValidators.js @@ -0,0 +1,101 @@ +const { EMAIL_CONFIG } = require('../config/emailConfig'); + +/** + * Validate email address format + * @param {string} email - Email address to validate + * @returns {boolean} True if valid email format + */ +function isValidEmailAddress(email) { + if (!email || typeof email !== 'string') return false; + const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/; + return emailRegex.test(email.trim()); +} + +/** + * Normalize recipients input to array of email strings + * Handles both array and single value, removes duplicates (case-insensitive) + * @param {string|Array} input - Recipient(s) to normalize + * @returns {Array} Array of unique email strings + */ +function normalizeRecipientsToArray(input) { + const arr = Array.isArray(input) ? input : [input]; + const trimmed = arr + .map((e) => (typeof e === 'string' ? e.trim() : '')) + .filter((e) => e.length > 0); + // Dedupe case-insensitively + const seen = new Set(); + return trimmed.filter((e) => { + const key = e.toLowerCase(); + if (seen.has(key)) return false; + seen.add(key); + return true; + }); +} + +/** + * Normalize recipients input to array of { email } objects + * Used by EmailBatchService for creating EmailBatch records + * @param {Array} input - Recipients array + * @returns {Array<{email: string}>} Array of recipient objects + */ +function normalizeRecipientsToObjects(input) { + if (!Array.isArray(input)) return []; + const emails = input + .filter((item) => { + if (typeof item === 'string') { + return item.trim().length > 0; + } + return item && typeof item.email === 'string' && item.email.trim().length > 0; + }) + .map((item) => ({ + email: typeof item === 'string' ? item.trim() : item.email.trim(), + })); + + // Dedupe case-insensitively + const seen = new Set(); + return emails.filter((obj) => { + const key = obj.email.toLowerCase(); + if (seen.has(key)) return false; + seen.add(key); + return true; + }); +} + +/** + * Validate HTML content size is within limit + * @param {string} html - HTML content to validate + * @returns {boolean} True if within limit + */ +function ensureHtmlWithinLimit(html) { + const maxBytes = EMAIL_CONFIG.LIMITS.MAX_HTML_BYTES; + const size = Buffer.byteLength(html || '', 'utf8'); + return size <= maxBytes; +} + +/** + * Normalize email field (to, cc, bcc) to array format. + * Handles arrays, comma-separated strings, single strings, or null/undefined. + * @param {string|string[]|null|undefined} field - Email field to normalize + * @returns {string[]} Array of email addresses (empty array if input is invalid) + */ +function normalizeEmailField(field) { + if (!field) { + return []; + } + if (Array.isArray(field)) { + return field.filter((e) => e && typeof e === 'string' && e.trim().length > 0); + } + // Handle comma-separated string + return String(field) + .split(',') + .map((e) => e.trim()) + .filter((e) => e.length > 0); +} + +module.exports = { + isValidEmailAddress, + normalizeRecipientsToArray, + normalizeRecipientsToObjects, + ensureHtmlWithinLimit, + normalizeEmailField, +}; diff --git a/src/utilities/transactionHelper.js b/src/utilities/transactionHelper.js new file mode 100644 index 000000000..eacc39bf4 --- /dev/null +++ b/src/utilities/transactionHelper.js @@ -0,0 +1,36 @@ +/** + * Transaction Helper Utility + * Provides a reusable wrapper for MongoDB transactions with proper error handling + */ + +const mongoose = require('mongoose'); +const logger = require('../startup/logger'); + +/** + * Execute a callback within a MongoDB transaction. + * Handles session creation, transaction commit/abort, and cleanup automatically. + * @param {Function} callback - Async function that receives a session parameter + * @returns {Promise} Result from the callback + * @throws {Error} Re-throws any error from the callback after aborting transaction + */ +async function withTransaction(callback) { + const session = await mongoose.startSession(); + session.startTransaction(); + + try { + const result = await callback(session); + await session.commitTransaction(); + return result; + } catch (error) { + try { + await session.abortTransaction(); + } catch (abortError) { + logger.logException(abortError, 'Error aborting transaction'); + } + throw error; + } finally { + session.endSession(); + } +} + +module.exports = { withTransaction };