v6.2.1

Version Number

6.2.1-176

New Features

• None

Patch Fixes Included

• This release includes OS package updates only.

Known Issues

• None

Release Artifacts

Maven

• com.github.jobservice:job-service-admin:6.2.1-176 (javadoc.jar,javadoc.jar.asc,pom,pom.asc,sources.jar,sources.jar.asc,war,war.asc)
• com.github.jobservice:job-service-contract:6.2.1-176 (jar,jar.asc,pom,pom.asc,sources.jar,sources.jar.asc)
• com.github.jobservice:job-service-caller:6.2.1-176 (pom,pom.asc)
• com.github.jobservice:job-service-util:6.2.1-176 (jar,jar.asc,javadoc.jar,javadoc.jar.asc,pom,pom.asc,sources.jar,sources.jar.asc)
• com.github.jobservice:worker-jobtracking-container:6.2.1-176 (pom,pom.asc)
• com.github.jobservice:job-service-config:6.2.1-176 (jar,jar.asc,javadoc.jar,javadoc.jar.asc,pom,pom.asc,sources.jar,sources.jar.asc)
• com.github.jobservice:job-service-html:6.2.1-176 (pom,pom.asc,war,war.asc)
• com.github.jobservice:job-service-postgres-container:6.2.1-176 (pom,pom.asc)
• com.github.jobservice:job-service-db:6.2.1-176 (jar,jar-with-dependencies.jar,jar-with-dependencies.jar.asc,jar.asc,pom,pom.asc,sources.jar,sources.jar.asc)
• com.github.jobservice:job-service-aggregator:6.2.1-176 (pom,pom.asc)
• com.github.jobservice:job-service-scheduled-executor:6.2.1-176 (jar,jar.asc,javadoc.jar,javadoc.jar.asc,pom,pom.asc,sources.jar,sources.jar.asc)
• com.github.jobservice:job-service-container:6.2.1-176 (pom,[pom.asc](https://oss.sonatype.org/content/groups/public/com/github...

v6.2.0

Version Number

6.2.0-136

New Features

• 627110: The Job Service Scheduled Executor now supports message prioritization, allowing it to reroute messages to staging queues.

It can be configured via the following environment variables:

CAF_WMP_ENABLED
description: Determines whether the Job Service Scheduled Executor should reroute a message to a worker's staging queue or not. If true, a message will attempt to be rerouted. If false, a message will not be rerouted and will be sent to the target queue rather than to a staging queue.
default: false

CAF_WMP_PARTITION_ID_PATTERN
description: Only applies when CAF_WMP_ENABLED is true. Used to specify the partition ID pattern. This pattern is used by the Job Service Scheduled Executor to extract the tenant ID from the partition ID. The tenant ID is then used to construct the staging queue name. The pattern must contain a named capturing group called tenantId, which is what is used to extract the tenant ID.
default: None
example: If the pattern is ^tenant-(?<tenantId>.+)# and the partition ID is tenant-acmecorp, the tenant ID extracted from this partition ID will be acmecorp.

CAF_WMP_TARGET_QUEUE_NAMES_PATTERN
description: Only applies when CAF_WMP_ENABLED is true. Used to specify the target queue names pattern. This pattern is used by the Job Service Scheduled Executor to check whether it should reroute a message to a staging queue or not. Only messages destined for target queues that match this pattern will be rerouted to staging queues.
default: None

• US632159: The Job Service Scheduled Executor's healthcheck will now check the status of the Rabbit MQ Management API whenever the CAF_WMP_ENABLED environment variable is true.

• D632168: Updated to use Tomcat version 9.0.73 This includes fixes for CVE-2023-24998.\n\n- US353672: Ping operation added to the Job Service's REST interface

Known Issues

• None

Release Artifacts

Maven

• com.github.jobservice:job-service-html:6.2.0-136 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-db-client:6.2.0-136 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:worker-jobtracking-shared:6.2.0-136 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:job-service:6.2.0-136 (javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar,war.md5,war.sha1,war)
• com.github.jobservice:job-service-ui:6.2.0-136 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-config:6.2.0-136 (jar.md5,jar.sha1,[jar](https://oss....

v6.1.0

Version Number

6.1.0-72

New Features

• US593021: Updated to run on Java 17.

Patch Fixes Included

• US572082: Gson version upgraded to 2.9.1
• US572083: Snakeyaml version upgraded to 1.32

Known Issues

• None

Release Artifacts

Maven

• com.github.jobservice:job-service-html:6.1.0-72 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-db-client:6.1.0-72 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:worker-jobtracking-shared:6.1.0-72 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:job-service:6.1.0-72 (javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar,war.md5,war.sha1,war)
• com.github.jobservice:job-service-ui:6.1.0-72 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-config:6.1.0-72 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:job-service-db:6.1.0-72 (jar-with-dependencies.jar.md5,[jar-with-dependencies.jar.sha1](https://oss.sonatype.org/content/groups/public/com/github/jobservice/job-service-db/6.1.0-72/job-service-d...

v6.0.1

Version Number

6.0.1-54

New Features

• None

Patch Fixes Included

• This release includes dependency updates only.

Known Issues

• None

Release Artifacts

Maven

• com.github.jobservice:job-service-html:6.0.1-54 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-db-client:6.0.1-54 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:worker-jobtracking-shared:6.0.1-54 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:job-service:6.0.1-54 (javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar,war.md5,war.sha1,war)
• com.github.jobservice:job-service-ui:6.0.1-54 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-config:6.0.1-54 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:job-service-db:6.0.1-54 (jar-with-dependencies.jar.md5,jar-with-dependencies.jar.sha1,[jar-with-dependencies.jar](https://oss.sonatype.org/content/groups/public/com/github/jobservice/job-service-db/6.0.1-54/job-service-db-6.0.1-54-jar-with...

v6.0.0

Version Number

6.0.0-31

New Features

• 359648: taskData no longer sent as Base64.
  taskData is no longer Base64 encoded in order to reduce message size.
• 353257: Flyway replaces Liquibase for database migrations.
  The job-service-db is now using Flyway for its versioning and migrations.
• US359648 / US397417: The Job Service can publish V4 messages.
  The Job Service can now send V3 or V4 format messages ( V3 is default ).
  The JOB_SERVICE_MESSAGE_OUTPUT_FORMAT environment variable on the job-service-scheduled-executor can be used to control it.

Breaking Changes

• 353257: The following environment variables are no longer supported:JOB_SERVICE_DATABASE_URL, JOB_SERVICE_DATABASE_HOSTNAME,
  CAF_DATABASE_URL, JOB_DATABASE_USERNAME, JOB_DATABASE_PASSWORD, JOB_DATABASE_APPNAME, CAF_DATABASE_USERNAME,
  CAF_DATABASE_PASSWORD, CAF_DATABASE_APPNAME. The variables now align with our standard such as:
  • JOB_DATABASE_USERNAME and CAF_DATABASE_USERNAME are replaced with JOB_SERVICE_DATABASE_USERNAME
  • JOB_DATABASE_PASSWORD and CAF_DATABASE_PASSWORD are replaced with JOB_SERVICE_DATABASE_PASSWORD
  • JOB_DATABASE_APPNAME and CAF_DATABASE_APPNAME are replaced with JOB_SERVICE_DATABASE_APPNAME
  • JOB_SERVICE_DATABASE_HOSTNAME is replaced with JOB_SERVICE_DATABASE_HOST
  • JOB_SERVICE_DATABASE_URL and CAF_DATABASE_URL are replaced with JOB_SERVICE_DATABASE_HOST, JOB_SERVICE_DATABASE_PORT and JOB_SERVICE_DATABASE_NAME altogether.

Known Issues

• None

v5.0.0

Version Number

5.0.0-4

New Features

• None

Breaking Changes

• SCMOD-13865: propagate_failures option has been removed.
  The job service now propagates all failures and it is no longer possible to disable the propagate failures option. Note that previously it was set to false by default.
• SCMOD-13870: Increase resilience in the case of rabbitMQ outage.
  New env variable to be added to job-service-scheduled-executor: CAF_RABBITMQ_MAX_PRIORITY to be set to 5

Bug Fixes

• SCMOD-15074: Fixed job cancellation and job deletion out of memory issues.

Known Issues

• None

Release Artifacts

Maven

• com.github.jobservice:job-service-html:5.0.0-4 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-db-client:5.0.0-4 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:worker-jobtracking-shared:5.0.0-4 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:job-service:5.0.0-4 (javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar,war.md5,war.sha1,war)
• com.github.jobservice:job-service-ui:5.0.0-4 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-config:5.0.0-4 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:job-service-db:5.0.0-4 (jar-with-dependencies.jar.md5,[jar-wit...

v4.0.0

Version Number

4.0.0-1076

New Features

• SCMOD-12730: Added support for pausing and resuming jobs.
  • Three new endpoints have been added to pause, resume, and get the status of a job.

Breaking Changes

• SCMOD-12505: Various updates to the 'Job Types' functionality.
  • The taskDataScript property should no longer be provided when adding a new job type yaml file. Instead, a taskScript property should be provided. See the Job-Types documentation for more information.
  • The names used for the configurationProperties now have a direct mapping to the environment variables used to populate them. For example, if a configuration property is named TASK_PIPE in a job type yaml definition, then the value for that configuration property is expected to be available in an environment variable named TASK_PIPE.
• SCMOD-12730: Workers built using framework versions prior to 5.0.0 are no longer supported.
  • The statusCheckUrl now points to the status endpoint instead of the isActive endpoint.
  • Instead of returning true or false, the statusCheckUrl will now return one of Active, Cancelled, Completed, Failed, Paused, or Waiting.
  • If the statusCheckUrl refers to a job that does not exist, the response from the endpoint will return a HTTP 404 status. Previously, the statusCheckUrl would have returned a HTTP 200 status with a response body of false when a job did not exist.
  • The CAF_STATUS_CHECK_TIME environment variable has been renamed to CAF_STATUS_CHECK_INTERVAL_SECONDS.
• SCMOD-12730: A new CAF_JOB_SERVICE_RESUME_JOB_QUEUE environment variable is required.
  • This should point to the input queue of the worker that is responsible for dispatching task messages for resumed jobs to the appropriate workers.

Bug Fixes

• SCMOD-10825: Fixed bug that was causing the cancellation of complex jobs to fail.

Known Issues

• None

Release Artifacts

Maven

• com.github.jobservice:job-service-html:4.0.0-1076 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-db-client:4.0.0-1076 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:worker-jobtracking-shared:4.0.0-1076 (jar.md5,jar.sha1,jar,javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar)
• com.github.jobservice:job-service:4.0.0-1076 (javadoc.jar.md5,javadoc.jar.sha1,javadoc.jar,pom.md5,pom.sha1,pom,sources.jar.md5,sources.jar.sha1,sources.jar,war.md5,war.sha1,war)
• com.github.jobservice:job-service-ui:4.0.0-1076 (pom.md5,pom.sha1,pom,war.md5,war.sha1,war)
• com.github.jobservice:job-service-config:4.0.0-1076 (jar.md5,jar.sha1,...

v3.5.0

Version Number

3.5.0-979

New Features

• SCMOD-9697 - Added support to getJobs REST calls to have the ability to sort by any fields.
• SCMOD-9988 - Lyra RabbitMQ client has been replaced by the RabbitMQ Java client.
• SCMOD-10455 - Fix delay between job completion and reporting
  The jobTracking behavior has been adjusted, so the job progress isn't updated on the fly but instead stored into a specific table (completed_subtask_report) that will be used "on demand" to update the job progress.
  Also, a new capability has been introduced to process the task report as a bulk instead of individually only.
  A transient exception has been added to the processing of completed tasks so for any issue, the information is kept and resent to the queue instead of being discarded.
  This decreases the pressure on the Postgres instance and allows better performance while reporting a job progress, specifically in the case of "metadata only" job.
• SCMOD-11144: Upgrade postgresql to 42.2.5
• SCMOD-11919: Implement report_progress
  That new feature allows to report a partially completed task to the database. Before that, a partial completion would be simply logged and only completed or failed tasks would be reported.
• SCMOD-11850: Optimized tests to use conditional wait statement

Bug Fixes

• SCMOD-11149 - Job-Service only checks if target queue of a job exists instead of attempt to create it.
  Job-service previously attempted to create the target queue of a new job. If the queue already exists with different settings, this results in a channel error. The old lyra client would silently handle this and recover the channel while the new client does not.
• SCMOD-10792: Don't delete dependent jobs until they're published
  Before that fix, when calling "get_dependent_jobs()", the corresponding rows in the job_task_data table would be removed as part of the process, then the dependent_jobs list would be sent back. If any exception was occurring in between, this would delete the job_task_data rows and result in the job progress not completing. This change would delete the rows only once the report is sent.
• SCMOD-12164: Add publish confirm and retry message
  The error handling around publishing messages in the job service has now been updated to wait for confirmation from Rabbit that the message was successfully published to the queue.
  A retry has also been added if the publishing fails, and a more helpful error should be logged if this occurs.

Known Issues

• None

Release Artifacts

Docker

• jobservice/job-service-postgres:3.5.0 (Static Tag)
• jobservice/job-service-postgres:3.5 (Dynamic Tag)
• jobservice/job-service-postgres:3 (Dynamic Tag)
• jobservice/job-service-postgres:latest (Dynamic Tag)
• jobservice/job-service-caller:3.5.0 (Static Tag)
• jobservice/job-service-caller:3.5 (Dynamic Tag)
• jobservice/job-service-caller:3 (Dynamic Tag)
• jobservice/job-service-caller:latest (Dynamic Tag)
• jobservice/job-service:3.5.0 (Static Tag)
• jobservice/job-service:3.5 (Dynamic Tag)
• jobservice/job-service:3 (Dynamic Tag)
• jobservice/job-service:latest (Dynamic Tag)
• jobservice/job-service-scheduled-executor:3.5.0 (Static Tag)
• jobservice/job-service-scheduled-executor:3.5 (Dynamic Tag)
• jobservice/job-service-scheduled-executor:3 (Dynamic Tag)
• jobservice/job-service-scheduled-executor:latest (Dynamic Tag)
• jobservice/worker-jobtracking:3.5.0 (Static Tag)
• jobservice/worker-jobtracking:3.5 (Dynamic Tag)
• jobservice/worker-jobtracking:3 (Dynamic Tag)
• jobservice/worker-jobtracking:latest (Dynamic Tag)

v3.4.0

Version Number

3.4.0-834

New Features

• SCMOD-8484: Propagate failures through subtasks
  Functionality has been added that allows for the failure of a task to propagate up through any tasks that are no longer able to run because the task has failed.
• SCMOD-9780: Updated images to use Java 11 image as their base image
• SCMOD-9456: Task tables for subjobs now created base on the identity of the root parent job rather than the short hash, this allows for smaller table names.
• SCMOD-9780: Added job failure roll up functionality. A failure in a prerequisite job can now cause all jobs waiting on it to be set to failure.
• SCMOD-9552: Job failure information updated to contain more detailed failure information, if a job fails due to a failure in a prerequisite job, that job id is listed in the failure source.
• SCMOD-9347: Added filtering on job searching
  Job searches can now supply filter params based on RSQL to filter which jobs should be returned, these filters can operate on labels, job id's, status, job name, last modified and create dates, partition id and percentage complete.

Known Issues

• None

Bug Fixes

• SCMOD-9455: Updated Job Service Contract
  The Job Service contract has been updated to be a true reflection of what is actually returned by the getJob and getJobs api.
  This change means that the createTime and lastUpdatedTime properties of a Job will now be represented by a Long Epoch value instead of a String in date-time format.
• SCMOD-9529: Corrected job sorting
  Functionality was previously broken and was returning jobs in a random order, this is now fixed and the order params for get jobs calls returns jobs in the specified orders.

Repository tidy

• SCMOD-10185: Removed redundant database procedure creation changesets.
• None: Removed thread from logging statement as this is logged as part of the standard CAF logging format.
• None: Moved frequent log statements to debug level as logs were filling up with.

v3.3.0

Version Number

3.3.0-2

New Features

• SCMOD-10244: Added support for suspending jobs

Known Issues

• None