[Ruby+Docker] Fix production environment issues

[p3rcypj]

📌 References

• Issue: https://app.clickup.com/t/86997w83w

📝 Implementation

The Django production environment was unable to deploy successfully on either the staging or production machines. Several issues were preventing production deployment. Some of these included:

• Deprecated methods that are no longer available (skip_before_filter → skip_before_action)
• Unused gems unsupported by the new Ruby version (gem gsl)
• Previously built-in Rails functionalities that have been moved to standalone gems (gem net-ftp)
• A new mandatory storage configuration from a dependency (ActiveStorage, even if it is not used)
• Static files not being served → Added Nginx to serve static files
• Digest configuration was disabled in the production.rb environment
• Missing assets from the asset pipeline (not correctly referenced in the views – digests not used consistently)
• Missing schemas in the database (from IEDB bionotes.epitope)
• Rails errors in the _footer.html.haml layout, causing several routes to return HTTP 500 errors

After resolving these issues, some route requests were still failing, resulting in HTTP 404 and HTTP 500 errors. This required further troubleshooting, often by "pulling the thread." Most of these issues were related to:

• Docker network problems with the previous Docker Compose configuration → temporarily resolved with a "patch" (not an ideal solution)
• Database checks for UniProt accession IDs (including ENSEMBL mappings – related to genomicIFrame, aka "GeneViewer")
• SVG icons not being served correctly (missing entries in mime.types)
• Missing redirections to BWS and LRS
• Incorrect self-referencing of Rails web service endpoints
• Inconsistencies in external database requests, including the use of repeated slashes (//) and the lack of HTTPS support
• Issues with mandatory folders in the local environment

Some improvements have also been made:

• Optimized the Dockerfile build process to reduce build times (via caching in earlier steps for expensive operations) when Rails code changes
• Reduced build context using .dockerignore
• Refined and reconsidered environment variables
• Enabled dynamic requests for BWS and LRS when needed
• Enabled the use of external BWS and LRS for local development
• Refined Docker Compose configurations
• Increased timeout settings for Rails web service requests in the viewer

🔥 Testing

For local deployment, neither BWS nor LRS is required anymore. Instead, requests will be redirected to https://3dbionotes.cnb.csic.es.

If redirection to local BWS or LRS is needed for development purposes, copying the Nginx service from docker-compose.staging.yml to docker-compose.development.yml and setting BWS_API_PORT and LRS_API_PORT (as referenced in the .env.sample file) will allow redirection to local services.

If only one of the two is needed, modifying ./nginx/nginx.conf will be necessary, but ./nginx/nginx.dev.conf can be used as a reference to delegate the unused service to https://3dbionotes.cnb.csic.es.

[p3rcypj]

Some end-of-file (EOF) changes occurred because the default code formatter was not installed on the machine I was using to modify the files.

[p3rcypj]

Docker Compose configurations and environment variables may change in the coming weeks as our new needs become clearer.

[adrianq]

@tokland I merged this PR because there is an intern working on something related but when you are back please take some time to review it. @p3rcypj will create a new PR for the review if needed

[tokland]

In-line comments:

[tokland]

Was this change about the timeout intended for this infrastructure PR? In any case, for that kind of situations where we repeatedly call a refined version of the function, we can create an in-line abstraction:

const getJSON2 = (url: string) => getJSON(url, { timeout: 100e3 });

(if you can find a better name than getJSON2...)

[tokland]

minor, but lately the consensus is to write nmrTargets: nmrTargets so we can easily navigate to the type OR the value.

[tokland]

A more resilient solution is to use a function that performs URI joining, such as join_url(BaseUrl, "/api/mappings/Uniprot/ENSEMBL/transcript", uniprot_acc).

This join_url function would be defined in a global module (which would use URI.join, for example).

[tokland]

Minor, but this Rails filters have a "if" param, right? skip_before_action :verify_authenticity_token, only: [:home], if: -> { Rails.env.development? }

[tokland]

As 1) this action is performed twice, and 2) we have to inspect the code to see what's doing -> create an abstraction with a declarative name

[tokland]

[format] I think Ruby formatters prefer not to add () for this kind of expressions

[tokland]

Same idea that URIs: abstraction or direct File.join

[tokland]

That code is not ours, so let's keep it as-is to minimize the diffs, but for future code like this, a wiser usage of the quote character and mult-iline joined strings are more readable. For example:

cmd = [
  'echo "\n" | water',
  "-asequence #{LocalPath}#{rand}_can_seq",
  "-bsequence #{LocalPath}#{rand}_trans_seq",
  "-gapopen 50",
  "-gapextend 0",
  "-datafile #{IdentityMatrix}",
  "-aformat3 markx10",
  "-stdout",
  "-aglobal3 Y",
  "-awidth3 1000000",
  "2> /dev/null"
].join(' ')

[tokland]

If it's deprecated, we need to insert the code? if it's something we really need to document, we can write a comment as text. Commented-out code always raises questions to the reader (is this temporal? will we need to uncomment it at some point?)

[tokland]

The maybe be explicit and set to true? or it's not the same?