`expires` and `cache-control` headers

[merlinnot]

I'm working on implementing signed exchanges for a website and I have a question and maybe a potential feature requests. Most of the HTML I'm serving has an expiration date of just a few days, however a lot of static content (hashed) uses 365 days. Signed exchanges can be generated for both, however in front of my Nginx servers I also have a load balancer with CDN capabilities. If it starts caching requests for signatures, then these will be stale. I couldn't find a way to differentiate expires and cache-control headers based on the fact if it's an SGX request or not. Should the module automatically set cache-control and expires to a reasonable value?

To make it more clear, that's my setup:

• Nginx servers in 12 geographical regions worldwide
• Google Cloud HTTPS load balancer
• Google Cloud CDN

Requests made from AMP caches would also end up on the load balancer and possibly be served by the CDN, unless there's a way I can prevent that from happening.

[twifkak]

This sounds like a good feature request, for the module to automatically set cache-control to a reasonable default. Perhaps public, max-age=86400. Also, a configuration param, because there is a trade-off between CPU usage (generating new signatures) and LCP (signatures that expire soon will have less utility in SXG caches, thus lower hit rate), and we should leave it up to site owners to decide where to sit on that curve.

In the meantime you should be able to hack the cache-control in your nginx config, something like:

if ($http_accept ~* "(^|,)\s*application/signed-exchange\s*;\s*v=b3\s*(,|$)") {
   expires 1d;
}

This emulates the module's logic:

nginx-sxg-module/ngx_http_sxg_filter_module.c

Lines 246 to 247 in 663d652

	highest_qvalue_is_sxg((const char*)table[i].value.data,
	table[i].value.len)) {

with a regex, based on this webpackager suggestion.

---

Restating the problem to make sure I understand:

Right now the module sends SXGs with no explicit cache headers besides Vary: Accept:

nginx-sxg-module/ngx_http_sxg_filter_module.c

Lines 841 to 870 in 663d652

	// Cleanup outer headers and trailers.
	ngx_memzero(&req->headers_out, sizeof(ngx_http_headers_out_t));
	if (ngx_list_init(&req->headers_out.headers, req->pool, 2,
	sizeof(ngx_table_elt_t)) != NGX_OK) {
	ngx_log_error(NGX_LOG_ERR, req->connection->log, 0,
	"nginx-sxg-module: failed to initialize outer header list");
	return NGX_ERROR;
	}
	if (ngx_list_init(&req->headers_out.trailers, req->pool, 1,
	sizeof(ngx_table_elt_t)) != NGX_OK) {
	ngx_log_error(NGX_LOG_ERR, req->connection->log, 0,
	"nginx-sxg-module: failed to initialize trailers list");
	return NGX_ERROR;
	}
	// Append nosniff to the new outer header entry.
	ngx_table_elt_t* x_content_type_options =
	ngx_list_push(&req->headers_out.headers);
	x_content_type_options->hash = 1;
	ngx_str_set(&x_content_type_options->key, "X-Content-Type-Options");
	ngx_str_set(&x_content_type_options->value, "nosniff");
	ngx_table_elt_t* vary = ngx_list_push(&req->headers_out.headers);
	vary->hash = 1;
	ngx_str_set(&vary->key, "Vary");
	ngx_str_set(&vary->value, "Accept");
	// Modify out of list outer header entries.
	ngx_str_set(&req->headers_out.content_type,
	"application/signed-exchange;v=b3");
	req->headers_out.content_type_len = req->headers_out.content_type.len;

So it's up to intermediary caches to determine lifetime. RFC7234 provides only vague guidance, so there's a lot of freedom. However, it's made more difficult by the fact that SXG and unsigned HTML are served at the same URL. I don't know a lot about Google Cloud CDN, but I guess you would need FORCE_CACHE_ALL for SXG-preferring Accept header values and USE_ORIGIN_HEADERS for the others. I'm guessing that's not possible.

So you need to set USE_ORIGIN_HEADERS, and have the origin report 365d expiration for unsigned and 1d expiration for signed.

[twifkak]

Also, if you're wanting to serve AMP SXGs, I recommend AMP Packager if possible. You can do it with nginx-sxg-module, but you'll need logic to parse the AMP-Cache-Transform header (at least approximately), run the transforms (could happen in a build step), and possibly some other minor things to meet Google AMP SXG Cache requirements.

[merlinnot]

Thanks for the detailed answers!

I'll add and test the if statement, that's a great tip.

Google Cloud CDN respects Vary: Accept and Vary: Accept-Encoding:

• https://cloud.google.com/cdn/docs/caching#vary-headers
• https://cloud.google.com/cdn/docs/caching#non-cacheable_content

But I'm not sure if it respects Vary: Accept, Accept-Encoding. I'm gonna need both, one for exchanges, one for gzip/brotli. For the time being I'm gonna deploy it without a CDN and look into that next week.

I'll also look deeper into the requirements of the AMP cache (the header you mentioned). I'd prefer to add a build step and keep using this extension, I put a lot of effort into Nginx already and redeploying every 90 days (max certificate lifetime) seems better than redeploying every 7 days (or more often, as far as I remember max lifetime for signatures is 7 days).

[twifkak]

Restating the feature request, since it got lost in my long reply:

1. Set the outer cache-control header on a SXG to be public, max-age=86400 by default -- for SXG responses, not HTML responses.
2. Introduce a new sxg_ parameter that overrides this value, only for SXG responses.