How can I remove trailing slash from site completely?

[TetraTheta]

I know this would be rather Hugo's problem, but even though many requested or asked about it, Hugo community seems to answer as 'no, we won't'. I see a lot of posts asking how to remove trailing slash from URL, but none of them seem to have usable solution.

First, I tried with NGINX configuration.
In my production server, NGINX config is like this:

server {
  server_name example.com;
  
  root /var/www/html/example.com/;
  index index.html;
  
  location ^~ /.build {
    deny all;
    return 403;
  }
  
  location / {
    try_files $uri $uri/index.html =404;
    rewrite ^/(.*)/$ /$1 permanent;
  }
  
  gzip on;
  gzip_disable "msie6";
  gzip_buffers 32 32k;
  gzip_vary on;
  gzip_comp_level 5;
  gzip_min_length 10240;
  gzip_proxied no-cache no-store private expired auth;
  gzip_types text/plain application/x-javascript text/xml text/css application/xml application/javascript;
}

This rewrites any slash-trailed URL to non-slash-trailed URL, but Google Search Console keep trying to index slash-trailed URL. I think that's because URLs in generated site are still slash-trailed.

So I decided to remove trailing slash from URL too, in generated site files. I created a NPM script to remove trailing slash from the generated site.

import { readFileSync, writeFileSync } from 'fs'
import { parse } from 'node-html-parser'
import glob from 'glob'

function removeTrailingSlash(dirPath) {
  const files = glob.sync(`${dirPath}/**/*.+(html|xml)`)
  const regexXML = /<(?:link|guid|loc)>(https?:\/\/[^<]+)<\/(?:link|guid|loc)>/g

  function rts(url) {
    if (url.length > 1 && url.endsWith('/')) {
      return url.slice(0, -1)
    }
    return url
  }

  files.forEach(file => {
    console.log(file)
    let content = readFileSync(file, 'utf8')
    if (file.endsWith('.html')) {
      const root = parse(content)
      root.getElementsByTagName('meta').forEach(m => {
        if (m.getAttribute('property') == 'og:url') {
          m.setAttribute('content', rts(m.getAttribute('content')))
        }
      })
      root.getElementsByTagName('a').forEach(a => {
        a.setAttribute('href', rts(a.getAttribute('href')))
      })
      content = root.toString()
    } else if (file.endsWith('.xml')) {
      content = content.replace(regexXML, (match, url) => {
        return match.replace(url, rts(url))
      })
    }
    writeFileSync(file, content)
  })
}
removeTrailingSlash('./public')
export { removeTrailingSlash }

In theory, this should remove all trailing slash that can be found on HTML/XML files. But this breaks generated site like this.

At this point, I really lost what to do. What should I do to remove trailing slash from my site, once and for all?

[razonyang]

Hmm, I have no idea on this, can't provide any help. Template methods produce URLs with trailing slash for pages, such as Permalink, RelPermalink, I'm afraid you will need to replace it from all HTML/JSON pages...

I don't think it's worth it to change this.

[TetraTheta]

If I can replace internal function like .Permalink to something like {{ .Permalink | strings.TrimSuffix "/" }}, but this might cause another issue like /blog/posts/article-01/ become /blogpostsarticle-01...

I really don't understand why this kind of request is ignored even though this is even from 2017...

Thanks for the insight. I'm not sure if this still works, but I'll try. This is the last hope I can have for now...

[razonyang]

I won't recommend changing templates, it will be a maintenance nightmare. I think rewrite rules is the simple way to go.

[TetraTheta]

But for some reason, Google Search Engine tries to index slash-trailed version of URL even though NGINX provides non-slash-trailed version.

[razonyang]

Yes, since HTML include the trailing slash URLs, you may need to combine your script and rewrite rules.
But I have no experience with it, couldn't provide help.

[TetraTheta]

Thanks for the help. This is so frustrating and disappointing...

[TetraTheta]

I'll leave this for the record.
I think I've managed to handle trailing slash of URL mostly.

I didn't mark this as answer because there might be an issue regarding of modifying HTML/XML files. At least, it looks like working on my site for now.

First, NGINX should be configured like this:

server {
  location / {
    try_files $uri $uri/index.html $uri/ =404;
    rewrite ^/(.*)/$ /$1 permanent;
  }
}

rewrite part is important because it will rewrite URL with trailing slash to URL without trailing slash.

I wrote remove_slash.mjs to write script for removing trailing slash in generated HTML/XML files.

import { readFileSync, writeFileSync } from 'fs'
import { load } from 'cheerio'
import glob from 'glob'

function removeTrailingSlash(dirPath) {
  const files = glob.sync(`${dirPath}/**/*.+(html|xml)`)
  const regexXML = /<(?:link|guid|loc)>(https?:\/\/[^<]+)<\/(?:link|guid|loc)>/g

  function rts(url) {
    if (url.length > 2 && url.endsWith('/')) {
      return url.slice(0, -1)
    }
    return url
  }

  files.forEach(file => {
    console.log(file)
    let content = readFileSync(file, 'utf8')
    if (file.endsWith('.html')) {
      const $ = load(content)
      $('a').each(function(){
        const href = $(this).attr('href')
        if (href) {
          $(this).attr('href', rts(href))
        }
      })
      $('meta').each(function(){
        const property = $(this).attr('property')
        if (property && property === 'og:url') {
          $(this).attr('content', rts($(this).attr('content')))
        }
      })
      content = $.html()
    } else if (file.endsWith('.xml')) {
      content = content.replace(regexXML, (match, url) => {
        return match.replace(url, rts(url))
      })
    }
    writeFileSync(file, content)
  })
}

I wish I don't have to install additional dependency to my project, but this is a only solution to me.
Parsing HTML content with regex is impossible, so I have to use libraries for parsing HTML content anyway. For XML files, that regex should be enough.

I included that function to my publish script. Right after the site is generated, that function will remove trailing slash from HTML/XML files.

---

But there are some caveats about it.

1. No processing search.json
   I don't know if search.json is verfied or not. I did not process CSS/JS because they are verified with integrity attribute.
2. Increased HTML file size
   cheerio puts " around value of attributes. Due to this behavior, HTML file size will be increased. For example, <a href=https://google.com/> will become <a href="https://google.com">. Since I don't have any objection to this behavior, I leave it as is.

I'll update this whenever I update my configuration. This is so disappointing because this would be easy to do if Hugo allowed non-slash-trailed URL generation globally...