https by default

[lool]

This mainly changes the default Debian mirror for the debootstrap action to use https instead of http, and switches to https in a few more places.

• refactor: Clarify default mirror for mmdebstrap
• feat!: https in default Debian debootstrap mirror
• fix: Use https links where possible

The only remaining file with http:// URLs is the Apache 2.0 license, but it's probably a bad idea to patch it for compliance tooling to work.

[lool]

For mmdebstrap, we could actually override its defaults to have debos guarantee https sources.list in its default configuration – let me know if you want me to send that change.

[lool]

I realize this PR might seem petty, I'm sending this because of the result of an actual cybersecurity assessment of an image based of qcom-deb-images :)

[sjoerdsimons]

The suggested url by deb.debian.org is http, not https ; I'd be curious to know what the assesments is, in the end the signing/validation is done by the gpg checks tracking back to the packages file, not by the network transport.

when building for non-native architectures using https can also cause image builds to be quite a bit slower as the whole encryption is emulated

[lool]

The suggested url by deb.debian.org is http, not https ; I'd be curious to know what the assesments is, in the end the signing/validation is done by the gpg checks tracking back to the packages file, not by the network transport.

When you say the suggested url, you reference the text file hosted on deb.debian.org itself? Pretty much all Debian tools had defaulted to http in the past, it's the only effective way to get caching: with https, apt has to establish an end to end encryption to a hostname with that cert's subject name, so either one has some use some tricks like http://HTTPS/// (apt-cacher-ng) or setup a root CA to do SSL interception.

Yes, the debian archive is securely delivered over http as the indices are GPG signed and there is a timestamp (valid until) to prevent replay attacks.

What was brought up in the security assessment that trigger my proposed changes is a lack of confidentiality: I can observe the software being installed on this or that system. It's minor, but valid, and http traffic will generally trigger alerts.

when building for non-native architectures using https can also cause image builds to be quite a bit slower as the whole encryption is emulated

Yeah, SSL at a large scale is costly for client and server (I guess we don't care too much about Fastly), good point about QEMU.

I guess it's a classical example of security vs practicality: http is generally nice for performance and caching, but will trigger security reviews and leak some data about systems. :-/

One thing we discussed with Robie for qcom-deb-images is to have an easy flag to override the mirror at build time, but keep https in the target images.

[sjoerdsimons]

The suggested url by deb.debian.org is http, not https ; I'd be curious to know what the assesments is, in the end the signing/validation is done by the gpg checks tracking back to the packages file, not by the network transport.

When you say the suggested url, you reference the text file hosted on deb.debian.org itself?

Yeah; Also:

• It's the default for debootstrap: https://salsa.debian.org/installer-team/debootstrap/-/blob/master/debootstrap?ref_type=heads#L53
• and the default for mmdebstrap: https://gitlab.mister-muffin.de/josch/mmdebstrap/src/branch/main/mmdebstrap#L5319
• and the reference manual: https://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_debian_archive_basics

The reason we set a default mirror for debootstrap is mostly historical as some versions would misbehave without it (iirc invalid apt sources list). That's something we could probably drop now so we just use the defaults for mmdebstrap/debootstrap.

Pretty much all Debian tools had defaulted to http in the past, it's the only effective way to get caching: with https, apt has to establish an end to end encryption to a hostname with that cert's subject name, so either one has some use some tricks like http://HTTPS/// (apt-cacher-ng) or setup a root CA to do SSL interception.

Ack; it's also why debos support propagate proxy environment settings as some use that for caching.

Yes, the debian archive is securely delivered over http as the indices are GPG signed and there is a timestamp (valid until) to prevent replay attacks.

What was brought up in the security assessment that trigger my proposed changes is a lack of confidentiality: I can observe the software being installed on this or that system. It's minor, but valid, and http traffic will generally trigger alerts.

when building for non-native architectures using https can also cause image builds to be quite a bit slower as the whole encryption is emulated

Yeah, SSL at a large scale is costly for client and server (I guess we don't care too much about Fastly), good point about QEMU.

I guess it's a classical example of security vs practicality: http is generally nice for performance and caching, but will trigger security reviews and leak some data about systems. :-/

Agreed; This is one of those fuzzy areas and i can see both sides of. Though I'd prefer Debian to take the lead here and debos following the defaults.

One thing we discussed with Robie for qcom-deb-images is to have an easy flag to override the mirror at build time, but keep https in the target images.

That makes total sense!. For this PR, we can merge the non-debootstrap commits. Is not setting a default debootstrap mirror something you'd like to look at?

[obbardc]

following Sjoerd's comments

[obbardc]

Suggested change

	If no mirror is specified debos will use https://deb.debian.org/debian as default.
	If no mirror is specified the default mirror of debootstrap will be used

[obbardc]

use debootstrap default mirror

[obbardc]

but there is also the part where we write /etc/apt/sources.list using this mirror which needs some attention.