cygwin: add a system builder with a working nix

[corngood]

Summary

This includes:

1. cross-build of nix (done)
2. tarball/installer build (proof of concept)
3. cygwin hosted builds (in progress)

(2) is very useful for actually testing things. You can easily create a tarball with a desired package set that's fairly simple to install and boot. However, I'm not sure it's suitable for the nixpkgs tree. It's quite similar to something like nix-darwin, but much more primitive. It currently uses some existing modules from nixos:

This was quite helpful to set up /etc/ssl, so https works in nix, and didn't (so far) require any changes to those modules. I know nixos kernel/init system abstraction has been discussed quite a bit, but I haven't checked in on it recently (other than vaguely following launchd support). I'd rather not maintain a separate nix-cygwin tree if possible.

This no longer requires a custom nixos eval, and is simply a couple of extra modules you can import: one to build a cygwin system, and one to build a tarball installer.

Building and Installing

On linux (or maybe other systems where nix works), in nixpkgs root:

Make a system config (cygwin-system.nix) with the cygwin modules imported, e.g.:

{
  lib,
  pkgs,
  config,
  ...
}:

{
  imports = [
    ./nixos/maintainers/scripts/cygwin/system.nix
    ./nixos/maintainers/scripts/cygwin/tarball.nix
  ];

  nixpkgs.buildPlatform = lib.mkDefault builtins.currentSystem;
  nixpkgs.hostPlatform = lib.mkDefault "x86_64-cygwin";
}

Run nix build -f nixos config.system.cygwin.tarball -I nixos-config=./cygwin-system.nix in root to get a tarball.

Then on windows

PS> .\tarball\install.ps1 cygwin
PS> .\cygwin\cygwin.cmd

WARNING: currently the install may print an error (about /dev) when run over top of an existing install. This can be treated as success.
WARNING: native windows symlinks may not be created properly on the first install. Running the install twice may be required.

This should boot the system and drop you in a logged in shell.

Using Nix on Cygwin

This branch has many fixes for packages on cygwin, and includes a working native stdenv, so for now you'll have to build using it, or a branch including the same changes.

With flakes you can do this like:

nix build github:corngood/nixpkgs/cygwin-wip#hello

If you build the tarball itself with a flake, you'll benefit from the lib.nixosSystem behaviour of adding the current nixpkgs to the registry as nixpkgs. You can do that with a standalone flake file, or via an abomination like:

nix build --impure --expr '((builtins.getFlake ("git+file://" + toString ./.)).lib.nixosSystem { modules = [ ./cygwin-system.nix ]; }).config.system.cygwin.tarball'

In that system you can use:

nix build nixpkgs#hello

Keep in mind that building a system like that will require copying nixpkgs to the store whenever it changes, so it'll take a bit longer, and result in a bigger image.

Remote Builder

I have cygwin working as a remote builder, but it's a little bit of a pain at the moment.

I manually created a host key in ~/.ssh/id_ed25519, and an empty sshd_config.

/run/current-system/sw/bin/sshd -f sshd_config -h ~/.ssh/id_ed25519 -o StrictModes=no

I also had to ln -s /run/current-system/sw/bin/bash /bin/bash for login to work (there must be a better way to fix that).

That should be enough to SSH in.

To work as a trusted builder I had to also generate a store signing key and enable it on cygwin:

  nix = {
    extraOptions = ''
      secret-key-files = /var/store-key
    '';
  };

And in the linux machine using it as a remote builder:

  nix.settings.trusted-public-keys = [ "[key]" ];
  nix.distributedBuilds = true;
  nix.buildMachines = [
    {
      hostName = "[vm]";
      system = "x86_64-cygwin";
      sshKey = "[ssh key]";
      maxJobs = 4; # arbitrary
    }
  ];

With this I can do e.g. nix build -f. --argstr system x86_64-cygwin hello from the linux machine.

At this point you can also switch to a new system configuration by:

• building config.system.cygwin.toplevel
• copying it to cygwin with nix copy --to
• replacing the symlink /nix/var/nix/profiles/system
• activating it with /nix/var/nix/profiles/system/activate

Questions

1. Should all the package fixes be separate PRs? Yes

TODO

• fix mass rebuilds
• fix symlinks in tarball (sort tar to keep links after targets?)
• fix permission error on /nix-path-registration at login
• error: killing process [x]: No such process
  • this is a known issue with nix on cygwin related to process lifetime, and is generally harmless
• hello builds on cygwin
• nixVersions.git builds on cygwin
• config.system.cygwin.toplevel builds on cygwin
• fix dependence on /bin/bash for sshd
• 'reboot' process for updating cygwin1.dll

Things done

• Built on platform:
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• Tested, as applicable:
  • NixOS tests in nixos/tests.
  • Package tests at passthru.tests.
  • Tests in lib/tests or pkgs/test for functions and "core" functionality.
• Ran nixpkgs-review on this PR. See nixpkgs-review usage.
• Tested basic functionality of all binary files, usually in ./result/bin/.
• Nixpkgs Release Notes
  • Package update: when the change is major or breaking.
• NixOS Release Notes
  • Module addition: when adding a new NixOS module.
  • Module update: when the change is significant.
• Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

---

Add a 👍 reaction to pull requests you find important.

[corngood]

PS> tar -x -f nixos-system-x86_64-cygwin.tar
PS> .\cygwin.cmd

Anyone have any thoughts on automating the install process? We could quite easily drop an install.cmd/ps1 next to the tarball, and continue to depend on the bsdtar that's included with (some versions?) of windows. We might want to enable case sensitive files before unpacking the tar, so having a script would be nice.

However, I don't really like the idea of the installer being two separate loose files. Perhaps there's a good way to embed the tarball in a cmd/ps1 script? Perhaps we could make an installer with mingw?

[RossSmyth]

Yeah tar has been included on Windows since 2019. Could make a SFX archive with 7zip

[corngood]

Could make a SFX archive with 7zip

Maybe worth a look if it can:

• run some custom commands before extracting (e.g. fsutil file setCaseSensitiveInfo)
• create symlinks properly