Make never stops making

[moonheart08]

...
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/vectors.mk -MG -MP -MQ obj/vectors.o -MQ dep/vectors.mk -o obj/vectors.o src/vectors.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/sgb.mk -MG -MP -MQ obj/sgb.o -MQ dep/sgb.mk -o obj/sgb.o src/sgb.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/misc.mk -MG -MP -MQ obj/misc.o -MQ dep/misc.mk -o obj/misc.o src/misc.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/header.mk -MG -MP -MQ obj/header.o -MQ dep/header.mk -o obj/header.o src/header.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/crash_handler.mk -MG -MP -MQ obj/crash_handler.o -MQ dep/crash_handler.mk -o obj/crash_handler.o src/crash_handler.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/vectors.mk -MG -MP -MQ obj/vectors.o -MQ dep/vectors.mk -o obj/vectors.o src/vectors.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/sgb.mk -MG -MP -MQ obj/sgb.o -MQ dep/sgb.mk -o obj/sgb.o src/sgb.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/misc.mk -MG -MP -MQ obj/misc.o -MQ dep/misc.mk -o obj/misc.o src/misc.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/header.mk -MG -MP -MQ obj/header.o -MQ dep/header.mk -o obj/header.o src/header.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/crash_handler.mk -MG -MP -MQ obj/crash_handler.o -MQ dep/crash_handler.mk -o obj/crash_handler.o src/crash_handler.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/vectors.mk -MG -MP -MQ obj/vectors.o -MQ dep/vectors.mk -o obj/vectors.o src/vectors.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/sgb.mk -MG -MP -MQ obj/sgb.o -MQ dep/sgb.mk -o obj/sgb.o src/sgb.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/misc.mk -MG -MP -MQ obj/misc.o -MQ dep/misc.mk -o obj/misc.o src/misc.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/header.mk -MG -MP -MQ obj/header.o -MQ dep/header.mk -o obj/header.o src/header.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/crash_handler.mk -MG -MP -MQ obj/crash_handler.o -MQ dep/crash_handler.mk -o obj/crash_handler.o src/crash_handler.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/vectors.mk -MG -MP -MQ obj/vectors.o -MQ dep/vectors.mk -o obj/vectors.o src/vectors.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/sgb.mk -MG -MP -MQ obj/sgb.o -MQ dep/sgb.mk -o obj/sgb.o src/sgb.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/misc.mk -MG -MP -MQ obj/misc.o -MQ dep/misc.mk -o obj/misc.o src/misc.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/header.mk -MG -MP -MQ obj/header.o -MQ dep/header.mk -o obj/header.o src/header.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/crash_handler.mk -MG -MP -MQ obj/crash_handler.o -MQ dep/crash_handler.mk -o obj/crash_handler.o src/crash_handler.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/vectors.mk -MG -MP -MQ obj/vectors.o -MQ dep/vectors.mk -o obj/vectors.o src/vectors.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/sgb.mk -MG -MP -MQ obj/sgb.o -MQ dep/sgb.mk -o obj/sgb.o src/sgb.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/misc.mk -MG -MP -MQ obj/misc.o -MQ dep/misc.mk -o obj/misc.o src/misc.asm
rgbasm -p 0xFE -isrc/ -isrc/include/ -Wall -Wextra -h -M dep/header.mk -MG -MP -MQ obj/header.o -MQ dep/header.mk -o obj/header.o src/header.asm
^Cmake: *** Deleting file 'dep/header.mk'

I imagine it's suppost to stop at some point? ^^

[ISSOtm]

This happens when some file needs to be made (INCLUDEed or INCBINed file), but Make doesn't know how to.

Here's what happens:

1. Make invokes RGBASM to generate dependency lists
2. RGBASM tries to INCLUDE or INCBIN file X, which doesn't exist, so it stops there (but outputs it as a dependency)
3. Make sees the dependency lists have been updated, and restarts from the beginning
4. Make sees the missing file, but doesn't know how to make it
5. GOTO 1

[ISSOtm]

(Pinning this because it's a common problem, so others are likely to encounter it)

[ISSOtm]

The problem hasn't gone away, actually, but Make 4.4 introduced a warning that may explain it.

I believe I have a fix, but I'd like someone affected by this problem to help test it. Any volunteers?

[ISSOtm]

gb-starter-kit incorrectly specified its dependencies, which could cause build failures like the above on all OSes. This was fixed in 8d6deab.

Unfortunately, investigation reveals a new, OS-dependent build issue with due to insufficient timestamp resolution.

This only occurs on Windows for all current¹ versions of Make, and on macOS with the default Make 3.81 (I have tested, and using Homebrew's Make 4.4.1 does not present the issue).

Workaround

Disable parallel building on Windows and Make prior to 4.0.
This should slow down the build enough to not trigger this issue.
That didn't work.

Technical details

This issue causes output like:

CreateProcess(C:\Program Files\Git\usr\bin\stat.exe,stat -c "%Y %y" assets/crash_font.1bpp.pb8.size,...)
Live child 0000000002382550 (assets/crash_font.1bpp.pb8.size) PID 37241680 
1699191197 2023-11-05 13:33:17.332802800 +0000
Reaping winning child 0000000002382550 PID 37241680 
stat -c '%Y %y' obj/crash_handler.mk
CreateProcess(C:\Program Files\Git\usr\bin\stat.exe,stat -c "%Y %y" obj/crash_handler.mk,...)
Live child 0000000002382550 (assets/crash_font.1bpp.pb8.size) PID 37241680 
1699191197 2023-11-05 13:33:17.222649000 +0000

  Prerequisite 'assets/crash_font.1bpp.pb8.size' is older than target 'obj/crash_handler.mk'.

Notice that the mtime is higher for the .size file than the .mk, but Make still considers the former to be older than the latter. This appears to be because Make only considers timestamps up to the second (FILE_TIMESTAMP_HI_RES in its source code), at least on Windows, and ties are resolved as "prerequisite is older".

Footnotes

1. I will be asking the Make maintainers about enabling high-resolution timestamps on Windows, so maybe a future version of Make will fix this. ↩

[ISSOtm]

A tentative patch that disabled parallelism on Windows and Make 3.81 did not resolve the issue, it only made it slightly less common. There is a fix for macOS—use a more recent Make—but nothing for Windows.

[Rangi42]

Doesn't look like this will be resolved on Windows any time soon:

https://lists.gnu.org/archive/html/bug-make/2025-02/msg00010.html

It is not that simple. GNU Make uses 'stat' to retrieve the time stamp of files, and the Windows implementation of 'stat' reports times without hi-res data. So to define FILE_TIMESTAMP_HI_RES in the native Windows port of GNU Make, someone will need to produce a reimplementation of 'stat' that returns higher-resolution time stamp.

(The MSYS2 port uses the Cygwin runtime library, whose 'stat' does provide hi-res times. But the native port cannot use the Cygwin runtime, it must use the native runtime or the Win32 APIs.)

Another sharp edge is, the hi-res timer on Windows may appear to move backwards in time as queries are made on different cores. Also see https://stackoverflow.com/q/44020619.

[ISSOtm]

Not quite:

Right. But that's not the hi-res time information I had in mind. I meant to use the 0.1 usec resolution of file times, and the corresponding APIs that return system time with that resolution. Those do not use the hi-res time described in the above SO article, and they don't move back, AFAIK.

@aaaaaa123456789 would know more about the involved Win32 API calls. I would've liked to contribute a patch, but I have no idea where to start.