[Windows]: Shorter paths in CI Environment?

[sbixl]

Hi,

is it perhaps possible to use shorter path names while building in a Jenkins CI environment (native Windows)?

It would be best if the path lengths were approximately the same as if I were running a local build. In my example I am trying to build clang natively under windows. Locally it works but as soon as I build it with a Jenkins, I run into the annoying problem of path limiting. The path limitation is unfortunately still a problem in 2025 even though many tools (CMake, Ninja) have done a lot to fix this problem.

The following is an error from the CI that can be traced back to the path limitation:

[2411/4927] Building RC object tools\clang\tools\extra\clang-tidy\misc\ConfusableTable\CMakeFiles\clang-tidy-confusable-chars-gen.dir\0e55121427eb224549f486eaeada315c\windows_version_resource.rc.res FAILED: [code=2] tools/clang/tools/extra/clang-tidy/misc/ConfusableTable/CMakeFiles/clang-tidy-confusable-chars-gen.dir/0e55121427eb224549f486eaeada315c/windows_version_resource.rc.res C:/home/jenkins/bob/33/2c/3DD190~1/WORKSP~1/usr/bin/cmcldeps.exe RC C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\src\llvm\resources\windows_version_resource.rc tools\clang\tools\extra\clang-tidy\misc\ConfusableTable\CMakeFiles\clang-tidy-confusable-chars-gen.dir\0e55121427eb224549f486eaeada315c\windows_version_resource.rc.res.d tools\clang\tools\extra\clang-tidy\misc\ConfusableTable\CMakeFiles\clang-tidy-confusable-chars-gen.dir\0e55121427eb224549f486eaeada315c\windows_version_resource.rc.res "Hinweis: Einlesen der Datei: " "C:/home/jenkins/bob/51/3c/2ef218d04a9d50895a74bf516153b9ffe525-3/workspace/BuildTools/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/cl.exe" C:\home\jenkins\bob\51\3c\2EF218~1\WORKSP~1\WINDOW~1\10\bin\100190~1.0\x64\rc.exe -DGTEST_HAS_RTTI=0 -DLLVM_BUILD_STATIC -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DRC_FILE_VERSION=\"0\" -DRC_INTERNAL_NAME=\"clang-tidy-confusable-chars-gen\" -DRC_PRODUCT_NAME=\"LLVM\" -DRC_PRODUCT_VERSION=\"0\" -DRC_VERSION_FIELD_1=20 -DRC_VERSION_FIELD_2=1 -DRC_VERSION_FIELD_3=8 -DRC_VERSION_FIELD_4=0 -I C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\build\tools\clang\tools\extra\clang-tidy\misc\ConfusableTable -I C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\src\clang-tools-extra\clang-tidy\misc\ConfusableTable -I C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\build\tools\clang\tools\extra\clang-tidy -I C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\src\clang\include -I C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\build\tools\clang\include -I C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\build\include -I C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\src\llvm\include -DWIN32 /nologo /fo tools\clang\tools\extra\clang-tidy\misc\ConfusableTable\CMakeFiles\clang-tidy-confusable-chars-gen.dir\0e55121427eb224549f486eaeada315c\windows_version_resource.rc.res C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\src\llvm\resources\windows_version_resource.rc C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace\src\llvm\resources\windows_version_resource.rc: fatal error C1083: Datei (vom Compiler generiert) kann nicht ge÷ffnet werden: "windows_version_resource.i": No such file or directory

[jkloetzke]

First let's break down the individual items in the path:

C:\J\workspace\devel__llvm__llvm-host-tools\devel\llvm\llvm-host-tools\build\1-712f-4635\workspace
~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ ~~~~~~~~~
      (1)                 (2)                           (3)                 (4)   (5)

1. Jenkins home with standard "workspace" directory. That's already as small as it can get.
2. The job name. This is hard coded at the moment.
3. Path for each Bob package. This can be overridden by a plugin.
4. Index for possibly ambiguous package path. Also hard coded.
5. Canary to handle Jenkins workspace reusage. Hard coded too.

So you can immediately shorten (3). In principle, we could do something about (2) and (5) as well. But I have to admit I have no idea right now.

[sbixl]

I thought about it again. I experimented a little bit and managed to build it. The dirty hack you can find here.

Is it maybe possible to introduce a new property for a recipe (which may only be available while using native Windows) that allows the src and build directory to point to an “external” shortened path via a symlink?

For example, you could define a root directory such as “C:\.bob” and then, when Bob is executed, folders are created in this directory that are assigned a unique hash (maybe we could use the Variant-Id or something similar here).

A symlink can then be created on each of these folders to represent the respective src or build directory.

This would also ensure that the paths are always the same length locally and on Jenkins. Of course, this feature should only be used when there is no other option.

Unfortunately, this problem with long paths will always occur in Windows. We also have a project where Zephy is built via bob. There, we are already pushing the limits in terms of path length.

Here is another similiar problem.

Could that be a solution to the problem, or is it just a dirty hack?

[jkloetzke]

Hmm, pointing to an external directory sounds really dangerous. It must never be accessed concurrently. This is hard to achieve. One way I could imagine is that, whenever the workspace is created, a uique, random directory is created in a shared location. That could then be symlinked.

But there are multiple drawbacks. First of all, the directories will accumulate in the global location and there is no good way to collect the garbage there. Then there is the problem that this collides with the shared packages logic which is already implemented by means of symlinks. I guess this can be solved somehow, though.

I wouldn't say that it's impossible but it certainly is a big effort.

BTW, is this just a Jenkins problem or do you run into the same problems on your local machine as well?