:farmer: gz_sim_{10,main}_cnlwin gz_physics-ign-physics5-clwin fails investigation

### Reference Builds
- https://build.osrfoundation.org/job/gz_physics-ign-physics5-clwin/100/
- https://build.osrfoundation.org/job/gz_sim-10-cnlwin/52
- https://build.osrfoundation.org/job/gz_sim-10-cnlwin/53
- https://build.osrfoundation.org/job/gz_sim-10-cnlwin/54

### gz_sim-main
Comparing [#101](https://build.osrfoundation.org/job/gz_sim-main-cnlwin/101/) (passing) and [#102](https://build.osrfoundation.org/job/gz_sim-main-cnlwin/102/) (Failing)

<details><summary>Check version differences log</summary>

```diff
- gz-physics version 9.0.0
+ gz-physics version 9.1.0

- Looking for gz-rendering -- found version 10.0.0
+ Looking for gz-rendering -- found version 10.0.1

- Looking for gz-physics -- found version 9.0.0
+ Looking for gz-physics -- found version 9.1.0

- Looking for gz-physics-heightmap -- found version 9.0.0
+ Looking for gz-physics-heightmap -- found version 9.1.0

- Looking for gz-physics-mesh -- found version 9.0.0
+ Looking for gz-physics-mesh -- found version 9.1.0

- Looking for gz-physics-sdf -- found version 9.0.0
+ Looking for gz-physics-sdf -- found version 9.1.0

- Looking for gz-sensors -- found version 10.0.0
+ Looking for gz-sensors -- found version 10.0.1

- Looking for gz-rendering -- found version 10.0.0
+ Looking for gz-rendering -- found version 10.0.1

- Looking for gz-sensors-air_pressure -- found version 10.0.0
+ Looking for gz-sensors-air_pressure -- found version 10.0.1

- Looking for gz-sensors-air_speed -- found version 10.0.0
+ Looking for gz-sensors-air_speed -- found version 10.0.1

- Looking for gz-sensors-altimeter -- found version 10.0.0
+ Looking for gz-sensors-altimeter -- found version 10.0.1

- Looking for gz-sensors-imu -- found version 10.0.0
+ Looking for gz-sensors-imu -- found version 10.0.1

- Looking for gz-sensors-force_torque -- found version 10.0.0
+ Looking for gz-sensors-force_torque -- found version 10.0.1

- Looking for gz-sensors-logical_camera -- found version 10.0.0
+ Looking for gz-sensors-logical_camera -- found version 10.0.1

- Looking for gz-sensors-magnetometer -- found version 10.0.0
+ Looking for gz-sensors-magnetometer -- found version 10.0.1

- Looking for gz-sensors-dvl -- found version 10.0.0
+ Looking for gz-sensors-dvl -- found version 10.0.1

- Looking for gz-sensors -- found version 10.0.0
+ Looking for gz-sensors -- found version 10.0.1

- Looking for gz-rendering -- found version 10.0.0
+ Looking for gz-rendering -- found version 10.0.1

- Looking for gz-sensors-rendering -- found version 10.0.0
+ Looking for gz-sensors-rendering -- found version 10.0.1

- Looking for gz-sensors-lidar -- found version 10.0.0
+ Looking for gz-sensors-lidar -- found version 10.0.1

- Looking for gz-sensors-gpu_lidar -- found version 10.0.0
+ Looking for gz-sensors-gpu_lidar -- found version 10.0.1

- Looking for gz-sensors -- found version 10.0.0
+ Looking for gz-sensors -- found version 10.0.1

- Looking for gz-sensors-camera -- found version 10.0.0
+ Looking for gz-sensors-camera -- found version 10.0.1

- Looking for gz-sensors -- found version 10.0.0
+ Looking for gz-sensors -- found version 10.0.1

- Looking for gz-sensors-boundingbox_camera -- found version 10.0.0
+ Looking for gz-sensors-boundingbox_camera -- found version 10.0.1

- Looking for gz-sensors-segmentation_camera -- found version 10.0.0
+ Looking for gz-sensors-segmentation_camera -- found version 10.0.1

- Looking for gz-sensors-depth_camera -- found version 10.0.0
+ Looking for gz-sensors-depth_camera -- found version 10.0.1

- Looking for gz-sensors -- found version 10.0.0
+ Looking for gz-sensors -- found version 10.0.1

- Looking for gz-sensors-rgbd_camera -- found version 10.0.0
+ Looking for gz-sensors-rgbd_camera -- found version 10.0.1

- Looking for gz-sensors -- found version 10.0.0
+ Looking for gz-sensors -- found version 10.0.1

- Looking for gz-sensors-thermal_camera -- found version 10.0.0
+ Looking for gz-sensors-thermal_camera -- found version 10.0.1

- Looking for gz-sensors -- found version 10.0.0
+ Looking for gz-sensors -- found version 10.0.1

- Looking for gz-sensors-wide_angle_camera -- found version 10.0.0
+ Looking for gz-sensors-wide_angle_camera -- found version 10.0.1
```

</details>

[#102](https://build.osrfoundation.org/job/gz_sim-main-cnlwin/102/) have this additional log that isn't in [#101](https://build.osrfoundation.org/job/gz_sim-main-cnlwin/101/).
```
[2710.032s] [34mcolcon.colcon_core.shell.template[0m [1;30mERROR[0m [31mFileNotFoundError processing template 'C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_powershell\shell\template\hook_prepend_value.ps1.em'[0m
[2710.033s] [34mcolcon.colcon_core.shell[0m [1;30mERROR[0m [31mException in shell extension 'powershell': [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_powershell\\shell\\template\\hook_prepend_value.ps1.em'
Traceback (most recent call last):
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\__init__.py", line 461, in create_environment_hook
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_powershell\shell\powershell.py", line 148, in create_hook_prepend_value
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\__init__.py", line 52, in expand_template
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\pathlib.py", line 1013, in open
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_powershell\\shell\\template\\hook_prepend_value.ps1.em'
[0m
[2710.033s] [34mcolcon.colcon_core.shell.template[0m [1;30mERROR[0m [31mFileNotFoundError processing template 'C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\hook_prepend_value.bat.em'[0m
[2710.034s] [34mcolcon.colcon_core.shell[0m [1;30mERROR[0m [31mException in shell extension 'bat': [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_core\\shell\\template\\hook_prepend_value.bat.em'
Traceback (most recent call last):
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\__init__.py", line 461, in create_environment_hook
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\bat.py", line 125, in create_hook_prepend_value
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\__init__.py", line 52, in expand_template
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\pathlib.py", line 1013, in open
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_core\\shell\\template\\hook_prepend_value.bat.em'
[0m
[2710.034s] [34mcolcon.colcon_core.shell.template[0m [1;30mERROR[0m [31mFileNotFoundError processing template 'C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\hook_prepend_value.dsv.em'[0m
[2710.034s] [34mcolcon.colcon_core.shell[0m [1;30mERROR[0m [31mException in shell extension 'dsv': [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_core\\shell\\template\\hook_prepend_value.dsv.em'
Traceback (most recent call last):
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\__init__.py", line 461, in create_environment_hook
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\dsv.py", line 75, in create_hook_prepend_value
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\__init__.py", line 52, in expand_template
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\pathlib.py", line 1013, in open
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_core\\shell\\template\\hook_prepend_value.dsv.em'
[0m
```
Regarding this cottsay said:
> The messages about colcon, though they aren't causing problems, are an indication of problems related to Python's search paths.
In short, importlib.metadata found the colcon-notification package metadata somewhere, but when colcon tries to import from the module accompanying it, Python can't find it. It's possible that part of colcon was installed outside of Pixi.

And jrivero:
> Thanks for the insight, indeed it can be perfectly possible that colcon is being installed in the host system and in Pixi we are using colcon-common-extensions.


Checking more of the log Output maybe this is the issue, because this dind't happened in 
```
  'C:\Windows\Temp\pixi\project\.pixi\envs\default\Library\bin\cmake.exe' is not recognized as an internal or external command, operable program or batch file.
```

### gz_sim_10
Comparing [#54](https://build.osrfoundation.org/job/gz_sim-10-cnlwin/54/) (failing) and [#55](https://build.osrfoundation.org/job/gz_sim-10-cnlwin/54/) (passing)

<details><summary>Check version differences log</summary>

```
- gz-rendering version 10.0.0
+ gz-rendering version 10.0.1

- Looking for gz-common -- found version 7.0.0
+ Looking for gz-common -- found version 7.0.1

- Looking for gz-common-graphics -- found version 7.0.0
+ Looking for gz-common-graphics -- found version 7.0.1

- Looking for gz-common-events -- found version 7.0.0
+ Looking for gz-common-events -- found version 7.0.1

- Looking for gz-common-geospatial -- found version 7.0.0
+ Looking for gz-common-geospatial -- found version 7.0.1

- Looking for gz-common -- found version 7.0.0
+ Looking for gz-common -- found version 7.0.1

- sdformat version 16.0.0
+ sdformat version 16.0.1
```
</details>

Checking the failing build [#54](https://build.osrfoundation.org/job/gz_sim-10-cnlwin/54/)  I also found the additional log
```
[1380.335s] [34mcolcon.colcon_core.shell.template[0m [1;30mERROR[0m [31mFileNotFoundError processing template 'C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_powershell\shell\template\hook_prepend_value.ps1.em'[0m
[1380.336s] [34mcolcon.colcon_core.shell[0m [1;30mERROR[0m [31mException in shell extension 'powershell': [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_powershell\\shell\\template\\hook_prepend_value.ps1.em'
Traceback (most recent call last):
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\__init__.py", line 461, in create_environment_hook
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_powershell\shell\powershell.py", line 148, in create_hook_prepend_value
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\__init__.py", line 52, in expand_template
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\pathlib.py", line 1013, in open
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_powershell\\shell\\template\\hook_prepend_value.ps1.em'
[0m
[1380.336s] [34mcolcon.colcon_core.shell.template[0m [1;30mERROR[0m [31mFileNotFoundError processing template 'C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\hook_prepend_value.bat.em'[0m
[1380.337s] [34mcolcon.colcon_core.shell[0m [1;30mERROR[0m [31mException in shell extension 'bat': [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_core\\shell\\template\\hook_prepend_value.bat.em'
Traceback (most recent call last):
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\__init__.py", line 461, in create_environment_hook
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\bat.py", line 125, in create_hook_prepend_value
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\__init__.py", line 52, in expand_template
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\pathlib.py", line 1013, in open
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_core\\shell\\template\\hook_prepend_value.bat.em'
[0m
[1380.337s] [34mcolcon.colcon_core.shell.template[0m [1;30mERROR[0m [31mFileNotFoundError processing template 'C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\hook_prepend_value.dsv.em'[0m
[1380.337s] [34mcolcon.colcon_core.shell[0m [1;30mERROR[0m [31mException in shell extension 'dsv': [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_core\\shell\\template\\hook_prepend_value.dsv.em'
Traceback (most recent call last):
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\__init__.py", line 461, in create_environment_hook
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\dsv.py", line 75, in create_hook_prepend_value
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\site-packages\colcon_core\shell\template\__init__.py", line 52, in expand_template
  File "C:\Windows\Temp\pixi\project\.pixi\envs\default\Lib\pathlib.py", line 1013, in open
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Windows\\Temp\\pixi\\project\\.pixi\\envs\\default\\Lib\\site-packages\\colcon_core\\shell\\template\\hook_prepend_value.dsv.em'
[0m
```

And the same error
```
'C:\Windows\Temp\pixi\project\.pixi\envs\default\Library\bin\cmake.exe' is not recognized as an internal or external command, operable program or batch file.

c:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(145,5): error MSB3073: The command "setlocal [C:\J\workspace\gz_sim-10-cnlwin\ws\build\gz-sim\test\integration\INTEGRATION_triggered_publisher.vcxproj]
```

Something interesting of this comparisson is that build [#55](https://build.osrfoundation.org/job/gz_sim-10-cnlwin/54/) passed, and one of the changes is in this pr `https://github.com/gazebosim/gz-sim/commit/a5e3144c69742fc7c05540d5bc50cb77b9af68aa` **maybe** this was the change that fixed it

```diff
# Workaround for bug in rules_foreign_cc when linking libzip.
# Similar to https://github.com/bazel-contrib/rules_foreign_cc/issues/1164
build --dynamic_mode=off
+
+ # OS specific settings
+ build --enable_platform_specific_config
+ build:macos --macos_minimum_os=10.15
```

Scot said:
> Context for discussion: https://github.com/colcon/colcon-core/blob/aedb99db582894b5b90b718f0eeecd463d3de1fc/setup.cfg#L156-L161

### My guess
After checking these four jobs it looks that this isn't a package versions problem, instead it looks that this is related to `cmake` configuration, because pixi environment isn't finding it. Another thing to add is that scpeters and jrivero could be right, colcon warning isn't the thing that is breaking the build, but also it can be related as this only pops up in the builds that fails, so I recommend to check cottsay comment.

Zulip thread: [#Gazebo General > gz_sim10-cnlwin jobs failing by a missing colcon module](https://openrobotics.zulipchat.com/#narrow/channel/526040-Gazebo-General/topic/gz_sim10-cnlwin.20jobs.20failing.20by.20a.20missing.20colcon.20module/with/569099753)

CC: @Crola1702 

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🧑‍🌾 gz_sim_{10,main}_cnlwin gz_physics-ign-physics5-clwin fails investigation #1442

Reference Builds

gz_sim-main

gz_sim_10

My guess

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

🧑‍🌾 gz_sim_{10,main}_cnlwin gz_physics-ign-physics5-clwin fails investigation #1442

Description

Reference Builds

gz_sim-main

gz_sim_10

My guess

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions