firefox times out in Github Actions

[mgred]

I've an Angular app closly modeled after the example in rules_nodejs that uses rules_webtesting for unit tests. Everything works fine locally, with Firefox and Chrome enabled. When I run the same tests in a Github Action Workflow (bazel test //...), the firefox-based test suites time out complaining: FirefoxHeadless has not captured in 60000 ms, killing. (full error log attached).

Does anybody know about this problem or has a clue what might go wrong here? I'm thankfull for any advice 🙏

Error Log

2021/12/08 18:38:44 Downloading https://releases.bazel.build/4.1.0/release/bazel-4.1.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 7c70f3aa-e82b-42b0-a492-729a0bc9315e
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 5 packages loaded
    currently loading: api/metadata/src ... (13 packages)
Loading: 5 packages loaded
    currently loading: api/metadata/src ... (13 packages)
Loading: 5 packages loaded
    currently loading: api/metadata/src ... (13 packages)
Loading: 5 packages loaded
    currently loading: api/metadata/src ... (13 packages)
Loading: 5 packages loaded
    currently loading: api/metadata/src ... (13 packages)
Loading: 7 packages loaded
    currently loading: api/metadata/src ... (11 packages)
Analyzing: 78 targets (18 packages loaded, 0 targets configured)
Analyzing: 78 targets (287 packages loaded, 14670 targets configured)
Analyzing: 78 targets (396 packages loaded, 15854 targets configured)
INFO: Analyzed 78 targets (1134 packages loaded, 53038 targets configured).
INFO: Found 71 targets and 7 test targets...
[0 / 2] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[62 / 71] Compiling TypeScript project //cdk/lib:lib [tsc -p cdk/tsconfig.json]; 8s remote-cache, linux-sandbox ... (3 actions running)
[138 / 147] Compiling TypeScript project //webapp/src/app:app [tsc -p webapp/tsconfig.json]; 3s remote-cache, linux-sandbox ... (3 actions, 2 running)
INFO: From Action webapp/src/index.html:
html-insert-assets: Failed to find index.dev.js to stamp. Will fallback to timestamp.
INFO: From Action webapp/src/_prodapp/webapp/src/index.html:
html-insert-assets: Failed to find index.prod.js to stamp. Will fallback to timestamp.
WARNING: /home/runner/work/data-catalog/data-catalog/webapp/src/BUILD.bazel:128:12: output 'webapp/src/pwa-config' of //webapp/src:pwa is a directory; dependency checking of directories is unsound
[252 / 257] 1 / 7 tests; Splitting Javascript webapp/src/initialize_testbed.js webapp/src/app/table/table.component.spec.js [esbuild]; 1s remote-cache, linux-sandbox ... (3 actions, 2 running)
[280 / 282] 2 / 7 tests; Testing //webapp/src/app:test.suite_firefox-local; 27s remote-cache, linux-sandbox ... (4 actions running)
[280 / 282] 2 / 7 tests; Testing //webapp/src/app:test.suite_firefox-local; 55s remote-cache, linux-sandbox ... (4 actions running)
[280 / 282] 2 / 7 tests; Testing //webapp/src/app:test.suite_firefox-local; 102s remote-cache, linux-sandbox ... (4 actions running)
[280 / 282] 2 / 7 tests; Testing //webapp/src/app:test.suite_firefox-local; 156s remote-cache, linux-sandbox ... (4 actions running)
FAIL: //webapp/src/app:test.suite_firefox-local (see /home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.log)
INFO: From Testing //webapp/src/app:test.suite_firefox-local:
==================== Test output for //webapp/src/app:test.suite_firefox-local:
TEST_UNUSED_RUNFILES_LOG_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.unused_runfiles_log
TEST_BINARY=webapp/src/app/test.suite_firefox-local.sh
TEST_TEMPDIR=test_tempdir.71sER1
TEST_WORKSPACE=data_catalog
RUNFILES_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/test.suite_firefox-local.sh.runfiles
TEST_UNDECLARED_OUTPUTS_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.outputs
***
RUN_UNDER_RUNFILES=1
TZ=UTC
TEST_TMPDIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/_tmp/f0c8d0e5f3a091cd44814ad9179d3537
TMPDIR=/tmp
XML_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.xml
WEB_TEST_METADATA=data_catalog/webapp/src/app/test.suite_firefox-local.gen.json
TEST_UNDECLARED_OUTPUTS_ANNOTATIONS_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.outputs_manifest
TEST_PREMATURE_EXIT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.exited_prematurely
TEST_INFRASTRUCTURE_FAILURE_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.infrastructure_failure
USER=runner
PYTHON_RUNFILES=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/test.suite_firefox-local.sh.runfiles
TEST_SRCDIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/test.suite_firefox-local.sh.runfiles
SHLVL=1
_RLOCATION_ISABS_PATTERN=^/[^/].*
PATH=.:/bin:/usr/bin:/usr/local/bin
GTEST_TMP_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/_tmp/f0c8d0e5f3a091cd44814ad9179d3537
TEST_WARNINGS_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.warnings
TEST_TARGET=//webapp/src/app:test.suite_firefox-local
JAVA_RUNFILES=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/test.suite_firefox-local.sh.runfiles
TEST_SIZE=large
EXPERIMENTAL_SPLIT_XML_GENERATION=1
TEST_TIMEOUT=900
TEST_LOGSPLITTER_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.raw_splitlogs/test.splitlogs
OLDPWD=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/52/execroot/data_catalog
BASH_FUNC_rlocation%%=() {  if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): start" 1>&2;
 fi;
 if [[ "$1" =~ $_RLOCATION_ISABS_PATTERN ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): absolute path, return" 1>&2;
 fi;
 echo "$1";
 else
 if [[ "$1" == ../* || "$1" == */.. || "$1" == ./* || "$1" == */./* || "$1" == "*/." || "$1" == *//* ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: rlocation($1): path is not normalized" 1>&2;
 fi;
 return 1;
 else
 if [[ "$1" == \\* ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: rlocation($1): absolute path without" "drive name" 1>&2;
 fi;
 return 1;
 else
 if [[ -e "${RUNFILES_DIR:-/dev/null}/$1" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): found under RUNFILES_DIR ($RUNFILES_DIR), return" 1>&2;
 fi;
 echo "${RUNFILES_DIR}/$1";
 else
 if [[ -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): looking in RUNFILES_MANIFEST_FILE ($RUNFILES_MANIFEST_FILE)" 1>&2;
 fi;
 local -r result=$(grep -m1 "^$1 " "${RUNFILES_MANIFEST_FILE}" | cut -d ' ' -f 2-);
 if [[ -e "${result:-/dev/null}" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): found in manifest as ($result)" 1>&2;
 fi;
 echo "$result";
 else
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): not found in manifest" 1>&2;
 fi;
 echo "";
 fi;
 else
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: cannot look up runfile \"$1\" " "(RUNFILES_DIR=\"${RUNFILES_DIR:-}\"," "RUNFILES_MANIFEST_FILE=\"${RUNFILES_MANIFEST_FILE:-}\")" 1>&2;
 fi;
 return 1;
 fi;
 fi;
 fi;
 fi;
 fi
}
BASH_FUNC_runfiles_export_envvars%%=() {  if [[ ! -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" && ! -d "${RUNFILES_DIR:-/dev/null}" ]]; then
 return 1;
 fi;
 if [[ ! -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" ]]; then
 if [[ -f "$RUNFILES_DIR/MANIFEST" ]]; then
 export RUNFILES_MANIFEST_FILE="$RUNFILES_DIR/MANIFEST";
 else
 if [[ -f "${RUNFILES_DIR}_manifest" ]]; then
 export RUNFILES_MANIFEST_FILE="${RUNFILES_DIR}_manifest";
 else
 export RUNFILES_MANIFEST_FILE=;
 fi;
 fi;
 else
 if [[ ! -d "${RUNFILES_DIR:-/dev/null}" ]]; then
 if [[ "$RUNFILES_MANIFEST_FILE" == */MANIFEST && -d "${RUNFILES_MANIFEST_FILE%/MANIFEST}" ]]; then
 export RUNFILES_DIR="${RUNFILES_MANIFEST_FILE%/MANIFEST}";
 export JAVA_RUNFILES="$RUNFILES_DIR";
 else
 if [[ "$RUNFILES_MANIFEST_FILE" == *_manifest && -d "${RUNFILES_MANIFEST_FILE%_manifest}" ]]; then
 export RUNFILES_DIR="${RUNFILES_MANIFEST_FILE%_manifest}";
 export JAVA_RUNFILES="$RUNFILES_DIR";
 else
 export RUNFILES_DIR=;
 fi;
 fi;
 fi;
 fi
}
BASH_FUNC_is_absolute%%=() {  [[ "$1" = /* ]] || [[ "$1" =~ ^[a-zA-Z]:[/\\].* ]]
}
_=/bin/printenv
08 12 2021 18:41:23.101:INFO [karma-server]: Karma v6.3.9 server started at http://localhost:9877/
08 12 2021 18:41:23.103:INFO [launcher]: Launching browsers FirefoxHeadless with concurrency unlimited
08 12 2021 18:41:23.111:INFO [launcher]: Starting browser FirefoxHeadless
08 12 2021 18:42:23.131:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:42:23.140:INFO [launcher]: Trying to start FirefoxHeadless again (1/2).
08 12 2021 18:43:23.171:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:43:23.177:INFO [launcher]: Trying to start FirefoxHeadless again (2/2).
08 12 2021 18:44:23.212:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:44:23.217:ERROR [launcher]: FirefoxHeadless failed 2 times (timeout). Giving up.
================================================================================
FAIL: //webapp/src/app/table:test.suite_firefox-local (see /home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.log)
INFO: From Testing //webapp/src/app/table:test.suite_firefox-local:
==================== Test output for //webapp/src/app/table:test.suite_firefox-local:
TEST_UNUSED_RUNFILES_LOG_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.unused_runfiles_log
TEST_BINARY=webapp/src/app/table/test.suite_firefox-local.sh
TEST_TEMPDIR=test_tempdir.ELgo0L
TEST_WORKSPACE=data_catalog
RUNFILES_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/table/test.suite_firefox-local.sh.runfiles
TEST_UNDECLARED_OUTPUTS_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.outputs
***
RUN_UNDER_RUNFILES=1
TZ=UTC
TEST_TMPDIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/_tmp/ec208004adbce0f871eddff7fc00a3c6
TMPDIR=/tmp
XML_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.xml
WEB_TEST_METADATA=data_catalog/webapp/src/app/table/test.suite_firefox-local.gen.json
TEST_UNDECLARED_OUTPUTS_ANNOTATIONS_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.outputs_manifest
TEST_PREMATURE_EXIT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.exited_prematurely
TEST_INFRASTRUCTURE_FAILURE_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.infrastructure_failure
USER=runner
PYTHON_RUNFILES=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/table/test.suite_firefox-local.sh.runfiles
TEST_SRCDIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/table/test.suite_firefox-local.sh.runfiles
SHLVL=1
_RLOCATION_ISABS_PATTERN=^/[^/].*
PATH=.:/bin:/usr/bin:/usr/local/bin
GTEST_TMP_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/_tmp/ec208004adbce0f871eddff7fc00a3c6
TEST_WARNINGS_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.warnings
TEST_TARGET=//webapp/src/app/table:test.suite_firefox-local
JAVA_RUNFILES=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/table/test.suite_firefox-local.sh.runfiles
TEST_SIZE=large
EXPERIMENTAL_SPLIT_XML_GENERATION=1
TEST_TIMEOUT=900
TEST_LOGSPLITTER_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.raw_splitlogs/test.splitlogs
OLDPWD=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/54/execroot/data_catalog
BASH_FUNC_rlocation%%=() {  if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): start" 1>&2;
 fi;
 if [[ "$1" =~ $_RLOCATION_ISABS_PATTERN ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): absolute path, return" 1>&2;
 fi;
 echo "$1";
 else
 if [[ "$1" == ../* || "$1" == */.. || "$1" == ./* || "$1" == */./* || "$1" == "*/." || "$1" == *//* ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: rlocation($1): path is not normalized" 1>&2;
 fi;
 return 1;
 else
 if [[ "$1" == \\* ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: rlocation($1): absolute path without" "drive name" 1>&2;
 fi;
 return 1;
 else
 if [[ -e "${RUNFILES_DIR:-/dev/null}/$1" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): found under RUNFILES_DIR ($RUNFILES_DIR), return" 1>&2;
 fi;
 echo "${RUNFILES_DIR}/$1";
 else
 if [[ -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): looking in RUNFILES_MANIFEST_FILE ($RUNFILES_MANIFEST_FILE)" 1>&2;
 fi;
 local -r result=$(grep -m1 "^$1 " "${RUNFILES_MANIFEST_FILE}" | cut -d ' ' -f 2-);
 if [[ -e "${result:-/dev/null}" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): found in manifest as ($result)" 1>&2;
 fi;
 echo "$result";
 else
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): not found in manifest" 1>&2;
 fi;
 echo "";
 fi;
 else
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: cannot look up runfile \"$1\" " "(RUNFILES_DIR=\"${RUNFILES_DIR:-}\"," "RUNFILES_MANIFEST_FILE=\"${RUNFILES_MANIFEST_FILE:-}\")" 1>&2;
 fi;
 return 1;
 fi;
 fi;
 fi;
 fi;
 fi
}
BASH_FUNC_runfiles_export_envvars%%=() {  if [[ ! -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" && ! -d "${RUNFILES_DIR:-/dev/null}" ]]; then
 return 1;
 fi;
 if [[ ! -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" ]]; then
 if [[ -f "$RUNFILES_DIR/MANIFEST" ]]; then
 export RUNFILES_MANIFEST_FILE="$RUNFILES_DIR/MANIFEST";
 else
 if [[ -f "${RUNFILES_DIR}_manifest" ]]; then
 export RUNFILES_MANIFEST_FILE="${RUNFILES_DIR}_manifest";
 else
 export RUNFILES_MANIFEST_FILE=;
 fi;
 fi;
 else
 if [[ ! -d "${RUNFILES_DIR:-/dev/null}" ]]; then
 if [[ "$RUNFILES_MANIFEST_FILE" == */MANIFEST && -d "${RUNFILES_MANIFEST_FILE%/MANIFEST}" ]]; then
 export RUNFILES_DIR="${RUNFILES_MANIFEST_FILE%/MANIFEST}";
 export JAVA_RUNFILES="$RUNFILES_DIR";
 else
 if [[ "$RUNFILES_MANIFEST_FILE" == *_manifest && -d "${RUNFILES_MANIFEST_FILE%_manifest}" ]]; then
 export RUNFILES_DIR="${RUNFILES_MANIFEST_FILE%_manifest}";
 export JAVA_RUNFILES="$RUNFILES_DIR";
 else
 export RUNFILES_DIR=;
 fi;
 fi;
 fi;
 fi
}
BASH_FUNC_is_absolute%%=() {  [[ "$1" = /* ]] || [[ "$1" =~ ^[a-zA-Z]:[/\\].* ]]
}
_=/bin/printenv
08 12 2021 18:41:24.525:INFO [karma-server]: Karma v6.3.9 server started at http://localhost:9876/
08 12 2021 18:41:24.527:INFO [launcher]: Launching browsers FirefoxHeadless with concurrency unlimited
08 12 2021 18:41:24.530:INFO [launcher]: Starting browser FirefoxHeadless
08 12 2021 18:42:24.580:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:42:24.588:INFO [launcher]: Trying to start FirefoxHeadless again (1/2).
08 12 2021 18:43:24.632:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:43:24.637:INFO [launcher]: Trying to start FirefoxHeadless again (2/2).
08 12 2021 18:44:24.687:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:44:24.696:ERROR [launcher]: FirefoxHeadless failed 2 times (timeout). Giving up.
================================================================================
[352 / 353] 6 / 7 tests, 2 failed; Testing //webapp/src/app/table/filter:test.suite_firefox-local; 16s remote-cache, linux-sandbox
[352 / 353] 6 / 7 tests, 2 failed; Testing //webapp/src/app/table/filter:test.suite_firefox-local; 106s remote-cache, linux-sandbox
[352 / 353] 6 / 7 tests, 2 failed; Testing //webapp/src/app/table/filter:test.suite_firefox-local; 0s remote-cache, linux-sandbox
FAIL: //webapp/src/app/table/filter:test.suite_firefox-local (see /home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.log)
INFO: From Testing //webapp/src/app/table/filter:test.suite_firefox-local:
==================== Test output for //webapp/src/app/table/filter:test.suite_firefox-local:
TEST_UNUSED_RUNFILES_LOG_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.unused_runfiles_log
TEST_BINARY=webapp/src/app/table/filter/test.suite_firefox-local.sh
TEST_TEMPDIR=test_tempdir.UpDqdM
TEST_WORKSPACE=data_catalog
RUNFILES_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/table/filter/test.suite_firefox-local.sh.runfiles
TEST_UNDECLARED_OUTPUTS_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.outputs
***
RUN_UNDER_RUNFILES=1
TZ=UTC
TEST_TMPDIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/_tmp/ebed677f15027bf98042737bbcde48e2
TMPDIR=/tmp
XML_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.xml
WEB_TEST_METADATA=data_catalog/webapp/src/app/table/filter/test.suite_firefox-local.gen.json
TEST_UNDECLARED_OUTPUTS_ANNOTATIONS_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.outputs_manifest
TEST_PREMATURE_EXIT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.exited_prematurely
TEST_INFRASTRUCTURE_FAILURE_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.infrastructure_failure
USER=runner
PYTHON_RUNFILES=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/table/filter/test.suite_firefox-local.sh.runfiles
TEST_SRCDIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/table/filter/test.suite_firefox-local.sh.runfiles
SHLVL=1
_RLOCATION_ISABS_PATTERN=^/[^/].*
PATH=.:/bin:/usr/bin:/usr/local/bin
GTEST_TMP_DIR=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/_tmp/ebed677f15027bf98042737bbcde48e2
TEST_WARNINGS_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.warnings
TEST_TARGET=//webapp/src/app/table/filter:test.suite_firefox-local
JAVA_RUNFILES=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/bin/webapp/src/app/table/filter/test.suite_firefox-local.sh.runfiles
TEST_SIZE=large
EXPERIMENTAL_SPLIT_XML_GENERATION=1
TEST_TIMEOUT=900
TEST_LOGSPLITTER_OUTPUT_FILE=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.raw_splitlogs/test.splitlogs
OLDPWD=/home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/sandbox/linux-sandbox/62/execroot/data_catalog
BASH_FUNC_rlocation%%=() {  if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): start" 1>&2;
 fi;
 if [[ "$1" =~ $_RLOCATION_ISABS_PATTERN ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): absolute path, return" 1>&2;
 fi;
 echo "$1";
 else
 if [[ "$1" == ../* || "$1" == */.. || "$1" == ./* || "$1" == */./* || "$1" == "*/." || "$1" == *//* ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: rlocation($1): path is not normalized" 1>&2;
 fi;
 return 1;
 else
 if [[ "$1" == \\* ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: rlocation($1): absolute path without" "drive name" 1>&2;
 fi;
 return 1;
 else
 if [[ -e "${RUNFILES_DIR:-/dev/null}/$1" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): found under RUNFILES_DIR ($RUNFILES_DIR), return" 1>&2;
 fi;
 echo "${RUNFILES_DIR}/$1";
 else
 if [[ -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): looking in RUNFILES_MANIFEST_FILE ($RUNFILES_MANIFEST_FILE)" 1>&2;
 fi;
 local -r result=$(grep -m1 "^$1 " "${RUNFILES_MANIFEST_FILE}" | cut -d ' ' -f 2-);
 if [[ -e "${result:-/dev/null}" ]]; then
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): found in manifest as ($result)" 1>&2;
 fi;
 echo "$result";
 else
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "INFO[runfiles.bash]: rlocation($1): not found in manifest" 1>&2;
 fi;
 echo "";
 fi;
 else
 if [[ "${RUNFILES_LIB_DEBUG:-}" == 1 ]]; then
 echo "ERROR[runfiles.bash]: cannot look up runfile \"$1\" " "(RUNFILES_DIR=\"${RUNFILES_DIR:-}\"," "RUNFILES_MANIFEST_FILE=\"${RUNFILES_MANIFEST_FILE:-}\")" 1>&2;
 fi;
 return 1;
 fi;
 fi;
 fi;
 fi;
 fi
}
BASH_FUNC_runfiles_export_envvars%%=() {  if [[ ! -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" && ! -d "${RUNFILES_DIR:-/dev/null}" ]]; then
 return 1;
 fi;
 if [[ ! -f "${RUNFILES_MANIFEST_FILE:-/dev/null}" ]]; then
 if [[ -f "$RUNFILES_DIR/MANIFEST" ]]; then
 export RUNFILES_MANIFEST_FILE="$RUNFILES_DIR/MANIFEST";
 else
 if [[ -f "${RUNFILES_DIR}_manifest" ]]; then
 export RUNFILES_MANIFEST_FILE="${RUNFILES_DIR}_manifest";
 else
 export RUNFILES_MANIFEST_FILE=;
 fi;
 fi;
 else
 if [[ ! -d "${RUNFILES_DIR:-/dev/null}" ]]; then
 if [[ "$RUNFILES_MANIFEST_FILE" == */MANIFEST && -d "${RUNFILES_MANIFEST_FILE%/MANIFEST}" ]]; then
 export RUNFILES_DIR="${RUNFILES_MANIFEST_FILE%/MANIFEST}";
 export JAVA_RUNFILES="$RUNFILES_DIR";
 else
 if [[ "$RUNFILES_MANIFEST_FILE" == *_manifest && -d "${RUNFILES_MANIFEST_FILE%_manifest}" ]]; then
 export RUNFILES_DIR="${RUNFILES_MANIFEST_FILE%_manifest}";
 export JAVA_RUNFILES="$RUNFILES_DIR";
 else
 export RUNFILES_DIR=;
 fi;
 fi;
 fi;
 fi
}
BASH_FUNC_is_absolute%%=() {  [[ "$1" = /* ]] || [[ "$1" =~ ^[a-zA-Z]:[/\\].* ]]
}
_=/bin/printenv
08 12 2021 18:44:30.331:INFO [karma-server]: Karma v6.3.9 server started at http://localhost:9876/
08 12 2021 18:44:30.332:INFO [launcher]: Launching browsers FirefoxHeadless with concurrency unlimited
08 12 2021 18:44:30.336:INFO [launcher]: Starting browser FirefoxHeadless
08 12 2021 18:45:30.379:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:45:30.386:INFO [launcher]: Trying to start FirefoxHeadless again (1/2).
08 12 2021 18:46:30.434:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:46:30.439:INFO [launcher]: Trying to start FirefoxHeadless again (2/2).
08 12 2021 18:47:30.484:WARN [launcher]: FirefoxHeadless has not captured in 60000 ms, killing.
08 12 2021 18:47:30.489:ERROR [launcher]: FirefoxHeadless failed 2 times (timeout). Giving up.
================================================================================
INFO: Elapsed time: 525.903s, Critical Path: 203.05s
INFO: 353 processes: 288 internal, 64 linux-sandbox, 1 worker.
INFO: Build completed, 3 tests FAILED, 353 total actions
//api/metadata/src:test                                                  PASSED in 4.4s
//webapp/src/app:test.suite_chromium-local                               PASSED in 1.7s
//webapp/src/app/table:test.suite_chromium-local                         PASSED in 1.2s
//webapp/src/app/table/filter:test.suite_chromium-local                  PASSED in 1.7s
//webapp/src/app:test.suite_firefox-local                                FAILED in 180.7s
  /home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/test.suite_firefox-local/test.log
//webapp/src/app/table:test.suite_firefox-local                          FAILED in 180.7s
  /home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/test.suite_firefox-local/test.log
//webapp/src/app/table/filter:test.suite_firefox-local                   FAILED in 181.2s
  /home/runner/.cache/bazel/_bazel_runner/c525028638995f32dc53ea162408dae4/execroot/data_catalog/bazel-out/k8-fastbuild/testlogs/webapp/src/app/table/filter/test.suite_firefox-local/test.log

Executed 7 out of 7 tests: 4 tests pass and 3 fail locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
INFO: Build completed, 3 tests FAILED, 353 total actions
Error: Process completed with exit code 3.

[devversion]

Relevant info: This is likely the error undernath. I patched the firefox launcher to print the error: Error: Access was denied while trying to open files in your profile directory.

bazel test packages/concatjs/web_test/test/karma:testing_firefox-local \
  --sandbox_writable_path="$HOME/Library/Application Support"

The problem seems to be that Firefox tries to write to the default library directory in e.g. Darwin, but in Bazel with the test being sandboxed this conflicts! The flag above allows this to happen, so the tests run as expected.

It seems like Firefox does always make this request, regardless of us specifying a temporary directory in the Karma launcher. The best solution seems to be to run the tests outside of the sandbox for Firefox, unless this maybe is resolved/gone in the future.

[devversion]

I've put together a standalone reproduction: https://github.com/devversion/firefox-sandbox-exec-repro/tree/master

[mgred]

@devversion sorry for my late reply, I got sidetracked from this issue. Thanks a lot for your further investrigations

The best solution seems to be to run the tests outside of the sandbox for Firefox

What do you exactly mean with that?

[devversion]

I was mostly thinking about giving e.g. the web test target a tag like no-sandbox. This will cause the test to run outside of sandbox. Sandboxing is actually a key feature of Bazel though, so it's a trade-off. The bugzilla issue I filed seems to be low priority on Mozilla's side.

[mgred]

Adding no-sandbox to the tags of web_test_suite did it for me 🤩
@devversion thanks for your support and explanation 🙏

[devversion]

@mgred no problem! can you actually keep this issue open if you don't mind? (it's a good tracking issue)