Fixed Order Dependent Flakiness in testValidFlowNamespaceWithAllValidCharsTypes

[LucasEby]

Fixes #1463

Motivation

This change removes an order dependency between SlangBuilderTest.testValidFlowNamespaceWithAllValidCharsTypes and SlangBuilderTest.testValidFlowNamespaceCaseInsensitive. Prior to this change, SlangBuilderTest.testValidFlowNamespaceWithAllValidCharsTypes only passes when SlangBuilderTest.testValidFlowNamespaceCaseInsensitive mocks and sets the behavior of metadataExtractor.extractMetadataModellingResult() to return EMPTY_METADATA beforehand. Order dependent tests like these are a cause of test flakiness which is particularly harmful in CI: they break the assumption that the tests can be reordered or run in parallel without changing their outcome. As a result, SlangBuilderTest.testValidFlowNamespaceWithAllValidCharsTypes can fail due to external factors despite the test and the code that it is testing remaining unchanged, causing unreliable results from CI and eroding developer trust in the test suite.

Modifications

A few lines were introduced at the beginning of SlangBuilderTest.testValidFlowNamespaceWithAllValidCharsTypes to mock and set the required behavior of metadataExtractor.extractMetadataModellingResult() to return EMPTY_METADATA. Previously, the test would only pass if another test had configured this mock beforehand. The mock configuration persists between tests because the tests use Autowired Spring beans which are shared across tests, and the Before method does not reset the metadataExtractor mock.

As a result, testValidFlowNamespaceWithAllValidCharsTypes is now self-contained and no longer depends on execution order, as it properly initializes its own required mock state.

Affected Tests

This change affects the existing tests:

• io.cloudslang.lang.tools.build.SlangBuilderTest#testValidFlowNamespaceWithAllValidCharsTypes

Tools Used

iDFlakies was utilized to identify the order dependent polluter(s), victim(s), and cleaner(s) in the test suite(s). The tool functions by repeatedly running a project's JUnit tests under many different deterministic test orders to detect flaky tests which pass and fail inconsistently, and then re-executes the failing orders to classify each flaky test as either order-dependent or non-order-dependent. iFixFlakies was executed on the output to further diagnose the source of the problems and additionally generate test helper patches from existing cleaners that were found. With the combination of these tools, I was able to more easily diagnose the source(s) of the problem(s) and create effective solution(s) for them.