-
Notifications
You must be signed in to change notification settings - Fork 33
Open
Description
Some of the ct testcases fail from time to time but not deterministically
This failure is quite common and hopeless to fix because of how timers work in Erlang. (A call to timer:sleep(20) is guaranteed to last at least 20 ms but there is no upper limit)
%%% xprof_tracing_SUITE ==> long_call: FAILED
%%% xprof_tracing_SUITE ==>
Failure/Error: ?assertMatch({ true , _ }, { Min < 22 * 1000 , Min })
expected: = { true , _ }
got: {false,24480} line: 396
Another example when samples are not yet available. (This happens quite often in long_call testcase)
%%% xprof_http_e2e_SUITE ==> capture_data_when_traced_test: FAILED
%%% xprof_http_e2e_SUITE ==>
Failure/Error: ?assertEqual(1, length ( proplists : get_value ( << "items" >> , Data ) ))
expected: 1
got: 0 line: 210
To make snapshots more deterministic (without waiting 1-2 seconds) is to add a function to xprof_core_trace_handler to trigger taking one synchronously (and callerlang:trace_delivered before). But as the key of the snapshots is the timestamp in seconds, there can only be one snapshot per second (and some testcases require two).
Or captured data is not yet available:
%%% xprof_http_e2e_SUITE ==> capture_data_with_formatted_exception_test: FAILED
%%% xprof_http_e2e_SUITE ==>
Failure/Error: ?assertMatch([ << "** exception error: no match of right hand side value ok" >> ], [ proplists : get_value ( << "res" >> , Item ) || Item <- proplists : get_value ( << "items" >> , Data ) ])
expected: = [ << "** exception error: no match of right hand side value ok" >> ]
got: [] line: 244