refine distance to ignore targets with bounds that have been met

[carolyner]

The algorithm currently sorts parameters based on distance from targets with unmet bounds, requiring all sub-targets within a group have bounds met. This results in the algorithm not prioritizing sub-targets with bounds that have not yet been met, and so missing points that are within all bounds. This was not possible when using alpha-levels (as in the 2019 AOAS paper).

Solution is to calculate distance using only targets with unmet bounds, until all tolerance bounds have been reached.

I believe this can be solved by modifying the distance calculation for grouped targets such that: until bounds are met for all grouped targets, only calculate the distance based on targets with unmet bounds. Then, once all bounds are met for the grouped target, calculate the distance across all targets.

[cmaerzluft]

@carolyner and @pedroliman, I pushed changes for this to UserScaling. It passed all my test scenarios. At the airport and the wifi keeps kicking me off the RAND network so I can't message on teams. The major changes were to eval_targets so you can take a look there to see what changed. Please test the changes and let me know if it looks good.

p.s. I made a small change to Targets.R but that was to correct an issue where the update attribute wasn't properly included when the targets object was subsetted using square brackets and groups (e.g. targets["G1", groups_given = TRUE]). This is the only place where that would happen and it didn't matter prior to the changes made above.

[carolyner]

The distance sort does not always identify targets that are within the range of all stopping bounds. In addition, we have confirmed that the algorithm can get to a point where all points predict targets within final stopping bounds yet the algorithm reports that bounds are not met AND returns with a message of "unable to update bounds", so that is appears that it has failed to converge.

One part of this fix, which would improve efficiency, would be to check for points that predict within targets at line 686 of imabc.R, by adding code to:

1. evaluate targets (call eval_targets) using criteria="stop"
2. setting the total distance to zero when all targets are within the stopping bounds.

This will place all of these sampled points at the top of the sort. If this approach is used, the centers will be selected first from points that are within the stopping bounds, and from those within stopping bounds those with the smallest weighted euclidean distance.

However, there appears to be an additional bug which is related either to the assignment/updating of the bound information or to how the bounds are updated in get_new_bounds.R The possible issues are:

1. The restrictions in lines 31-40 of get_new_bounds.R are problematic
2. The scaling restriction, in lines 47-55 of get_new_bounds.R are problematic: these restrictions should only be applied across sub-targets with bounds that have not been met.

[cmaerzluft]

I just made three changes to the code to address these.

1. get_new_bounds now calculates deltas relative to the targets rather than the stopping bounds (but new bounds can never be more strict than the stopping bounds)
2. target_distance now calculates any observed targets within their stopping bounds as having distance = 0.

• This means that both the total distance and euclid distance will be 0 for any draws with target values within the stopping bounds.

3. line 782 of imabc is now more lenient on when to try to update bounds. Before the code would not update the bounds if new_iter_valid_n == current_good_n. This meant that if the algorithm happen to have all points - both previous iteration and current iterations sum to current_good_n then it wouldn't try to update them. This was done to keep the algorithm from updating when no points were added, but it seems in this case, it would prevent the algorithm from updating the target bounds because all the points from the previous iteration and this iteration (which defines current_good_n) are within the stopping bounds.

• At line 677 we also have a rule that we must have iter_valid_n > 0 in order to try to update the target bounds. I wonder if this could cause issues on a restart. On its face it seems like it wouldn't be a problem since new values would be simulated that would be within the targets but I don't know. It is there so we don't try to update the bounds when we haven't generated any new points worth trying.