Guarantee weights_below to be finite in MOTPE

[eukaryo]

Motivation

Resolve #5374

Description of the changes

I added code to guarantee weights_below to be finite in MOTPE. Weights fall-back to uniform if it contains inf, nan, or a value large enough to be inf when summed.

[nabenabe0928]

Probably, what we should do is as follows:

def _calculate_leave_one_out_hypervolume_contributions(
    loss_vals: np.ndarray, ref_point: np.ndarray
) -> np.ndarray:
    n_below = len(loss_vals)
    on_front = _is_pareto_front(loss_vals, assume_unique_lexsorted=False)
    pareto_sols = loss_vals[on_front]
    contribs = np.zeros(n_below, dtype=float)
    hv = WFG().compute(pareto_sols, ref_point)
    if np.isfinite(hv):
        leave_one_out_masks = ~np.eye(pareto_sols.shape[0]).astype(bool)
        hvs_leave_one_out = np.asarray(
            [WFG().compute(pareto_sols[loo], ref_point) for loo in leave_one_out_masks]
        )
        contribs[on_front] = hv - hvs_leave_one_out
    else:
        _, inv, pareto_sol_counts = np.unique(
            pareto_sols, axis=0, return_counts=True, return_inverse=True
        )
        on_front_and_not_duplicated = np.zeros(n_below, dtype=bool)
        # pareto_sol_counts[inv, i] is how many times we see pareto_sols[i] in pareto_sols.
        on_front_and_not_duplicated[on_front] = pareto_sol_counts[inv] > 1
        contribs[on_front_and_not_duplicated] = 1.0

    return contribs

In the code above, we separate the process for the finite hv case and not.
If hv is finite, we can follow what we did in the past.
If hv is not finite, we do the following:

1. For non-Pareto solutions, define the contribution as zero,
2. For duplicated Pareto solutions, define the contribution as zero, and
3. For unique Pareto solutions, define the contribution as finite.

The second point follows the original concept of the leave-one-out contribution, meaning that if a solution duplicates, its hypervolume contribution becomes zero.
Note that hypervolume contribution in a Pareto-optimal set becomes zero if and only if a solution duplicates in the set.

Then the corresponding lines in _calculate_weights_below_for_multi_objective becomes:

    if n_below == 0:
        weights_below = np.asarray([])
    elif n_below == 1:
        weights_below = np.asarray([1.0])
    else:
        worst_point = np.max(loss_vals, axis=0)
        ref_point = np.maximum(1.1 * worst_point, 0.9 * worst_point)
        ref_point[ref_point == 0] = EPS
        contribs = _calculate_leave_one_out_hypervolume_contributions(lvals, ref_point) + EPS
        weights_below = np.clip(contribs / np.max(contribs), 0, 1)

We can separate _calculate_leave_one_out_hypervolume_contributions as a file, e.g. optuna._hypervolume.llo_hv_contrib.py.

[eukaryo]

Thank you for your detailed advice! However, some of them are beyond the scope of this PR because it is just intended to fix the issue.
Currently I believe that:

1. if the reference_point contains non-finite value, it is reasonable to fallback all weights_below to 1.0 because all of them should be non-finite values. Specifically, this line results nan under such condition. (I found it just now. This is not numpy subtraction, so it does not cause RuntimeWarning.)
2. Elseif the trial exists that contains non-finite value (-inf), the initial PR code will fallback all weights_below to 1.0. However, I have reconsidered that the weights_below for trials that do not contain -inf should be zero or epsilon.
3. Mathematically, trials with non-finite values are incomparable, so it is justified to assume that all of them are on the pareto front and set the weights_below to 1.0.

[nabenabe0928]

LGTM

[nabenabe0928]

@not522 Could you review this PR?

[github-actions]

This pull request has not seen any recent activity.

[eukaryo]

@y0z Could you review this PR? (I am aware that you are very busy with another higher priority project; this re-assignment could be a formality.)