Design float:round langlib function

[KavinduZoysa]

Overview

According to spec, we have to implement the following function.

# Round a float value to a specified number of digits.
# Returns the float value that is an integral multiple of 10 raised to the power of `-fractionDigits` and closest to `x`.
# If there are two such values, choose the one whose final digit is even
# (this is the round-to-nearest rounding mode, which is the default for IEEE and for Ballerina).
# A value of `fractionDigits` greater than 0 thus corresponds to the number of digits after the decimal
# point being `fractionDigits`; a value of 0 for `fractionDigits` rounds to an integer.
# If `x` is NaN, +0, -0, +∞ or -∞, then the result is `x``.
# When `fractionDigits` is 0, this is
# the same as Java Math.rint method, .NET Math.Round method and
# IEEE roundToIntegralTiesToEven operation
# Note that `<int>x` is the same as `<int>x.round(0)`.
#
# + x - float value to operate on
# + fractionDigits - the number of digits after the decimal point
# + return - float value closest to `x` that is an integral multiple of 10 raised to the power of `-fractionDigits`
public isolated function round(float x, int fractionDigits = 0) returns float = external;

In the current implementation, we do not have fractionDigits and we only support rounding to a float without zero decimal places.

According to the spec, the result of the rounding operation should be,

• an integral multiple of 10^(-fractionDigits)
• the closest to input float value (according to the spec we have to use roundTiesToEven rounding mode, see the section of 4.3.1 in IEEE)

Also,

• if fractionDigits == 0 we can use Java Math.rint
• Output will be same as input if the input is one of NaN, +0, -0, +∞ or -∞.

Implementation Suggestions

Option 1:

    static double round(double x, int fractionDigits) {
        double factor = Math.pow(10, fractionDigits);
        return Math.rint(x*factor)/factor;
    }

Even if the logic is correct this returned error values bacause double operations do not happen as expected.

Option 2:

    static double round(double x, int fractionDigits) {
        BigDecimal bd = BigDecimal.valueOf(x);
        bd = bd.setScale(fractionDigits, RoundingMode.HALF_EVEN);
        return bd.doubleValue();
    }

This method gives the correct output. But there are two drawbacks

1. If fractionDigits == Integer.MAX_VALUE this throws an exception as shown below.

Exception in thread "main" java.lang.ArithmeticException: BigInteger would overflow supported range
	at java.base/java.math.BigInteger.reportOverflow(BigInteger.java:1151)
	at java.base/java.math.BigInteger.pow(BigInteger.java:2482)
	at java.base/java.math.BigDecimal.bigTenToThe(BigDecimal.java:4023)
	at java.base/java.math.BigDecimal.bigMultiplyPowerTen(BigDecimal.java:4158)
	at java.base/java.math.BigDecimal.setScale(BigDecimal.java:2884)
	at java.base/java.math.BigDecimal.setScale(BigDecimal.java:2824)

2. If fractionDigits is close to Integer.MAX_VALUE, it takes some time to give the output.

Because of these drawbacks we have to handle this specially. Please read option 3.

Option 3:
Because of the two drawbacks identified in option 2, we have to add a special calculation.

x	fractionDigits	decimal digits	output
3.123456789	2	9	3.12
3.123456789	4	9	3.1235
3.123456789	9	9	3.123456789
3.123456789	10	9	3.123456789
3.123456789	Integer.MAX_VALUE	9	3.123456789

According to above table we can conculde that if the fractionDigits greater than or equal decimal digits of x, the output is same as x. So we can fix above drawbacks using following method.

    static double round(double x, int fractionDigits) {
        BigDecimal bd = BigDecimal.valueOf(x);
        BigDecimal bdTemp = bd;
        int digitsTmp = fractionDigits;
        while (digitsTmp-- != 0) {
            if (bdTemp.remainder(BigDecimal.ONE).compareTo(BigDecimal.ZERO) != 0) { // remainder != 0
                bdTemp = bdTemp.multiply(BigDecimal.TEN);
                continue;
            } else {
                return x;
            }
        }
        return bd.setScale(fractionDigits, RoundingMode.HALF_EVEN).doubleValue();
    }

Highly appreciate your input on this, please refer #35543.

[KavinduZoysa]

@hasithaa, @gimantha, @MaryamZi, @pcnfernando, @KRVPerera, @chiranSachintha, @suleka96

[KavinduZoysa]

Since we represent ballerina int as long in java, we have to change the type of fractionDigits from int to long. With this change we have to face another problem that BigDecimal.setScale does not accept long. Also we cannot cast long to int because it is not safe. Therefore I have added an extra condion as shown below.

    public static double round(double x, long fractionDigits) {
        if (fractionDigits > Integer.MAX_VALUE) {
            // As per IEEE, exponent of double value lies from -308 to 307.
            // Also, the maximum decimal digits that can have in double value are 15 or 16.
            // Therefore, if `fractionDigits` is very large, `x` will not be changed.
            return x;
        }
        // Down cast can be done safely because of above condition.
        int fractionDigitsAsInt = (int) fractionDigits;
        BigDecimal xInBigDecimal = BigDecimal.valueOf(x);
        BigDecimal xTmp = xInBigDecimal;
        int digitsTmp = fractionDigitsAsInt;
        while (digitsTmp-- > 0) {
            if (xTmp.remainder(BigDecimal.ONE).compareTo(BigDecimal.ZERO) == 0) {
                return x;
            }
            xTmp = xTmp.multiply(BigDecimal.TEN);
        }
        return xInBigDecimal.setScale(fractionDigitsAsInt, RoundingMode.HALF_EVEN).doubleValue();
    }

[KavinduZoysa]

Please find the progress in PR #35606

[KRVPerera]

If fractionDigits range is small why do we check for Integer:MAX_VALUE?

[KavinduZoysa]

The BigDecimal.setScale accepts an int, therefore we have to downcast our fractionDigits value safely.

Also, it is not mentioned that fractionDigits is small. Even if we know that max-min values that can have for float, float coefficient, float-exponent we cannot directly rely on that.

Because of these reasons I have checked the Integer:MAX_VALUE.

[KavinduZoysa]

There is a small correction that I have to do.
If the fractionDigits < 0, existing implementation should be panic. But it is not correct. We have to return a value.
Correction is available at 7b63cdb

[KavinduZoysa]

while loops are removed in 3f30ae2