Rounding issues

[Former user]

Hi, When I enter 3.1+4.2 on http://symjaweb.appspot.com/

I get the following output:

3.1+4.2 7.300000000000001

The expected output would be 7.3.

I guess this is because you seem to use double over BigDecimal? Are there any workarounds or plans to fix this behaviour?

Thanks, Stefan

[Stefan Norberg]

Sorry, wasn't logged in...

[Axel Kramer]

Symbolically you can input

>>> 31/10+42/10
73/10

You can try increase the accuracy with the undelying apfloat library like this.

>>> N(3.1+4.2, 30)

7.3

but you are right for speed we re using the Java double type in the standard case.

[Axel Kramer]

Another option is to format the output like in this example

    public void testNumberFormat() {
        StringBuilder buf = new StringBuilder();
        try {
            DecimalFormatSymbols otherSymbols = new DecimalFormatSymbols(Locale.US);
            DecimalFormat decimalFormat = new DecimalFormat("0.0####", otherSymbols);
            OutputFormFactory factory = OutputFormFactory.get(true, false, decimalFormat);

            IExpr expr = F.num("12345.123456789");
            factory.convert(buf, expr);
        } catch (IOException e) {
            e.printStackTrace();
        }
        assertEquals(buf.toString(), "12345.12346");
    }

[Axel Kramer]

Please use the Github issue tracker to open a new issue if this issue shouldn't be closed Github issues