ieee754 encode/decode incorrect

Issue #995 new
Gabriel Burke created an issue

A=ieee754_half_decode(ieee754_half_encode(1/3))
= 0.333251953125

B=ieee754_half_decode(ieee754_half_encode(2/3))
= 0.66650390625

C=ieee754_half_decode(ieee754_half_encode(1/3)) + ieee754_half_decode(ieee754_half_encode(2/3))
= 0.999755859375

D=ieee754_half_encode(1)
= 0x3C00

E=ieee754_half_encode(0.99951171875)
= 0x3BFF

F=ieee754_half_encode(0.999755859375)
= 0x7C00

G=ieee754_half_decode(F)

Current result: NaN

With ieee754_half, the exact value with no error when adding 1/3 to 2/3 is C (0.999755859375). Depending on your rounding policy, this either rounds down to E (the largest representable number less than 1) or up to D (1). F comes back as 0x7C00 which is actually the representation for +infinity, not 1. Also, G = NaN when it should really return +infinity. Bottom line, both F and G produce incorrect values.

It appears the same issue impacts all other implementation of ieee754 encode/decode

Comments (0)

  1. Log in to comment