Crashes when calculating unmask

[Former user]

Originally reported on Google Code with ID 180

What steps will reproduce the problem?
1. try to calculate unmask(0x12;2)
2. or unmask(2;2)
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?
SVN Rev. 1303 compiled on openSUSE 10.3 (i586)

Please provide any additional information below.

Reported by yolla.indria on 2008-04-06 22:24:15

[Former user]

``` unmask (2;2) or unmask (0x12;2) returns -2 in my version (Kubuntu). No crash ```

Reported by `wolf.lammen` on 2008-04-06 23:29:07

[Former user]

``` Same as with Wolf here. ```

Reported by `helder.pereira.correia` on 2008-04-07 06:48:40

[Former user]

``` Reproducable under OpenSUSE 10.3 with Qt 4.4. ```

Reported by `ariya.hidayat` on 2008-04-10 08:00:52 - Labels added: Priority-High - Labels removed: Priority-Medium

[Former user]

``` Here is the partial valgrind report. Wolf, can you have a look? Looks like something uninitialized have been used.

13673== Invalid read of size 4

13673== at 0x80E1E91: _setunsigned (floatconvert.c:98)

13673== Address 0xBE98DF0C is just below the stack ptr. To suppress, use:

--workaround -gcc296-bugs=yes

13673

13673== Use of uninitialised value of size 4

13673== at 0x80E1EBB: _setunsigned (floatconvert.c:100)

13673== by 0x80EB6D6: float_setscientific (floatnum.c:918)

13673== by 0x80EB7AF: float_setinteger (floatnum.c:953)

13673== by 0x80E1E90: _setunsigned (floatconvert.c:97)

13673

13673== Use of uninitialised value of size 4

13673== at 0x80EA602: _checkmul (floatlong.c:359)

13673== by 0x80EB6D6: float_setscientific (floatnum.c:918)

13673== by 0x80EB7AF: float_setinteger (floatnum.c:953)

13673== by 0x80E1E90: _setunsigned (floatconvert.c:97)

13673

13673== Use of uninitialised value of size 4

13673== at 0x80EA639: _checkmul (floatlong.c:362)

13673== by 0x80EB6D6: float_setscientific (floatnum.c:918)

13673== by 0x80EB7AF: float_setinteger (floatnum.c:953)

13673== by 0x80E1E90: _setunsigned (floatconvert.c:97)

13673

13673== Use of uninitialised value of size 4

13673== at 0x80EA63C: _checkmul (floatlong.c:363)

13673== by 0x80EB6D6: float_setscientific (floatnum.c:918)

13673== by 0x80EB7AF: float_setinteger (floatnum.c:953)

13673== by 0x80E1E90: _setunsigned (floatconvert.c:97)

13673

13673== Use of uninitialised value of size 4

13673== at 0x80EA63D: _checkmul (floatlong.c:363)

13673== by 0x80EB7AF: float_setinteger (floatnum.c:953)

13673== by 0x80E1E90: _setunsigned (floatconvert.c:97)

```

Reported by `ariya.hidayat` on 2008-04-10 08:04:50

[Former user]

``` My notebook is currently broken, so I have no SVN access and cannot update the sources. Maybe some of the others can help out. Using the information from Ariya, I spotted a possible problem in floatnum.c in the function set_integer. This function converts a standard int into a floatnum representation. It does so by getting an ASCII representation of the int and feeding that string into floatnums ASCII conversion routines. In the course of events it allocates a temporary buffer on the stack and uses sprintf to create an ASCII string of the int. The size of the buffer is based on the size of an unsigned (usually 32 bits). Since a single decimal digit contains slightly more than 3 bit information, the buffer should contain 32/3 + 1 bytes to hold the decimal digit sequence. What has been forgotten here, are two extra bytes needed for a possible sign and a terminating \0, so we have a classical buffer overflow here. If somebody could change the buffer size into char buf[BITS_IN_UNSIGNED/3 + 3]; and check whether the crash disappears on SuSE, that would be nice. ```

Reported by `wolf.lammen` on 2008-04-10 09:02:03

[Former user]

``` Correction: The name of the faulting function is float_setinteger ```

Reported by `wolf.lammen` on 2008-04-10 10:07:19

[Former user]

Reported by `helder.pereira.correia` on 2008-04-10 11:09:37 - Status changed: `Accepted` - Labels added: Milestone-0.10.1

[Former user]

``` applied the fix to trunk and 0.10 ```

Reported by `wolf.lammen` on 2008-04-10 15:06:07

[Former user]

``` The fix is confirmed, thanks! Valgrind also does not complain anymore. ```

Reported by `ariya.hidayat` on 2008-04-10 15:18:58 - Status changed: `Fixed`

[Former user]

``` And I checked that during the evaluation of unmask(2;2) the buffer holds "-2147483648", which is Nullbyte incl. 1 byte more than the the buffer was long. ```

Reported by `wolf.lammen` on 2008-04-10 15:23:41

[Former user]

Reported by `helder.pereira.correia` on 2008-04-10 22:29:06