TObjectIntHashMap has problem with equals and hashcodes

[Aleksey Linetskiy]

When adding a record to TObjectIntHashMap, I got the following error. Please note, that both strings are referring to the same object, and the hashCodes are, of course, the same.

Here is the code snippet where the error occurred : tags = new TObjectIntHashMap<String>(); scanner.nextLine(); while (scanner.hasNextLine()) { String str = scanner.nextLine().trim(); if(str.equals("")) { continue; } String[] tag = str.split(",");

            int start_of_str = 0;
            if(tag.length > 1 && tag[1].equalsIgnoreCase("TRUE"))
            {
                start_of_str = 1;
            }
            tags.put(tag[0], start_of_str);
            }

Stacktrace:

java.lang.IllegalArgumentException: Equal objects must have equal hashcodes. During rehashing, Trove discovered that the following two objects claim to be equal (as in java.lang.Object.equals()) but their hashCodes (or those calculated by your TObjectHashingStrategy) are not equal.This violates the general contract of java.lang.Object.hashCode(). See bullet point two in that method's documentation. object #1 =class java.lang.String id= 29669059 hashCode= -597802845 toString= user_agent=Mp3Bot; object #2 =class java.lang.String id= 29669059 hashCode= -597802845 toString= user_agent=Mp3Bot

            at gnu.trove.impl.hash.TObjectHash.buildObjectContractViolation(TObjectHash.java:464)
            at gnu.trove.impl.hash.TObjectHash.throwObjectContractViolation(TObjectHash.java:426)
            at gnu.trove.map.hash.TObjectIntHashMap.rehash(TObjectIntHashMap.java:191)
            at gnu.trove.impl.hash.THash.postInsertHook(THash.java:388)
            at gnu.trove.map.hash.TObjectIntHashMap.doPut(TObjectIntHashMap.java:265)
            at gnu.trove.map.hash.TObjectIntHashMap.put(TObjectIntHashMap.java:240)

[Rob Eden]

The problem will be that your hashcodes are computed from mutable state in the object and that state changed at some point.

[Aleksey Linetskiy]

I am using Strings as keys here, and I thought that Strings in Java are immutable...

[Rob Eden]

Ah, I see that now. Do you have a set of text that replicates the issue? Does it happen consistently?

[Aleksey Linetskiy]

It just happened once. I will try to obtain the data.

[Aleksey Linetskiy]

Here is the data that caused this issue.

[Rob Eden]

What version of Trove? I don't see that mentioned...

[Aleksey Linetskiy]

3.0.3

[Rob Eden]

This test case works with that file. Could you see if it works for you?

public class Issue36Test {
    public static void main( String[] args ) throws Exception {
        Scanner scanner = new Scanner( new File( args[ 0 ] ) );
        TObjectIntHashMap<String> tags = new TObjectIntHashMap<>();
        scanner.nextLine();
        while ( scanner.hasNextLine() ) {
            String str = scanner.nextLine().trim();
            if ( str.equals( "" ) ) {
                continue;
            }
            String[] tag = str.split( "," );
            int start_of_str = 0;
            if ( tag.length > 1 && tag[ 1 ].equalsIgnoreCase( "TRUE" ) ) {
                start_of_str = 1;
            }
            tags.put( tag[ 0 ], start_of_str );
        }

        System.out.println( "Map size: " + tags.size() );
    }
}

[Rob Eden]

NOTE: Attachment was deleted to keep it private.

[Aleksey Linetskiy]

The test case worked for me as well. The only difference was that I had to change one line:

TObjectIntHashMap<String> tags = new TObjectIntHashMap<String>();

Since we are building for Java 1.5 and the code won't compile otherwise. Not sure whether this is a relevant difference....

[Rob Eden]

We're unable to reproduce, but it's clearly an issue since the exception is citing the same (immutable) object. There must be a logic issue in there somewhere.

I've escalated the severity, though it might be difficult to fix without a test case demonstrating the failure. I'll start by a general logic review.

[Rob Eden]

@alekseylinetskiy - Any chance this was using in a multi-threaded environment without proper locking? It's a long-shot as I wouldn't expect this as a typical failure symptom in that case, but it's worth a shot. :-)

[Martin Sivák]

I just hit this with a "normal" object:

Caused by: java.lang.IllegalArgumentException: Equal objects must have equal hashcodes. During rehashing, Trove discovered that the following two objects claim to be equal (as in java.lang.Object.equals()) but their hashCodes (or those calculated by your TObjectHashingStrategy) are not equal.This violates the general contract of java.lang.Object.hashCode(). See bullet point two in that method's documentation.

object #1 =class org.marsik.elshelves.backend.entities.Purchase id= 1853601219 hashCode= 340716350 toString= org.marsik.elshelves.backend.entities.Purchase{id=null, uuid=7a938348-6ae9-11e5-89fc-820f8dc8fba1}; object #2 =class org.marsik.elshelves.backend.entities.Purchase id= 1853601219 hashCode= 340716350 toString= org.marsik.elshelves.backend.entities.Purchase{id=null, uuid=7a938348-6ae9-11e5-89fc-820f8dc8fba1}

Please note that is also cites the same hashCode for both objects. The object is mutable, but the hashCode only uses the uuid (which is identical as well). It is easy to reproduce, but hard to isolate as it happened during data restore with about a megabyte of data (it happens every time I try that though).

The trove version was 3.0.3.

[Rob Eden]

@MarSik - Can you take a look at issue #24 and see if this helps? This comes up fairly regularly when using mutable objects. It can be challenging to implement it correctly.

[Lee Harland]

Hi came across this thread when googling for a very similar error. Created a TObjectIntHashMap<String> and processing a large amount of data and can reproduce this consistently; unfortunately no matter how hard i try i cant make a unit test that fails, i only see it when i'm running our proprietary and very large codebase so at present i'm unable to ship any test case (i have done as much as i could to try to "break" trove and so far not managed it). I guess if there is anything i can debug i'm happy to run my script again if there's anything i can do. [Other than this trove is marvellous].

java.lang.IllegalArgumentException: Equal objects must have equal hashcodes. During rehashing, Trove discovered that the following two objects claim to be equal (as in java.lang.Object.equals()) but their hashCodes (or those calculated by your TObjectHashingStrategy) are not equal.This violates the general contract of java.lang.Object.hashCode(). See bullet point two in that method's documentation. object #1 =class java.lang.String id= 891529619 hashCode= 947736095 toString= #document/gtr:project/gtr:publications/gtr:publication/gtr:id; object #2 =class java.lang.String id= 1005384738 hashCode= 947736095 toString= #document/gtr:project/gtr:publications/gtr:publication/gtr:id

at gnu.trove.impl.hash.TObjectHash.buildObjectContractViolation(TObjectHash.java:464)
at gnu.trove.impl.hash.TObjectHash.throwObjectContractViolation(TObjectHash.java:426)
at gnu.trove.map.hash.TObjectByteHashMap.rehash(TObjectByteHashMap.java:191)
at gnu.trove.impl.hash.THash.postInsertHook(THash.java:388)
at gnu.trove.map.hash.TObjectByteHashMap.doPut(TObjectByteHashMap.java:265)
at gnu.trove.map.hash.TObjectByteHashMap.put(TObjectByteHashMap.java:240)

It is running in a multi-threaded app but the instance itself is confined to a single thread so its not a simple concurrency issue (although i obviously dont know what could be going on deep down).

Thanks

[jimdavies]

With no repeatable test case, whilst I cannot rule out a bug with Trove, I cannot say with certainty that there is definitely a bug in trove. I'm going to close this for the moment, but please report any new information.

If it is a bug in Trove, I will make fixing it a priority.