9.24(.1) Gson time serialization issue

[Former user]

I've got a unit test with Payload.toString() which is failing after the upgrade to 9.24 or 9.24.1 with the switch to GSON:

java.lang.AssertionError: 
Expected: "auth_time":1518022800
but: was "auth_time":1.5180228E9

Somehow the defautl serialization of GSON for numbers applies with a scientific notation being used.

[Vladimir Dzhuvinov]

Would you post a reproducible snippet to demonstrate the issue? In particular, how the Payload got constructed.

[D Laurent]

    SignedJWT signedJWT = SignedJWT.parse(idToken);
    JWSHeader header = signedJWT.getHeader();
    Payload payload = signedJWT.getPayload();
    Base64URL signature = signedJWT.getSignature();
    assertThat(header.getAlgorithm(), is(JWSAlgorithm.PS512));

    assertThat(payload.toString(), contains("\"auth_time\":1518022800,"));

I’m validating a standard OIDC ID Token from an IdP which contains the auth_time claim.

The new “payload.toString()” with GSON is deserializing using a scientific notation “1.5 E9”

[Vladimir Dzhuvinov]

I added this test to try to replicate the issue: 7d12a75d

The test passes and I have abs no idea what’s going on here :) Would you paste the b64url of the JWT payload?

[Wojciech Czarnecki]

I also got (probably the same) issue when updating from 9.23 to 9.24.1. I put integer in claims, and later it changes to double.

This is surely related to GSON, see: https://github.com/google/gson/issues/1084

The problem is in JWT.getJWTClaimsSet() method, to be more precise in this.getPayload().toJSONObject():

‌

[Vladimir Dzhuvinov]

I’m now contemplating dropping GSON and favor of a stricter and more predictable JSON lib. Suggestions?

[Wojciech Czarnecki]

Just use Jackson ;)

Or apply this: Gson gson = new GsonBuilder().setObjectToNumberStrategy(ToNumberPolicy.LONG_OR_DOUBLE).create();

More details here: https://github.com/google/gson/pull/1290

[D Laurent]

Ok. I isolated the interesting bits in the unit test.

Here is a sample id token:

eyJraWQiOiIzRDctZ3otR1dqZnctR1NGeWpYSlhNbVMyR25vVDJZM0JLZG16NUY4RVc4IiwiYWxnIjoiRWREU0EifQ.eyJzdWIiOiJqb2huLmRvZSIsImF1ZCI6IjdqM2hNSUppSmp4cmUwaTNHMjZJYWZtSDJKSFVqN2NiQ01sN1lTN2l5cUkiLCJhdXRoX3RpbWUiOjEuNjYwNzMwOTg4RTksImlzcyI6Imh0dHBzOi8vZXhhbXBsZS5vcmciLCJleHAiOjE2NjA3MzQ1ODgsImlhdCI6MTY2MDczMDk4OH0.OLuVbbK63NT1edpisP_qmOuZe16SUXAS9T-Gt76MNYrfGTORWPe6c3lINg8GiyHUxc-SXODQQJcAx_3dv-ngeF0SX2qY1IYGARuAlSEL2tOlX-Tk-0seruIeWwiLwBsuMkCxgdbL8nK-H7YdeLE6YCsA

‌

Here is the unit test:

    private static String generateIdToken(long currentTimeMillis)
            throws JOSEException, com.nimbusds.oauth2.sdk.ParseException {
        Issuer iss = new Issuer("https://example.org");
        Subject sub = new Subject("john.doe");
        List<Audience> aud = Collections.singletonList(new Audience("7j3hMIJiJjxre0i3G26IafmH2JHUj7cbCMl7YS7iyqI"));
        long iatMillis = currentTimeMillis;
        Date iat = new Date(iatMillis);
        long idTokenMaxAgeHours = 1L;
        long expMillis = iatMillis + TimeUnit.HOURS.toMillis(idTokenMaxAgeHours);
        Date exp = new Date(expMillis);

        IDTokenClaimsSet idTokenClaimsSet = new IDTokenClaimsSet(iss, sub, aud, exp, iat);
        // auth_time
        idTokenClaimsSet.setAuthenticationTime(new Date(iatMillis));

        JWSAlgorithm jwsAlgorithm = JWSAlgorithm.EdDSA;

        OctetKeyPair signingKeyPair = new ExtendedOctetKeyPairGenerator(Curve.Ed448).algorithm(JWSAlgorithm.EdDSA)
                .keyUse(KeyUse.SIGNATURE)
                .keyIDFromThumbprint(true)
                .generate();

        JWSHeader header = new JWSHeader.Builder(jwsAlgorithm).keyID(signingKeyPair.getKeyID()).build();
        SignedJWT signedJWT = new SignedJWT(header, idTokenClaimsSet.toJWTClaimsSet());
        signedJWT.sign(new Ed448Signer(signingKeyPair));
        String idToken = signedJWT.serialize();

        System.out.println("idToken = " + idToken);
        return idToken;
    }

    @Test
    public void testDeserializeWithGson() throws Exception {

        long currentTimeMillis = System.currentTimeMillis();

        String idToken = generateIdToken(currentTimeMillis);

        SignedJWT signedJWT = SignedJWT.parse(idToken);
        JWSHeader header = signedJWT.getHeader();
        Payload payload = signedJWT.getPayload();
        Base64URL signature = signedJWT.getSignature();
        assertThat(header.getAlgorithm(), is(JWSAlgorithm.EdDSA));

        System.out.println("payload.toString() = " + payload.toString());

        long expectedAuthTime = currentTimeMillis / 1000L;
        assertThat(payload.toString(), containsString(
                        "\"auth_time\":" + expectedAuthTime + ","));

        Map<String, Object> payloadJson = payload.toJSONObject();

        assertThat(payloadJson.get("sub"), is("john.doe"));

        assertThat(payloadJson.get("iss"), is("https://example.org"));
        assertThat(payloadJson.get("aud"), is("7j3hMIJiJjxre0i3G26IafmH2JHUj7cbCMl7YS7iyqI"));
        assertThat(payloadJson.get("auth_time"), is(expectedAuthTime));
    }

‌

Output:

payload.toString() = {"sub":"john.doe","aud":"7j3hMIJiJjxre0i3G26IafmH2JHUj7cbCMl7YS7iyqI","auth_time":1.660730988E9,"iss":"https://example.org","exp":1660734588,"iat":1660730988}

‌

java.lang.RuntimeException: java.lang.AssertionError:
Expected: a string containing ""auth_time":1660730988,"
but: was "{"sub":"john.doe","aud":"7j3hMIJiJjxre0i3G26IafmH2JHUj7cbCMl7YS7iyqI","auth_time":1.660730988E9,"iss":"https://example.org","exp":1660734588,"iat":1660730988}"

‌

[D Laurent]

json-smart 2.4.8 can be compiled for Java 7.

Java 8 is actually only required for 2 test classes: TestGitHubIssue and TestUtf8.

Since you are shading it, you could include classes compiled for Java 7 instead.

I’m thinking json-smart because it’s very lightweight (compared to Jackson) and you can be certain of keeping backward compatibility.

[Vladimir Dzhuvinov]

Thanks for the test snippet, I appreciate that. I plugged it in and the issue was made visible.

I plugged in the suggested Gson gson = new GsonBuilder().setObjectToNumberStrategy(ToNumberPolicy.LONG_OR_DOUBLE).create(); but that didn’t fix the double issue. I’m getting increasingly perplexed why Gson does the things it does :)

Forking JSON Smart and recompiling it for Java 7 will work to solve the immediate issue at hand, but in practice this will mean taking another project under our wing and the responsibility to keep it maintained it to a degree, even if this appears limited right now.

[Wojciech Czarnecki]

It’s strange that ToNumberPolicy.LONG_OR_DOUBLE didn’t work, during my tests when I modified instance of Gson defined in JSONObjectUtils everything worked fine.

[Aliaksei Astashenka]

Got this(?) issue when upgrading from 9.23 to 9.24.1. A lot of failed tests looking similar:

expected: -6846781232409087277L
 but was: -6.846781232409087E18

expected: 1234L
 but was: 1234.0

etc..

Any quick workaround for this? Thanks!

[Wojciech Czarnecki]

Either downgrade to 9.23 or hack Gson used by JSONObjectUtils with something like this (this is just a PoC, don’t use it ):

static {
    try {
        java.lang.reflect.Field field = com.nimbusds.jose.util.JSONObjectUtils.class.getDeclaredField("GSON");
        field.setAccessible(true);
        com.nimbusds.jose.shaded.gson.Gson gson = (com.nimbusds.jose.shaded.gson.Gson) field.get(null);
        field = com.nimbusds.jose.shaded.gson.Gson.class.getDeclaredField("factories");
        field.setAccessible(true);
        List<com.nimbusds.jose.shaded.gson.TypeAdapterFactory> list = new ArrayList<>((List<com.nimbusds.jose.shaded.gson.TypeAdapterFactory>) field.get(gson));
        list.remove(1);
        list.add(1, com.nimbusds.jose.shaded.gson.internal.bind.ObjectTypeAdapter.getFactory(com.nimbusds.jose.shaded.gson.ToNumberPolicy.LONG_OR_DOUBLE));
        field.set(gson, list);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

‌

[Yavor Vasilev]

Does anyone know what setNumberToNumberStrategy(ToNumberPolicy.LONG_OR_DOUBLE) actually does and whether it’s called during JSON parsing?

[Vladimir Dzhuvinov]

The .setObjectToNumberStrategy(ToNumberPolicy.LONG_OR_DOUBLE) apparently did the job. I think I had the tests or something else messed up to say yesterday that it didn't.

The commit with the fix: 72e3fc1d

Check out v9.24.2 in a few minutes or so.