Reading LastModifiedDate for aws-s3 via Jets3t fails Java-8 but not java-7

Create issue
Issue #213 resolved
Former user created an issue

JetS3t fails to get LastModifiedDate for existing S3 object in Java 8 but succeeds in Java 7.

package JsTest

import org.jets3t.service.S3Service; import org.jets3t.service.ServiceException; import org.jets3t.service.impl.rest.httpclient.RestS3Service; import org.jets3t.service.model.S3Object; import org.jets3t.service.security.AWSCredentials;

public class Jetsettester {

public static void main(String[] args) { String awsAccessKey = "<access_key>"; String awsSecretKey = "<secret>"; AWSCredentials awsCredentials = new AWSCredentials(awsAccessKey, awsSecretKey); S3Service s3Service = new RestS3Service(awsCredentials); try { S3Object objectDetailsOnly = (S3Object) s3Service.getObjectDetails("ENTER_BUCKET_NAME", "EXISTING_SAMPLE_KEY.TXT"); Object md = objectDetailsOnly.getMetadata("Last-Modified"); // Object type is 'String' in Java 8 - and 'Date' in Java 7 Date dt = objectDetailsOnly.getLastModifiedDate(); // Hence, raises ClassCastException only in Java 8 . In Java 7 - it is correct Date class } catch (ServiceException e) { e.printStackTrace(); } } }

Ensure a bucket and key exists in Amazon S3. Add it in the code below along with AWS secret and key Run the above code using Java 7 it runs successfully. More specifically, the return type of Object md is type String and return type Date dt is Date. Repeat with Java 8 it fails with a ClassCastException. More specifically, JetS3t is is returning a String date - instead of Date object.

Comments (10)

  1. James Murty repo owner

    I cannot reproduce this issue, the code you provided works fine for me with Java 8:

    $ java -version
    java version "1.8.0_05"
    Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
    Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)
    

    I suspect something else is going on for you. Please provide more information about your environments that work and don't work, or I will need to close this issue as invalid.

  2. Sky Nss

    I just tried with latest java build too.

    java version "1.8.0_51"
    Java(TM) SE Runtime Environment (build 1.8.0_51-b16)
    Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)
    

    I tried it with eclipse maven set to java compliance level either 1.7 or 1.8 with the same result. I tried it on windows 7 and on CentOS 6.6 I tried the java both from JDK as well as JRE

    One more detail - not sure if it matters - is these objects I am accessing might have been created by VFS-S3 (https://github.com/abashev/vfs-s3) a long time ago. Here is the object in AWS Console:

    Last-Modified: Fri May 15 03:01:37 GMT-700 2015
    Key:x-amz-meta-__user__  Val: {last-modified=Fri May 15 06:01:12 EDT 2015}
    Key:x-amz-meta-last-modified Val:Fri May 15 03:01:13 PDT 2015
    

    Help would be appreciated. BTW - there are millions of such objects that worked with previous java - which are now broken by new java.

    I think what is happening is when the metadata is read from the remote S3 object and is set within JetS3t, somehow, first the 'Last-Modified' Key is set as 'Date' object and subsequently, it is found to be duplicate of lowercase 'last-modified' which is now overwritten with a new String date. So the order in which metadata is set locally seems to matter because lowercase 'last-modified' is string, and CamelCase 'Last-Modified' is Date object

    Could it be that in Line 456 of ServiceUtils - the CamelCase 'Last-modified' should be changed to also accept lower case so that in both cases, date conversion is attempted?

  3. James Murty repo owner

    Fix lookup of HTTP header metadata with clashing user-metadata, re #213

    Solve issues where user-set metadata names clash with HTTP response header names, when the user-set names have different capitalisation.

    • Make it possible to retrieve metadata items specifically set from HTTP response headers, as opposed to user-set metadata items.
    • When looking up generic metadata via StorageObject#getMetadata always prefer the HTTP values over other possibilities that might clash from user or service metadata sets.

    → <<cset 8abe35aafd82>>

  4. James Murty repo owner

    Hi @skynss I think I have a workable fix for this issue.

    As a general work-around, when looking up object metadata any values set via HTTP response headers (like Last-Modified) are now preferred to the other kinds of metadata.

    There is also a more robust solution now possible, via the #getHttpMetadata method now available on storage objects that allow users to unambiguously retrieve HTTP header response data.

  5. James Murty repo owner

    Thanks very much for the detailed follow-up explanation, which allowed me to reproduce the issue and work on a fix. The switch from Java 7 to 8 must involve a different ordering of hash map key values, which triggered this nasty latent bug in your case.

    I believe this is now fixed, but please follow-up in this ticket if you still have problems with the latest code.

  6. Sky Nss

    Thanks for fixing. much appreciated. So that I can plan, do we have any idea so far of release of next version?

  7. James Murty repo owner

    I don't have specific plans for releasing the next version 0.9.4 but it has been a while since the last release, and there have been plenty of fixes, so I will aim to make a release soon. Let's say by mid-August – come back here and remind me if it hasn't happened by then.

  8. James Murty repo owner

    Hi @reneloi and @skynss I have just released JetS3t version 0.9.4 to the Maven central repository, with this fix and others. Once the central repo is synced – which might take a few hours – 0.9.4 should be available to you.

  9. Log in to comment