1. James Murty
  2. jets3t
  3. Issues
Issue #101 invalid

ThreadedStorageService ServiceException only reports 1 failure instead of all failures/successes

csweet
created an issue

When a ServiceException occurs during a ThreadedStorageService call, it would be nice if there was a way to determine which of the StorageObjects were successful, which failed and which were not executed.

In the logs, I can see that 5 out of 20 of the copy commands failed, but the ServiceException only returned back the key for the last one that failed. As for the 6th copy, could not determine if it was either successful or not run due to "Firing ERROR event and cancelling all threads".

This means that my program only knows that 1 copy failed and need to rerun the other 19. In reality the program may not need to rerun any of the object copies since 5 failed and the the rest could have been successful. Worst case scenario, it would take a total of 210 copies to find out that all 20 copies failed.

Comments (2)

  1. James Murty repo owner

    You can monitor ThreadedStorageService events by implementing a StorageServiceEventListener and adding it to the service's collection of listeners, either when constructing a ThreadedStorageService or via the addServiceEventListener() method after the fact.

    For your specific case, you could subclass the StorageServiceEventAdaptor class and implement just the method event(CopyObjectsEvent event) which is fired every time a copy object event occurs. If you monitor the CopyObjectsEvent.EVENT_IN_PROGRESS events you can collect information about which copy events have been completed as they occur (completed copies are available from the the getCopyResults() method) and then use this data to recommence copies if a threaded operation fails.

    For a rough example, see how the method s3ServiceEventPerformed(final CopyObjectsEvent event) in the Cockpit.java application file works. However, be aware that this uses the deprecated method name "s3ServiceEventPerformed" whereas StorageServiceEventListener methods are named simply "event" and have different accessor methods.

    Hope this helps, James

  2. csweet reporter

    (Reply via cary...@shaw.ca):

    I have utilized

    threaded-service.ignore-exceptions-in-multi=true to obtain the list of sourceObjectKeys that have errors using:

    if (ServiceEvent.EVENT_IGNORED_ERRORS == event.getEventCode()) { for (Throwable th : event.getIgnoredErrors()) { if (th instanceof ServiceException) { ServiceException ex = (ServiceException) th; if (!keepErrors[0]) {

    errorList.add(ex.getRequestPath().substring(1).replace("%2F", "/")); } } } }

    I then wanted to obtain the list of completed objects by using EVENT_IN_PROGRESS with getCopyResults() as you suggested, but only found the following details in the results map: { ETag="bca5e03769a9d8e7e61504069b4f9e13", Date=Sun Aug 07 10:49:02 MDT 2011, Content-Length=234, id-2=wOwsGw+tj5h0BL5yMrYOUhIu8cjsN0P4kDp/2uPzz7ZAdztdRconS9V89jQGbclX, Last-Modified=Sun Aug 07 10:49:02 MDT 2011, request-id=8662AC96967623C2, Content-Type=application/xml }

    None of which I am able to link back to the original sourceObjectKeys that were passed in. Since I have disabled the cancel on error routine, I could assume that anything that is not an error was successful, but it would be nice to link the return data with the original request.

    Perhaps the copyObjectImpl method of the RestStorageService class could add a key value from the source storage object. Another point the values could be obtained would be in via the ResultsTuple. getNewlyCompletedResults could query the runnable about the calling attributes.

  3. Log in to comment