Wiki

Clone wiki

BibSonomy / development / modules / synchronization / Synchronization

General information

BibSonomy / PUMA provides a synchronization interface for publications and website bookmarks.

Overview

Mode of Operation

  • Before the regular synchronization works correctly you must ensure that both systems are synchronized. This can done by an initial synchronization
  • An important requirement is that BibSonomy always must have saved the date of the last synchronization with the corresponding service
  • The client (eg JabRef or PUMA) sends a list of IDs (Intra-/Interhash) to the server (BibSonomy) including the Create-/Update-Date, desired conflict resolution strategy and its client ID.
  • The server decides which posts are deleted, inserted or changed (see "Possible states") and triggers a conflict resolution strategy. As a result he sends the client a list of hashes and operations.
  • The client executes the given operations.

how the synchronization works

see below for more information.

Open questions

Actually, the procedure described above is a large transaction, so no further posts should be made/changed during the synchronization. We could "lock" the user's account for that period. Problem: What happens when the transaction suddenly aborts - when will the account be unlocked again? Any other steps to be taken care of?

Possible states

  • We assume that, every time you sync, the sync-time is being saved. Further we assume that at this time, both systems were synchronized. (We discuss the problems of this assumption in another place,)
  • A post on the client may consequently have a newer or an older processing-date as the date of the last synchronization. Same for the server-side.
  • In addition, a post only exists on the client or the server-side.

Therefore, the following combinations are possible:

Server → Client ↓ older post newer post no post exists
older post do nothing (maybe check if post-date is different) UPDATE* (post changed on server, update client-side) DELETE* post deleted on server, delete on client-side too)
newer post UPDATE* (post changed on client, update server-side) use conflict-solve strategy CREATE* (post created on client, add to server-side aswell)
no post exists DELETE* post deleted on client, delete on server-side too) CREATE* (post created on server, add to client-side aswell)

Possible operations that are carried out by client

The operations sent to the client must perform the relevant post on the server, ie "UPDATE" and "CREATE". The *-variants (CREATE* UPDATE* DELETE*) mean that the client has to perform the particular operation on its own data. At an UPDATE* and CREATE* the client sends the current post to the server, or we will send it directly.

Conflict resolution strategies

Is a post available on both, the client & server, and both posts have been changed since the last synchronization, so there is a conflict. The server then decides on the basis of the Client desired conflict resolution strategy, how to deal with the post:

  • client wins: post from client overwrites the one on the server. server responds with UPDATE*
  • server wins: server post overwrites the client post. server responds with UPDATE*
  • last changed wins: the last changed post overwrites the other one client / server responds with UPDATE*
  • first changed wins: the first changed post overwrites the other client / server responds with UPDATE*
  • ask user: the client asks the user which post shall be overwritten

see also: [1]

  • Notice: If a post changed on client and server and thereby the hashes were changing, so that the two posts no longer can be assigned, then we get duplicates.

Directions

Configuration

Multiple clients per SyncClient

In order to allow multiple clients per SyncClient, the client:// URI SyncServices were introduced. For synchronization, in addition, a UUID that must be unique per device, is required. The SyncClient ID has the following format:

 client://<SYNC_CLIENT>/<UUID>

​e.g.

 client://android/sdafl13234dfsf2sdf23

Authentication and Authorization

  • An essential requirement (necessary for the PUMA - > BibSonomy Synchronization) is to save the posts with the right (usually in the past lying) date.
  • Since it's not so simple we must consider, which service we grant the right to set the date of a post to itself safely.
  • Another essential factor is: the system Sync Framework must know the service to which it allows setting the date.
  • That is, if BibSonomy would allow PUMA users that they synchronize their PUMA accounts with BibSonomy, then it needs to know the BibSonomy-PUMA instance. Of course, this also can be solved by reversing the direction of the synchronization and tell all PUMA instances that BibSonomy knows.
  • Solution of the problem by means of authentication ("Which service is that? ") and authorization (" Is the service allowed to set the date? ").
  • Auth carried out by public-key-cryptography and signed requests
  • Would not a simply IP address based variant be enough? The IP addresses of the authorized instances are known.
  • I.e., we have a (Admin) infrastructure to store and exchange keys for services (/admin/sync)

Requirements

Technical

Set the Tomcat Property that allows encoded slashes in URLs:

org.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true

Configuration

Registered SyncClient and SyncServer on the respective systems.

Expiration of a synchronization

Request a SyncPlan

Send to /api/sync/<SYNC_SERVICE_ID> via POST, all local entries with date of creation, hash and edit date:

<syncPosts>
    <syncPost createDate="2.2.2012 14:14:03" changeDate="2.2.3013 14:51:51" hash="INTRAHASH1" />
</syncPosts>

possible parameters:

  • strategy
  • direction
  • resourcetype
  • deviceInfo

The server starts a new Sync (status: Planet) and responds with a SyncPlan, to be executed by the client. For example:

<syncPosts>
    <syncPost createDate="2.2.2012 14:14:03" changeDate="2.2.3013 14:51:51" hash="INTRAHASH1" action="CREATE_SERVER" />
</syncPosts>

Mark Sync as running

The client does a PUT request to /api/sync/<SYNC_SERVICE_ID>, parameter status=RUNNING and marks Sync as Running.

Execute the SyncPlan

The client performs the sync plan of the server which it has received in the first step.

Finish Sync

If the Sync Plan has been implemented, the client marks the synchronisation as DONE. See above, only instead of RUNNING, DONE is being transmitted as status.

Comments

Every time the sync can be marked as failed. For this purpose, update the status to ERROR status.

Java Implementation

Please make sure to set up your Tomcat/JVM with the certificate of the corresponding test-system (Biblicious/Puma)

SSL-Certificates

The interface to the synchronization is part of the Logic-interface. It's implementation is in the database module.

It contains the following methods (please adjust):

public List<Synchronization> getSynchronization(final String user, final Class<? extends Resource> resourceType, final List<SynchronizationObject> posts, final ConflictResolutionStrategy strategy, final String serviceIdentifier) {
...
}

public interface SyncLogicInterface {

    public void storeNewClientForUser(final String userName, final int serviceId, final Properties userCredentials);

    public List<SyncService> getSyncServicesForUser(User user);

    public Date getCurrentSyncDate(String userName, int serviceId, int contentType);

    public void setCurrentSyncDone(SynchronizationData data);

    public SynchronizationData getCurrentSynchronizationDataForUserForServiceForContent (String userName, int serviceId, int contentType);

    public SynchronizationData getLastSynchronizationDataForUserForContentType (String userName, int serviceId, int contentType);

    public List<SynchronizationPost> getSyncPostsListForUser (Class<? extends Resource> resourceType, String userName);

    public Map<String, SynchronizationPost> getSyncPostsMapForUser(String userName);

    public List<SynchronizationPost> getSynchronization(final String userName, Class<? extends Resource> resourceType, final List<SynchronizationPost> clientPosts, final ConflictResolutionStrategy strategy, final String serviceIdentifier);
}

Updated