HTTPS SSH

Infinispan DynamoDB Cache Store

Cache loader / writer for an AWS DynamoDB database backend.

Requirements

  • Infinispan 9.3+
  • Java 8+
  • AWS DynamoDB database

Features

  • Implements the complete AdvancedLoadWriteStore SPI
  • Provides an interface for transforming Infinispan entries to / from structured DynamoDB items
  • Provides an optional interface for executing arbitrary queries against DynamoDB, bypassing the standard Infinispan load store API
  • Optional transparent application of a range key to facilitate multi-tenancy: sharing a table among multiple Infinispan caches of the same type, while keeping data access isolated
  • Optional transparent DynamoDB table name prefixing
  • Optional support for server-side (data-at-rest) encryption when the DynamoDB table is created
  • Optional creation of the DynamoDB table with an enabled stream of view type NEW_AND_OLD_IMAGES, required for setting up a global table with replicas in two or more AWS regions
  • Dropwizard Metrics: Read, put, delete, process and purge operation timers
  • System property interpolation for all configuration properties using a ${sys-prop-name:default-value} format
  • Multi-level logging via Log4j2
  • Open source (Apache 2.0 license)

Usage

  • Add the Maven dependency coordinates for the DynamoDB cache store to your project.
  • Implement DynamoDBItemTransformer to translate between Infinispan entries (key / value pairs with optional metadata) and DynamoDB items.
  • Configure a DynamoDB store for each Infinispan cache that requires one, by setting the attributes specified in DynamoDBStoreConfiguration. Also, see the example below. Note that the DynamoDB store can safely shared between multiple replicated / distributed instances of a cache. It can also be used in read-only mode.
  • Make sure the AWS credentials for accessing the DynamoDB table(s) are configured in way that the default AWS credentials provider chain can look them up, e.g. by setting the aws.accessKeyId and aws.secretKey Java
    system properties. See http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html

Maven

Maven coordinates:

<dependency>
    <groupId>com.nimbusds</groupId>
    <artifactId>infinispan-cachestore-dynamodb</artifactId>
    <version>[ version ]</version>
</dependency>

where [ version ] should be the latest stable version.

Example configuration

<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="urn:infinispan:config:9.3 http://www.infinispan.org/schemas/infinispan-config-9.3.xsd"
            xmlns="urn:infinispan:config:9.3"
            xmlns:dynamodb="urn:infinispan:config:store:dynamodb:1.2">

    <cache-container name="myCacheContainer" default-cache="users">
        <jmx duplicate-domains="true"/>
        <local-cache name="users">
            <eviction size="100"/>
            <persistence>
                <dynamodb:dynamodb-store
                    shared="true"
                    region="us-east-1"
                    item-transformer="com.nimbusds.infinispan.persistence.dynamodb.UserItemTransformer"
                    table-prefix="myapp_"
                />
            </persistence>
        </local-cache>
    </cache-container>

</infinispan>

Change Log

  • version 1.0 (2017-09-29)

    • First public release, requires Infinispan 8.2+ and Java 8+.
  • version 1.1 (2017-10-11)

    • Simplifies DynamoDBItemTransformer interface. The hash key attribute name is returned directly by the getHashKeyAttributeName method. The hash key value is resolved and returned directly by the resolveHashKey method. This is a breaking change.
  • version 1.2 (2017-10-17)

    • Adds DynamoDBQueryExecutor interface and supporting classes for executing direct queries against the DynamoDB database. The attributes used as keys in the queries must be indexed (as global secondary index).
  • version 1.2.1 (2017-10-17)

    • Updates SimpleMatchQueryExecutor to allow key names that otherwise clash with reserved key words, such as SUB, in DynamoDB.
  • version 1.3 (2017-10-18)

    • Adds MetadataUtils with methods for encoding Infinispan metadata timestamps into DynamoDB items and parsing the metadata back.
    • Moves loggers into own package.
  • version 1.4 (2017-10-18)

    • Updates MetadataUtils.addMetadata to return the (modified) Item.
  • version 1.5 (2017-10-21)

    • Removes aws-access-key-id and aws-secret-access-key DynamoDB store configuration attributes, switches to AWS SDK's DefaultAWSCredentialsProviderChain which provides a richer set of options for setting the AWS credentials.
  • version 1.5.1 (2017-11-02)

    • Adds "endpoint" parameter check for non-blank.
  • version 1.5.2 (2017-11-02)

    • Fixes "region" parameter parsing (issue #5).
  • version 1.5.3 (2017-11-24)

    • Switches to com.amazonaws:aws-java-sdk-bundle:1.11.235 dependency with shaded transient dependencies to prevent potential conflicts.
  • version 1.6 (2018-04-25)

    • Upgrades to org.infinispan:infinispan-core:9.0.0Final+
    • Upgrades to com.nimbusds:infinispan-cachestore-common:1.4+
    • Upgrades to com.nimbusds:common:2.27+
    • Upgrades to io.dropwizard.metrics:metrics-core:3.1.2+
  • version 1.6.1 (2018-04-25)

    • Updates config XML schema.
  • version 1.6.2 (2018-04-27)

    • Adds @Store(shared = true) annotation.
  • version 2.0 (2018-05-25)

    • Implements AdvancedCacheExpirationWriter which passes the entire purged Infinispan entry to the listeners, not just the key.
  • version 2.0.1 (2018-05-27)

    • Refactors ExpiredEntryReaper.
  • version 2.1 (2018-07-08)

    • Supports DynamoDB table creation with data encryption at rest (server-side encryption). Adds new "encryption-at-rest" {true|false} XML attribute to the "dynamodb-store" XML element, defaults to "false" (no encryption).
    • Bumps AWS SDK dependency to 1.11.362.
  • version 2.1.1 (2018-07-08)

    • Removes deprecated DynamoDBStoreConfigurationParser.
  • version 2.2 (2018-09-05)

    • Adds a new "enable-stream" configuration XML attribute to create the DynamoDB table with an enabled stream of view type NEW_AND_OLD_IMAGES. Streams of this type are required to setup a global DynamoDB table with replicas in two or more AWS regions (issue #7).
  • version 2.3 (2018-09-24)

    • Updates SimpleMatchQueryExecutor to fall back to table scan if no global secondary index (GSI) is specified.
    • Updates SimpleMatchQueryExecutor to accept MatchQuery instances, converting them to SimpleMatchQueries if necessary.
    • Bumps minimal com.nimbusds:infinispan-cachestore-common dependency dependency to 2.1.
  • version 2.4 (2018-09-28)

    • Adds a new "enable-continuous-backups" configuration XML attribute to create the DynamoDB table with continuous backups / point in time recovery enabled.
  • version 2.5 (2018-10-01)

    • Sanitises DynamoDB items before writing them to the table. Empty strings, binary data and sets, including those in nested maps, are automatically pruned to prevent ValidationExceptions.
  • version 2.6 (2018-10-03)

    • Extends the programmatic configuration to enable setting a Dropwizard metric registry other than the default singleton com.nimbusds.common.monitor.MonitorRegistries. Use with new ConfigurationBuilder().persistence().metricRegistry(metricRegistry).
  • version 2.7 (2018-11-29)

    • Replaces DynamoDBStore.process method with publishEntries to fix a threading issue when pre-loading the cache from a DynamoDB table, requires Infinispan 9.3+ (see pull request #2 and chat in http://c2id.co/6u ).
    • Infinispan 9.3.0Final becomes the minimal supported version.
  • version 2.7.1 (2018-11-29)

    • Artificial version bump to repeat failed Maven Central release of 2.7.