5. Data converters

Databridge provides a number of built-in converters you can use 'out of the box' to transform property values from their source representation to a different representation in the graph. They are implemented as plugins (see the plugins section), and invoked using the convert directive.

#!javascript
 { "name": "dob", "column": "DOB", "convert": "..." }

Dates

unix_date format

converts a date in the specified format to a 64-bit integer unix-epoch timestamp (seconds since midnight on 1st January 1970)

long_date format

converts a date in the specified format to a 64-bit integer timestamp (milliseconds since midnight on 1st January 1970)

iso8601_date format

converts a date in the specified format to an ISO8601 date string in the format "yyyyMMdd". An ISO8601 date string in this form can be read by humans, can be manipulated as a number, and will work correctly in queries requiring date comparisons.

days format

converts a date value to a 64-bit integer representing the total number of days since midnight 1 Jan 1970. Dates prior to 1 Jan 1970 are represented as negative values (i.e. days before the epoch)

julian format

converts a date value to a 64-bit integer representing the Julian day of that date. The Julian epoch (i.e. Julian day 0) starts at noon on 1 Jan, 4713 BC (Julian Calendar). Dates prior to this date are represented as negative values (i.e. days before the epoch)

Numbers

string

ensures the input value will be imported as a string, preventing it from interpreted as a number.

floor

converts the input value to a real number n, and returns the largest 64-bit integer not greater than n

ceil

converts the input value to a real number n, and returns the smallest 64-bit integer not less than n

round precision

converts the input value to a real number n, and returns that number rounded to the specified precision

real

converts the input value to a real (double) value

integer

converts the input value to a 64-bit integer

Using a built-in converter

The built-in data converters are defined using their simple names, unix_date, iso8601_date and so on. Custom data converters (see the next tutorial) are defined using the plugins namespace, e.g. plugins.my_converter

Passing arguments

You can pass arguments to a data converter. For example, date converters require an argument describing the format of the input date field, expressed as a pattern conforming to the Java date format specification

Arguments are introduced using one or more colons to separate them from the converter name, and from each other:

"convert":"converter:arg1:arg2..."

Example: Converting the Satellite launch date to ISO8601 format

Let's convert the Satellite launch_date to into an ISO8601 formatted date. The built-in converter to use is iso860_date. It takes a single argument which is the format of the Launched column in the source data. We can use the following JSON to perform the conversion.

#!javascript
  { "name": "launch_date", "column": "Launched", "convert": "iso8601_date:dd MMM yyyy" }

Using a custom data converter

Sometimes you may find yourself needing to perform a custom data transformation that isn't supported by the Databridge core data converters. To do this you need to write a custom converter.

There are three steps to successfully creating and using a custom data converter, as described in the following sections

Writing the converter

Converters must be written in Groovy, and should conform to this simple template:

package plugins // all custom converters must be in this package
import com.graphaware.neo4j.databridge.plugins.*

def bind() {
    { args -> ... } as Converter
}
this

Let's suppose we want to write a converter that will translate the codes for orbital location codes (LEO, MEO, HEO, etc) into their human readable versions, i.e.

Here's the converter code to do this:

// orbit_location.groovy
package plugins
import com.graphaware.neo4j.databridge.plugins.*

map = [
        'LEO': 'Low-Earth Orbit', 
        'MEO': 'Mid-Earth Orbit',
        'HEO': 'High-Earth Orbit',
        'L1':  'Lissajous 1',
        'L2':  'Lissajous 2'
]

def bind() {

    { args -> 
         map.get(args[0])
    } as Converter
}
this

Registering your converter

In order to use your converter, the importer needs to find it when it starts up.

The Databridge installation directory contains a plugins folder. Simply copy the converter script orbit_location.groovy to that folder, and it will be automatically loaded and bound to the name plugins.orbit_location at startup.

Defining the converter in the schema mapping

The final step is to use the converter in a schema mapping. This is exactly the same process as defining a built-in converter, except that the converter name must start with "plugins." as describe above, and illustrated in the following example.

{
    "nodes": [
      {
        "type": "Location",
        "labels": [ { "name": "Location" } ],
        "properties": [ { 
           "name": "location", 
           "column": "Alt",
           "convert": "plugins.orbit_location" } ],
        "identity": [ "location" ],
        "update_strategy": "unique"
      }
   ]
}

That's all you need to do, but if you're interested in learning more about how data converters work under the hood, please refer to the Plugin Architecture documentation.