Wiki
Clone wikineo4j-databridge / 5. Data converters
5. Data converters
Databridge provides a number of built-in converters you can use 'out of the box' to transform property values from their source representation to a different representation in the graph. They are implemented as plugins (see the plugins section), and invoked using the convert directive.
#!javascript { "name": "dob", "column": "DOB", "convert": "..." }
Dates
unix_date format
converts a date in the specified format to a 64-bit integer unix-epoch timestamp (seconds since midnight on 1st January 1970)
long_date format
converts a date in the specified format to a 64-bit integer timestamp (milliseconds since midnight on 1st January 1970)
iso8601_date format
converts a date in the specified format to an ISO8601 date string in the format "yyyyMMdd". An ISO8601 date string in this form can be read by humans, can be manipulated as a number, and will work correctly in queries requiring date comparisons.
days format
converts a date value to a 64-bit integer representing the total number of days since midnight 1 Jan 1970. Dates prior to 1 Jan 1970 are represented as negative values (i.e. days before the epoch)
julian format
converts a date value to a 64-bit integer representing the Julian day of that date. The Julian epoch (i.e. Julian day 0) starts at noon on 1 Jan, 4713 BC (Julian Calendar). Dates prior to this date are represented as negative values (i.e. days before the epoch)
Numbers
string
ensures the input value will be imported as a string, preventing it from interpreted as a number.
floor
converts the input value to a real number n, and returns the largest 64-bit integer not greater than n
ceil
converts the input value to a real number n, and returns the smallest 64-bit integer not less than n
round precision
converts the input value to a real number n, and returns that number rounded to the specified precision
real
converts the input value to a real (double) value
integer
converts the input value to a 64-bit integer
Using a built-in converter
The built-in data converters are defined using their simple names, unix_date
, iso8601_date
and so on. Custom data converters (see the next tutorial) are defined using the plugins
namespace, e.g. plugins.my_converter
Passing arguments
You can pass arguments to a data converter. For example, date converters require an argument describing the format of the input date field, expressed as a pattern conforming to the Java date format specification
Arguments are introduced using one or more colons to separate them from the converter name, and from each other:
"convert":"converter:arg1:arg2..."
Example: Converting the Satellite launch date to ISO8601 format
Let's convert the Satellite launch_date to into an ISO8601 formatted date. The built-in converter to use is iso860_date
. It takes a single argument which is the format of the Launched
column in the source data. We can use the following JSON to perform the conversion.
#!javascript { "name": "launch_date", "column": "Launched", "convert": "iso8601_date:dd MMM yyyy" }
Using a custom data converter
Sometimes you may find yourself needing to perform a custom data transformation that isn't supported by the Databridge core data converters. To do this you need to write a custom converter.
There are three steps to successfully creating and using a custom data converter, as described in the following sections
Writing the converter
Converters must be written in Groovy, and should conform to this simple template:
package plugins // all custom converters must be in this package import com.graphaware.neo4j.databridge.plugins.* def bind() { { args -> ... } as Converter } this
Let's suppose we want to write a converter that will translate the codes for orbital location codes (LEO, MEO, HEO, etc) into their human readable versions, i.e.
code | name |
---|---|
LEO | Low-Earth Orbit |
MEO | Mid-Earth Orbit |
HEO | High-Earth Orbit |
L1 | Lissajous 1 |
L2 | Lissajous 2 |
Here's the converter code to do this:
// orbit_location.groovy package plugins import com.graphaware.neo4j.databridge.plugins.* map = [ 'LEO': 'Low-Earth Orbit', 'MEO': 'Mid-Earth Orbit', 'HEO': 'High-Earth Orbit', 'L1': 'Lissajous 1', 'L2': 'Lissajous 2' ] def bind() { { args -> map.get(args[0]) } as Converter } this
Registering your converter
In order to use your converter, the importer needs to find it when it starts up.
The Databridge installation directory contains a plugins folder. Simply copy the converter script orbit_location.groovy to that folder, and it will be automatically loaded and bound to the name plugins.orbit_location at startup.
Defining the converter in the schema mapping
The final step is to use the converter in a schema mapping. This is exactly the same process as defining a built-in converter, except that the converter name must start with "plugins." as describe above, and illustrated in the following example.
{ "nodes": [ { "type": "Location", "labels": [ { "name": "Location" } ], "properties": [ { "name": "location", "column": "Alt", "convert": "plugins.orbit_location" } ], "identity": [ "location" ], "update_strategy": "unique" } ] }
That's all you need to do, but if you're interested in learning more about how data converters work under the hood, please refer to the Plugin Architecture documentation.
Updated