Introduction

This repository contains the source code and supporting documents associated with a low-cost hardware data logger designed to capture NMEA2000 and NMEA0183 packets and store them to SD card, with WiFi over-the-air transfer to a capture device (expected to be a smartphone or tablet). The goal of the project is to make the device for as low a cost a possible, allowing for as many as possible to be fielded for any given cost, but still to provide everything required to capture, upload, process, and submit the data to the IHO international Data Center for Digital Bathymetry. The ultimate goal to encourage and enable anyone who wants to better map their local area to do so at the lowest possible cost, while at the same time standardising the formatting and submission process to smooth the passage of data into the appropriate database.

Components

Hardware

The capture hardware has six essential components:

• A microcontroller to sequence operations.
• An SD card to store the captured data.
• A CAN bus controller to talk to the NMEA2000 bus.
• A CAN bus transceiver to provide the physical layer interface to the NMEA2000 bus.
• An RS-422 transceiver and opto-isolators to provide the physical layer interface to the NMEA0183 bus.
• A WiFi interface for communication with the logger, and transfer of data.

Multiple implementations of this design are possible.

Firmware

The firmware used in the logger monitors the input buses, records data in a series of binary format files, and allows interaction from the outside for debugging, inspection, and data off-load purposes.

The firmware is written in C++ using the NMEA2000 library, the appropriate adapter for the hardware implementation, e.g., ESP32 CAN or Arduino Due CAN, and the Arduino support for the microcontroller (e.g., ESP32 Core). Other requirements may be necessary for some implementations (e.g., the Adafruit Bluefruit library for SPI-based Bluetooth connectivity where the microcontroller has no native support).

Data Parser

The firmware stores the data captured from the bus as a binary data packet on the attached SD card in order to reduce the overhead of storing data. Once transferred, a Python parser script is available to read and reconstitute the data on the command line (and as an import-able library), and can also be used to write or transcribe the binary data files (allowing it to be used to translate other formats into the one used here).

Desktop GUI Tool

The logger hardware is usually accessed for programming with the firmware, or debugging, through a USB to UART converter (which is expected to have 5V power but 3.3V data). However, once programmed, it is often simpler to access the logger through the Desktop GUI, which provides capabilities to send arbitrary commands to the logger, set and retrieve configurations, save configurations locally for later reference, set and retried "lab defaults" backup configurations, and to inspect and configure advanced facilities (e.g., NMEA0183 sentence filters, JSON metadata, and algorithm requests).

Cloud Processing

The concept of operations for the logger is to process the data in the cloud, and to allow for direct submission to the IHO Data Center for Digital Bathymetry. The repository has an implementation using AWS Lambda, the setup for which is described here.

Local Processing Tools

The distribution has a number of tools packaged as a Python tool with sub-commands. In addition to providing the ability to parse WIBL files to check the contents, they can be used to edit the contents of WIBL binary files (in a limited context), upload files to AWS for processing, and run both the processing (WIBL to GeoJSON) and submission (upload to DCDB) components from the desktop. A data simulator is also provided to generate sample data files for testing. More details here