How to use the library

How to get the library

Currently the only way to get the library is via download here on BitBucket. You can find the downloads section here.

The future plan is to provide this with via Maven.

Once you have the library ensure that is in your project's classpath to use.

Creating your log data storage class

Before we can create a log parser implementation we must first create the class in which we want to store the log data. This allows flexibility to your project so you can handle the parsed data in an Object Oriented manner.

Implementing the LogEntry interface

The library provides you with the LogEntry interface which contains no method definitions. This class is used to identify your class as a log data storage class and to allow an instance of this class to be returned to you from the log parser. First you will want to create your class with the fields that you want to populate. Each field must be a String. The library currently cannot convert the log line's string value into any other type.

package my.package;

import org.jrsofty.parser.logging.api.LogEntry;

public class Squid3LogData implements LogEntry {

  private String ip;
  private String dateString;
  private String method;
  private String requestedUrl;
  private String httpType;
  private String status;
  private String dataSize;
  private String referringUrl;
  private String userAgent;
  private String value1;

  public String getIp() {
    return this.ip;
  }

  public void setIp(final String ip) {
    this.ip = ip;
  }

  public String getDateString() {
    return this.dateString;
  }

  public void setDateString(final String dateString) {
    this.dateString = dateString;
  }

  public String getMethod() {
    return this.method;
  }

  public void setMethod(final String method) {
    this.method = method;
  }

  public String getRequestedUrl() {
    return this.requestedUrl;
  }

  public void setRequestedUrl(final String requestedUrl) {
    this.requestedUrl = requestedUrl;
  }

  public String getHttpType() {
    return this.httpType;
  }

  public void setHttpType(final String httpType) {
    this.httpType = httpType;
  }

  public String getStatus() {
    return this.status;
  }

  public void setStatus(final String status) {
    this.status = status;
  }

  public String getDataSize() {
    return this.dataSize;
  }

  public void setDataSize(final String dataSize) {
    this.dataSize = dataSize;
  }

  public String getReferringUrl() {
    return this.referringUrl;
  }

  public void setReferringUrl(final String referringUrl) {
    this.referringUrl = referringUrl;
  }

  public String getUserAgent() {
    return this.userAgent;
  }

  public void setUserAgent(final String userAgent) {
    this.userAgent = userAgent;
  }

  public String getValue1() {
    return this.value1;
  }

  public void setValue1(final String value1) {
    this.value1 = value1;
  }

}

What is important to remember here is that to parse a log line you must capture all the data of the line with your Formatting, and each of formatting token used must be represented in this log data class. Otherwise the library will throw an exception.

Also important is that every field must have a getter and setter in the form of getXXXX setXXXX if these are absent the log parser cannot store the log information.

Next you will need to map your fields to the formatting tokens that you will use to pass to parse out the information from your line of log data. To do this the library provides the annotation @LogElementMapping which is used on the fields of your class. See the example below on how to map your class fields.

package my.package;

import org.jrsofty.parser.logging.api.LogElementMapping;
import org.jrsofty.parser.logging.api.LogEntry;

public class Squid3LogData implements LogEntry {

  /*
  * To capture my squid3 access logs I use these formatting tokens.
  * %ip4 %dtm{dd/MMM/yyyy:HH:mm:ss Z} %opt{GET,POST,DELETE,PUT,HEAD,OPTIONS,CONNECT} %url %str %int %int %url %msg %str
  */

  @LogElementMapping(logToken = "%ip4")
  private String ip;
  @LogElementMapping(logToken = "%dtm")
  private String dateString;
  @LogElementMapping(logToken = "%opt")
  private String method;
  @LogElementMapping(logToken = "%url")
  private String requestedUrl;
  @LogElementMapping(logToken = "%str")
  private String httpType;
  @LogElementMapping(logToken = "%int")
  private String status;
  @LogElementMapping(logToken = "%int")
  private String dataSize;
  @LogElementMapping(logToken = "%url")
  private String referringUrl;
  @LogElementMapping(logToken = "%msg")
  private String userAgent;
  @LogElementMapping(logToken = "%str")
  private String value1;

  //getters and setters removed for page size. Nothing changes there.

}

The log parser uses reflection to identify the mapping and puts the data into each field of your class.

Creating the log parser.

Once you have your class ready you can create your log parser. Start by first creating a default LogParserFactory instance.

    LogParserFactory instance = LogParserFactory.createDefaultInstance();

From the LogParserFactory instance you can create a new parser instance. To create your log parser you must provide it with a String containing the formatting tokens for the data that you want to capture, and a reference to the class that implemented the LogEntry interface with the mapped fields. In this case you must pass the class itself and not an instance of the class.

    String squid3FormatTokens = "%ip4 %dtm{dd/MMM/yyyy:HH:mm:ss Z} %opt{GET,POST,DELETE,PUT,HEAD,OPTIONS,CONNECT} %url %str %int %int %url %msg %str";

    LogParser parser = instance.createLogParser(squid3FormatTokens, Squid3LogData.class);

To parse your line of log data you simply pass it to the parsing method as so.

   String lineOfLogData = "192.168.178.26 [15/Jul/2017:15:08:56 +0200] GET \"http://www.example.com/forum/index.php?action=forum\" \"HTTP/1.1\" 200 8600 \"-\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0\" TCP_MISS:HIER_DIRECT";

   LogEntry parsedEntry = parser.parseLogString(lineOfLogData);

   // you can now test the parsedEntry variable to see that it is an instance of your original class.

In the Customizing section we will describe how you can create your own implementations of the LogParserFactory and LogParser interfaces.

Wiki

Log-Parsing-Library / Usage

How to use the library

How to get the library

Creating your log data storage class

Implementing the LogEntry interface

Creating the log parser.