What is openQA?

openQA is a framework and platform for Question Answering development and publication.

What is the License?

openQA framework is licensed with Apache 2.0.

Why should I use openQA?

The principle of openQA is to make easy the QA development. In that regard openQA offer already a set of built in functionalities as server, web and command interfaces, cache and QALD benchmark integration, between others.

Moreover, openQA is built in an extensible platform that allows users to easily modify behaviors and functionalities.

What are the design principles behind the framework?

openQA is built on top of four design patterns: Service Locator, Provider Model, Factory, Dependency Injection and Service Layer.

Service Locator and Provider Model are well known and spread design pattern for architecture widely adopted in the development of reusable components in several technologies, e.g., JDBC, Java Cryptography Extension, Java Naming and Directory Interface, Java Application Program Interface for XML Processing, and Geographic Information Systems (Geotools). It provide a mechanism for locate services in the class path. Provide Model allows the user to choose an implementation between a list of located implementations. The implementation is than instantiated over a Factory.

During the engine execution every component receive the available services and contexts via Dependency Injection. Which separates the creation of a client's dependencies from its own behavior, allowing program designs to be loosely coupled and to follow the dependency inversion and single responsibility principles.

The architecture is constructed in a Service Layer, where the framework is the enterprise layer, the workflow is the process layer that made use the services in the service Layer which can make use of one or more components to access or manipulate the objects.

OK, I got it! How can I use openQA?

openQA have is built on top of two environments one for (1) development and another for (2) deployment.

The openQA engine can be used over command line interface where the user can pass as argument a plug-in directory plus a desired command.

With the openQA engine the user can query, benchmark and test the application before deployment.

e.g.

(1) Download the openQA engine.

(2) Create a folder called 'plugin'.

(3) Download the openQA example PAR project.

(4) Place the downloaded openQA example PAR into the 'plugin' directory.

(5) execute the following command line:

java -jar openqa.engine...jar plugin -query "leipzig"

If your application is working, now you can download and run it using any existing plug-in.

(*) ps: if you are receiving some error on your log file, please consult Common errors section to check how can it be solved. In case you can find it there, please report the problem at issues.

How can I use openQA in my existing project?

1) Add the following dependency on your project:

<dependency>
  <groupId>org.aksw.openqa</groupId>
  <artifactId>engine</artifactId>
  <version>0.0.7-beta</version>
</dependency>

2) Add the internal AKSW archive repository on your pom file:

<repositories>
    <repository>
      <id>maven.aksw.internal</id>
      <name>University Leipzig, AKSW Maven2 Repository</name>
      <url>http://maven.aksw.org/archiva/repository/internal</url>
    </repository>
  ...
</repositories>

How can I create my own plugin?

You need to pack your system into a jar file with all dependencies included and a few special files and directories. We assume you have a Java Maven project with the usual file structure (src/main/java/your/packages and src/main/resources), otherwise adjust as necessary (e.g. Maven will put all files from src/main/resources into the root of the jar, if you use other methods you may need to put the files there directly).

You can either build a new project for the plugin that has a dependency on your QA system or you just integrate the following into your existing project. Once you are decided and set this up, follow these steps:

1. Add a dependency to the openQA engine (see question above).
2. Extend AbstractQueryParser with YourOwnQueryParser (example).
3. Extend AbstractQueryParserFactory with YourOwnQueryParserFactory (example).
4. Create the following files:

• src/main/resources/your.package.YourOwnQueryParser.inf contains information about your plugin (example):

label=Your Great Parser Name
author=Your Name
contact=Your homepage or email
description=Your description
license=Your license

• src/main/resources/META-INF/your.package.YourOwnQueryParser.ini contains initial values for user modifiable parameters. Allowed types are strings, primitives and enums (example). An example:

endpoint=http://dbpedia.org/sparql
timeout=1000

• src/main/resources/META-INF/services/org.aksw.openqa.component.answerformulation.IQueryParserFactory contains just a single line with the full name (including package) of your parser factory, such as your.package.YourOwnQueryParserFactory

5. Include the Maven assembly plugin in your pom.xml:

<build>
        <plugins>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
            </plugin>
        </plugins>
</build>

6. Assemble the jar: mvn clean compile assembly:single 7.
   • Download openqa.jar and place it in a folder.
   • Create the subdirectory plugin/yoursystem and place your jar there.
7. Execute openQA using java -jar openqa.jar plugin -query "my query"

What is the payload cost of the plug-in architecture?

Non ;-)...

Place openqa.engine.examples PAR file into the plug-in directory and run the framework via command line.

The difference between the time to deploying, loading and start to process the query is 0.

java -jar openqa.engine...jar plugin -query "leipzig"
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.manager.plugin.PluginManager  - Deploying existing PAR files...
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.manager.plugin.PluginManager  - Loading plug-ins...
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.main.OpenQA  - Processing query: leipzig
19:23:48 29 Oct 2015 - [main] INFO  org.aksw.openqa.AnswerFormulation  - Processing query leipzig
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Interpreting
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Number of interpretations: 1
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Interpreting runtime: 10
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Retrieving
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Number of retrieved results: 1
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Retrieval runtime: 0
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Synthesizing
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Number of synthesis: 1
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Synthesis runtime: 0
19:23:48 29 Oct 2015 - [main] DEBUG org.aksw.openqa.AnswerFormulation  - Answerformulation Runtime(ms) 10
[{"LITERAL_TEXT_PARAM": "openQA is unavailable, due to LOL"}]

How can I deploy my implemented application using openQA?

As discussed before openQA have two environments: command line and application. The application is designed for application deployment and is made by the openQA Webserver.

The openQA Webserver already supply the basics for your application such as a REST API, a UI interface as well as a control panel, but it can be extended.

It works in Tomcat, a web application container.

You can download Tomcat at http://tomcat.apache.org/.

The application plug-ins can be deployed in the openQA Webserver by following:

1) Download openQA Webserver WAR file.

2) Place the openQA Webserver WAR file into the Tomcat webapps folder and wait till Tomcat deploy it. ps: in case your Tomcat have the option auto-deployment setted to true.

3) Copy your plug-ins PAR files to the folder called 'plugins' located into the openQA webserver folder in the webapps directory.

4) Start/Restart the Tomcat.

...
  \
   Tomcat                       (Tomcat directory)
     |\
     | ...             
     |\
       webapps                  
           \
            openqa.webserver    (Web server directory)
                    |\
                    | ...
                    |\
                    | plugins   (Plugins directory)

How can I use openQA to test or evaluate my application/benchmark?

The openQA has full support for QALD (Question Answer Over Linked Data) benchmark.

1) You can use it on your code

• Instantiate your dataset or question and evaluated it:

#!Java
File qaldAnswersFile = new File(...); // Instantiate the QALD answers file
Dataset qaldAnswers = QALDBenchmark.deserialize(qaldAnswersFile);

Dataset myDataset = new Dataset();
myDataset.setId("myDataset 1"); // setting the dataset name
Question myQuestion = new Question();
...
// evaluating
QALDBenchmarkResult benchmarkResult = QALDBenchmark.evaluate(myDataset, qaldDataset);
System.out.println(benchmarkResult.toString());

2) You can use it to write unit tests

#!Java
PluginManager pluginManager = ...;
Double expectedFmeasure = ...;
Double expectedPrecision = ...;
Double expectedRecall = ...;
Question theQuestion = ...;
QALDBenchmark.evaluate(theQuestion, 
"en" /* language */, 
"string" /* question type */, 
pluginManager); // this line is dismissed if you do not want to generate the answer with openQA. 
                //You can generate the answer with your application and instantiate it here ;-) ...
Question expectedAnswer = ...;
...
// evaluating
QuestionResult questionResult = QALDBenchmark.evaluate(theQuestion, expectedAnswer);
Assert.assertEquals(expectedFmeasure, questionResult.getFmeasure());
Assert.assertEquals(expectedPrecision , questionResult.getPrecision());
Assert.assertEquals(expectedRecall, questionResult.getRecall());

3) You can evaluate your application

• Generate the answer for a particular QALD version using your openQA pipeline:

  java -jar openQA.jar //pluginDir -answer QALDBenchmarkQuestions.xml outputedSystemAnswersFileName.xml "en" "string"
  

4) You can evaluate your system questions

• Run openQA with a particular QALD version:

  java -jar openQA.jar //pluginDir -evaluate QALDBenchmarkWithAnswers.xml MYSystemAnswers.xml
  

5) You can evaluate your own benchmark

java -jar openQA.jar //pluginDir -evaluate MYBenchmarkWithAnswers.xml MYSystemAnswers.xml

Can I change the default pipeline?

Absolutely! :-)

openQA is design on a default pipeline, but it is flexible!

So you can change as you want.

For instance, the default pipeline is given by the class AnswerFormulation.defaultPipeline :

#!Java

public class DefaultPipeline extends ArrayList<Stage> {
public DefaultPipeline() {
    Stage stage0 = new Stage(); // writing stage 0
    Set<String> componentIDs = new HashSet<String>();
    stage0.setId("QueryParser"); // setting the name of the stage (ps: use your imagination)
    componentIDs.add("QueryParserProvider"); // adding component id that will be executed on the stage
    stage0.setComponentIDs(componentIDs); // setting list of components that will be executed on the stage
    add(stage0); // adding the stage to the pipe line       
       ... // repeat the process till you get bored :-)
   }
   ...
}

In order to change it you have to add the plug-in ID that you want to use in each stage, see the code above.

After setup your pipeline, you can just execute it using the AnswerFormulation class:

#!Java
String query = "How can I change the openQA pipeline?";
PluginManager pluginManager = new ...
List<Stage> myPipeline = ...
...
AnswerFormulation as = new AnswerFormulation();
QueryResult queryResult = as.process(query, pluginManager, myPipeline);

What is the PAR file and how can I generate it?

openQA plug-ins usually come with libraries and configuration file. However, this files makes the deployment, transportation and publishing a hard task.

To overcome this obstacles we design the PAR files. PAR files are openQA Plug-in ARchives, an abstraction of the openQA plug-ins.

They contain the files necessary to make the plug-in work (JAR + configuration files and/or folders).

The PAR file of a plug-in deployed (openqa.engine.examples) in the plugin directory (pluginDir) can be easily generated with openQA over a command line, by passing the directory name:

java -jar openQA.jar //pluginDir -par openqa.engine.examples

Can I change the default plug-in configuration?

Yes you can. To change the default plugin parameters you must create an ini plug-in file (e.g. org.aksw.openqa.component.answerformulation.interpreter.impl.SinaInterpreter.ini) and place it in the plug-in directory.

Another solution is copy the file in the META-INF directory and place it in the plug-in directory.

To know more about how to create a plug-in or plug-in directory please consult our example section.

The name of file should be complete, that is, should contain all the path packages.

Common errors

There are some common errors and recurring issues in openQA:

TBSL

org.h2.jdbc.JdbcSQLException: Exception opening port H2 TCP Server (tcp://192.168.1.110:60318) (port may be in use)

If you are receiving the error above in your log file, it means that the SOLR service is not up and running.

You can try to setup your own service by contacting the TBSL team or report that it is unreachable at https://github.com/AKSW/AutoSPARQL .

org.h2.jdbc.JdbcSQLException: Exception opening port H2 TCP Server (tcp://192.168.1.110:60318) (port may be in use),
cause: timeout [90061-120] at org.h2.message.Message.getSQLException(Message.java:106) at 
org.h2.message.Message.getSQLException(Message.java:117) at org.h2.tools.Server.start(Server.java:361) at 
org.h2.engine.Database.startServer(Database.java:708) at org.h2.engine.Database.open(Database.java:584) at 
org.h2.engine.Database.openDatabase(Database.java:222) at org.h2.engine.Database.<init>(Database.java:216) at 
org.h2.engine.Engine.openSession(Engine.java:58) at org.h2.engine.Engine.openSession(Engine.java:140) at 
org.h2.engine.Engine.getSession(Engine.java:120) at 
org.h2.engine.SessionFactoryEmbedded.createSession(SessionFactoryEmbedded.java:17) at 
org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:245) at 
org.h2.engine.SessionRemote.createSession(SessionRemote.java:223) at org.h2.jdbc.JdbcConnection.<init>
(JdbcConnection.java:110) at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:94) at 
org.h2.Driver.connect(Driver.java:58) at java.sql.DriverManager.getConnection(DriverManager.java:664) at 
java.sql.DriverManager.getConnection(DriverManager.java:247) at org.dllearner.kb.sparql.ExtractionDBCache.<init>
...