Commits

György Kohut  committed bc14fa9 Merge

Merge branch 'hpfeeds'

  • Participants
  • Parent commits 8165f50, 5fa7644

Comments (0)

Files changed (59)

 <classpath>
 	<classpathentry kind="src" path="src/main/java"/>
 	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
-	<classpathentry kind="lib" path="lib/slf4j-api-1.6.1.jar"/>
 	<classpathentry kind="lib" path="lib/javaee-api-6.0.jar"/>
-	<classpathentry kind="lib" path="lib/org.osgi.core-4.2.0.jar"/>
-	<classpathentry kind="lib" path="lib/org.osgi.compendium-4.2.0.jar"/>
-	<classpathentry kind="lib" path="lib/org.osgi.enterprise-4.2.0.jar"/>
-	<classpathentry kind="lib" path="lib/xadisk-1.1.jar"/>
-	<classpathentry kind="lib" path="lib/js-1.7R2.jar"/>
-	<classpathentry kind="lib" path="lib/logback-classic-0.9.28.jar"/>
-	<classpathentry kind="lib" path="lib/logback-core-0.9.28.jar"/>
+	<classpathentry kind="lib" path="lib/slf4j-api-1.6.4.jar"/>
+	<classpathentry kind="lib" path="lib/xadisk-1.2.1.jar"/>
+	<classpathentry kind="lib" path="lib2/hpfeeds-java-0.1.0.jar"/>
 	<classpathentry kind="lib" path="lib/jackson-core-asl-1.7.1.jar"/>
 	<classpathentry kind="lib" path="lib/jackson-mapper-asl-1.7.1.jar"/>
 	<classpathentry kind="lib" path="lib/concurrentlinkedhashmap-lru-1.2.jar"/>
 lib/*
 target/*
+conf/local.conf
 <?xml version="1.0" encoding="UTF-8"?>
 <projectDescription>
-	<name>NewProject</name>
+	<name>hbbackend</name>
 	<comment></comment>
 	<projects>
 	</projects>

File .settings/org.eclipse.core.resources.prefs

File contents unchanged.

File .settings/org.eclipse.jdt.core.prefs

File contents unchanged.
 == Requirements ==
 
-* JDK 6 Update 24+
-* GlassFish 3.1.1b10
-* PostgreSQL 9.0.4
+* JDK 6+
+* GlassFish 3.1.1
+* PostgreSQL 9+
 
 == Compiling ==
 
 * Requirements:
- * Apache Ant 1.8.2
+ * Apache Ant 1.8+
  * Apache Ivy 2.2  
 
-1. Retrieve project dependencies:
+
+1. Retrieve the project dependencies:
 ant retrieve
 
-2. If the default compile time configuration properties in conf/default.conf won't suffice, copy the file
-(e.g. to conf/local.conf) and set them. Set the config.file property when running build targets to specify the custom
-configuration file (e.g. ant -Dconfig.file=conf/local.conf).
+2. There are additional dependencies that currently cannot be downloaded using Ivy and must be placed into the directory
+'lib2/' manually.
 
-3. Build all modules simply by running ant. The modules are placed in target/modules.
+ 1. Download http://jdbc.postgresql.org/download/postgresql-9.1-902.jdbc4.jar
+ 2. Download https://bitbucket.org/gyoergy/hpfeeds-java into some other directory and compile it by running
+ 'mvn package'. Copy the resulting .jar 'target/hpfeeds-java-0.1.0.jar'.
 
-== Modules ==
+3. The file 'conf/default.conf' contains the default compile time configuration. Copy it to e.g. 'conf/local.conf' and
+customize the values. ('conf/local.conf' is in .gitignore.)
 
-Currently, the functional modules are:
+4. Finally, build the modules by running e.g. 'ant -Dconfig.file=conf/local.conf'. The modules are placed into target/modules.
 
-submithttp - submit_http handler
-maxmindgeoip - looks up geo IP information for every submit_http submission using the MaxMind GeoLite City database [1]
-ipfilter - provides unique IPs for other modules
-cymruwhois - Team Cymru bulk mode whois query [2]
-shadowserverdns - Shadowserver DNS origin query [3]
-virustotal - VirusTotal API file report query
 
-[1] http://www.maxmind.com/app/geolitecity
-[2] http://www.team-cymru.org/Services/ip-to-asn.html
-[3] http://www.shadowserver.org/wiki/pmwiki.php/Services/IP-BGP
+== Modules ==
 
-The modules are interconnected by the following schema. All destinations are topics, all consumers are durable
-subscribers.
+The current modules are:
 
-submithttp out: new_attack, new_binary
-maxmindgeoip in: new_attack out: new_attack_geoip
-ipfilter in: new_attack out: new_ip
-cymruwhois in: new_ip
-shadowserverdns in: new_ip
-virustotal in: new_binary
+hpfeedsra - resource adapter providing interfacing with hpfeeds for modules
+hpfeeds - hpfeeds submission handler intended for the channels dionaea.capture and mwbinary.dionaea.sensorunique
 
-For simplicity, the modules are configured in compile time by the properties in conf/default.conf. Properties that are
-not specific to a particular module are prefixed with "main", otherwise module specific options are prefixed with the
-module's name.
 
 == Initial Setup ==
 
-1. Create the directory structure. It is recommended to use a similar to the following one. E.g.:
+1. Create the directory structure similar to the following one:
+E.g.:
 
 mkdir /tmp/hbbackend/
-mkdir /tmp/hbbackend/main/ # the main storage directory for binaries and corresponding files (main.storagedir)
-mkdir /tmp/hbbackend/upload/ # for work in progress file uploads
+mkdir /tmp/hbbackend/main/ # the main storage directory for binaries and corresponding files ('main.storagedir')
 mkdir /tmp/hbbackend/geoip/ # for geo IP databases
 mkdir /tmp/hbbackend/log/ # logs
-mkdir /tmp/hbbackend/conf/ # for configurations files run time configuration files (logger configuration)
-mkdir /tmp/hbbackend/xadisk/ # XADisk's working directory
+mkdir /tmp/hbbackend/conf/ # for run time configuration files (logger configuration)
+mkdir /tmp/hbbackend/xadisk/ # XADisk working directory
 
-2. Create the database by executing schema/hbbackend.sql (tables) and schema/hbbackend_functions.sql (functions and
-triggers). E.g.:
+2. Create the database by executing 'schema/hbbackend.sql' (tables) and 'schema/hbbackend_functions.sql' (functions).
+E.g.:
 
 psql < schema/hbbackend.sql
 psql < schema/hbbackend_functions.sql
 wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
 gunzip GeoLiteCity.dat.gz
 
-4. Create GlassFish domain. Throughout the following steps, the name "hbbackend" with a port base of 9990 will be
+4. Create the GlassFish domain. Throughout the following steps, the name 'hbbackend' and the port base 9900 will be
 used. E.g.:
 
 asadmin create-domain --portbase 9900 hbbackend
 
-5. Copy library dependencies into the domain library directory.
+5. Copy the dependencies into the domain library directory. E.g.:
 
-cp lib/postgresql-9.0-*.jdbc4.jar \
-	lib/xadisk-*.jar \
-	lib/slf4j-api-1.6.*.jar \
-	lib/logback-core-0.9.*.jar \
-	lib/logback-classic-0.9.*.jar\
-	lib/concurrentlinkedhashmap-lru-1.2.jar \
+cp lib2/postgresql-9.1-902.jdbc4.jar \
+   lib2/hpfeeds-java-0.1.0.jar \
+   lib/xadisk-*.jar \
+   lib/slf4j-api-1.6.*.jar \
+   lib/logback-core-1.0.*.jar \
+   lib/logback-classic-1.0.*.jar\
+   lib/concurrentlinkedhashmap-lru-1.2.jar \
 <path>/glassfish3/glassfish/domains/hbbackend/lib/
 
-6. Copy the logback configuration file conf/logback.xml to e.g. /tmp/hbbackend/conf/ and customize the log destination
-files properties if the defaults won't suffice.
+6. Copy the Logback configuration file conf/logback.xml to e.g. /tmp/hbbackend/conf/ and customize the property
+'LOG_DIR' to point to the appropriate directory for the logs. E.g.:
 
 cp conf/logback.xml /tmp/hbbackend/conf/
 
-7. Start asaddmin with the appropriate port (e.g. asadmin --port 9948) and run the following commands to set up the
-domain including JMS resources. All following commands assume values given in the above, otherwise default values. The
-PostgreSQL role is assumed to be "hbbackend" with the password "hbbackend".
+7. Start 'asadmin' with the appropriate port (e.g. 'asadmin --port 9948') and run the following commands to set up the
+domain including JMS resources. All commands refer to example values (paths) from above, so it will probably be
+necessary to edit them too before executing. The PostgreSQL role is assumed to be 'hbbackend' with the password
+'hbbackend'. The hpfeeds configuration for hpfeedsra needs to be filled in.
 
 # start
 start-domain hbbackend
 # jdbc resource
 create-jdbc-resource --connectionpoolid PgPool jdbc/hbbackend
 
+## thread pools
+create-threadpool --minthreadpoolsize=5 --maxthreadpoolsize=50 xadisk-thread-pool
+create-threadpool --minthreadpoolsize=16 --maxthreadpoolsize=16 hpfeedsra-thread-pool
+# must restart domain for thread pools to become available
+restart-domain hbbackend
+
 ## xadisk
-create-threadpool --minthreadpoolsize=5 --maxthreadpoolsize=50 hbbackend-xadisk-thread-pool
-# must restart domain for the thread pool to become available
-restart-domain hbbackend 
-create-resource-adapter-config --threadpoolid hbbackend-xadisk-thread-pool --property xaDiskHome=/tmp/hbbackend/xadisk:instanceId=hbbackend xadisk
-deploy --name xadisk lib/xadisk-1.1.rar
+create-resource-adapter-config --threadpoolid xadisk-thread-pool --property xaDiskHome=/tmp/hbbackend/xadisk:instanceId=hbbackend xadisk
+deploy --name xadisk lib/xadisk-1.2.1.rar
 create-connector-connection-pool --raname xadisk --connectiondefinition org.xadisk.connector.outbound.XADiskConnectionFactory --property instanceId=hbbackend --transactionsupport XATransaction xadisk/ConnectionFactory
 ping-connection-pool xadisk/ConnectionFactory
 create-connector-resource --poolname xadisk/ConnectionFactory xadisk/ConnectionFactory
 set configs.config.server-config.jms-service.jms-host.default_JMS_host.property.imq\\.autocreate\\.topic=false
 # destinations
 create-jmsdest --desttype topic new_attack
-create-jmsdest --desttype topic new_attack_geoip
 create-jmsdest --desttype topic new_binary
 create-jmsdest --desttype topic new_ip
 create-jms-resource --restype javax.jms.Topic --property Name=new_attack jms/new_attack
-create-jms-resource --restype javax.jms.Topic --property Name=new_attack_geoip jms/new_attack_geoip
 create-jms-resource --restype javax.jms.Topic --property Name=new_binary jms/new_binary
 create-jms-resource --restype javax.jms.Topic --property Name=new_ip jms/new_ip
-# connection factories
+# connection factory
 create-jms-resource --restype javax.jms.ConnectionFactory jms/ConnectionFactory
-create-jms-resource --restype javax.jms.ConnectionFactory --property ClientId=testdurable jms/TestDurableConnectionFactory
-create-jms-resource --restype javax.jms.ConnectionFactory --property ClientId=ipfilter jms/DurableConsumer/ipfilter
-create-jms-resource --restype javax.jms.ConnectionFactory --property ClientId=cymruwhois jms/DurableConsumer/cymruwhois
-create-jms-resource --restype javax.jms.ConnectionFactory --property ClientId=shadowserverdns jms/DurableConsumer/shadowserverdns
-create-jms-resource --restype javax.jms.ConnectionFactory --property ClientId=maxmindgeoip jms/DurableConsumer/maxmindgeoip
-create-jms-resource --restype javax.jms.ConnectionFactory --property ClientId=virustotal jms/DurableConsumer/virustotal
 ping-connection-pool jms/ConnectionFactory
 
 # logback config location
 # monitoring
 enable-monitoring --modules connector-connection-pool=HIGH:connector-service=HIGH:deployment=HIGH:ejb-container=HIGH:http-service=HIGH:jdbc-connection-pool=HIGH:jms-service=HIGH:jvm=HIGH:thread-pool=HIGH:transaction-service=HIGH:jms-service=HIGH:web-container=HIGH
 
+# hpfeeds config
+create-resource-adapter-config --threadpoolid hpfeedsra-thread-pool --property host=hpfeeds.honeycloud.net:port=10000:ident=IDENT:secret=SECRET:channels=dionaea.capture,mwbinary.dionaea.sensorunique  org.honeynet.hbbackend.hpfeedsra
+
 # finally, restart once again for a clean start
 restart-domain hbbackend
 
+
 == Deploying Modules ==
 
-Generally, the order of deployment should be the reverse of the flow of the system. Consumers of a destination should be
-deployed before producers to avoid losing messages. For durable consumers it's only critical at the first deployment,
-but following the rule is always a safe bet. Accordingly, submithttp should be the last in the row. When deploying for
-the first time, check the broker that all durable subscriptions are created before deploying submithttp.
+Generally, the order of deployment should be the opposite of the flow of the system. Consumers should be deployed before
+producers to avoid losing messages. (For durable consumers, it's only critical at the first deployment, but following
+the rule is always a safe bet.) The exception is hpfeedsra as it listens for hpfeeds worker modules, so it needs to
+already be resident when those are deployed.
 
 E.g.:
 
-deploy target/modules/org.honeynet.hbbackend.virustotal.jar
-deploy target/modules/org.honeynet.hbbackend.cymruwhois.jar
-deploy target/modules/org.honeynet.hbbackend.shadowserverdns.jar
-deploy target/modules/org.honeynet.hbbackend.ipfilter.jar
-deploy target/modules/org.honeynet.hbbackend.maxmindgeoip.jar
-deploy target/modules/org.honeynet.hbbackend.submithttp.war
+deploy target/modules/org.honeynet.hbbackend.hpfeedsra.rar
+deploy target/modules/org.honeynet.hbbackend.hpfeeds.jar
+
+NOTE: currently, the code is scaled back, so there's only hpfeeds submission handling and no consumers mentioned above.
+
+
+== Undeploying Modules ==
+
+The order should be the opposite of the order of the deployment with the exception that this time, it needs to be
+considered that hpfeeds worker modules are consumers of hpfeedsra, which means it's safest to undeploy hpfeedsra first.
+
+E.g.:
+
+undeploy org.honeynet.hbbackend.hpfeedsra
+undeploy org.honeynet.hbbackend.hpfeeds
+
+NOTE: currently, hpfeedsra must always be undeployed before hpfeeds, as disconnecting hpfeeds worker modules gracefully is
+work in progress.
 
 <project name="hbbackend" default="build"
 	 xmlns:ivy="antlib:org.apache.ivy.ant">
     <property name="lib.dir" value="lib"/>
+    <property name="lib2.dir" value="lib2"/>
     <property name="target.dir" value="target"/>
     <property name="target.classes.dir" value="${target.dir}/classes"/>
     <property name="target.modules.dir" value="${target.dir}/modules"/>
 
     <path id="lib.path.id">
       <fileset dir="${lib.dir}"/>
+      <fileset dir="${lib2.dir}"/>
     </path>
     
 
 
 
     <target name="build"
-	    depends="build-submithttp,
-		     build-ipfilter,
-		     build-cymruwhois,
-		     build-virustotal,
-		     build-shadowserverdns,
-		     build-maxmindgeoip
+	    depends="build-hpfeedsra,
+		     build-hpfeeds
 		     "
 	    description="--> build all modules"
     />
 
 
 
-    <target name="build-submithttp" depends="compile" description="module">
-      <package-war name="submithttp">
-	<embed>
-	  <embed-package package="common.storage"/>
-	  <embed-package package="common.types.common"/>
-	  <embed-package package="common.types.submithttp"/>
-	</embed>
-      </package-war>
-    </target>
-
-    <target name="build-ipfilter" depends="compile" description="module">
-      <package-ejb name="ipfilter">
-	<embed>
-	  <!-- <embed-package package="common"/> -->
-	</embed>
-      </package-ejb>
-    </target>
-
-    <target name="build-cymruwhois" depends="compile" description="module">
-      <package-ejb name="cymruwhois">
-	<embed>
-	  <!-- <embed-package package="common"/> -->
-	</embed>
-      </package-ejb>
+    <target name="build-hpfeedsra" depends="compile" description="resource adapter">
+      <package-rar name="hpfeedsra"/>
     </target>
 
-    <target name="build-shadowserverdns" depends="compile" description="module">
-      <package-ejb name="shadowserverdns">
+    <target name="build-hpfeeds" depends="compile" description="module">
+      <package-ejb name="hpfeeds">
 	<embed>
 	  <!-- <embed-package package="common"/> -->
 	</embed>
       </package-ejb>
     </target>
 
-    <target name="build-maxmindgeoip" depends="compile" description="module">
-      <package-ejb name="maxmindgeoip">
-	<embed>
-	  <embed-package package="common.types.common"/>
-	  <embed-package package="common.types.maxmindgeoip"/>
-	  <embed-package package="geoip" parent="com.maxmind"/>
-	</embed>
-      </package-ejb>
-    </target>
-
-    <target name="build-virustotal" depends="compile" description="module">
-      <package-ejb name="virustotal">
-	<embed>
-	  <!-- <embed-package package="common"/> -->
-	</embed>
-      </package-ejb>
-    </target>
 
 
 

File conf/default.conf

 main.storagedir = /tmp/hbbackend/main
 
-submithttp.uploaddir = /tmp/hbbackend/upload
-virustotal.apikey = bc2f65f325d26891a7cad544323189ebd922e7bb0bbf5fca47851b879c369fc8
-maxmindgeoip.dbfile = /tmp/hbbackend/geoip/GeoLiteCity.dat
+hpfeeds.attack.channel = dionaea.capture
+hpfeeds.binary.channel = mwbinary.dionaea.sensorunique
+#hpfeeds.binary.channel = honeeebox.binary.test
+
+maxmindgeoip.dbfile = /tmp/geoip/GeoLiteCity.dat

File conf/logback.xml

 <configuration>
 
-  <property name="DEST_FILE-hbbackend" value="/tmp/hbbackend/log/hbbackend.log"/>
-  <property name="DEST_FILE-hbbackend.submithttp" value="/tmp/hbbackend/log/hbbackend.submithttp.log"/>
+  <property name="LOG_DIR" value="/tmp/hbbackend/log"/>
+
+  <property name="DEST_FILE-hbbackend" value="${LOG_DIR}/hbbackend.log"/>
+  <property name="DEST_FILE-hbbackend.hpfeeds" value="${LOG_DIR}/hbbackend.hpfeeds.log"/>
+  <property name="DEST_FILE-hbbackend-DEBUG" value="${LOG_DIR}/hbbackend.debug.log"/>
+  <property name="DEST_FILE-hbbackend.hpfeeds-DEBUG" value="${LOG_DIR}/hbbackend.hpfeeds.debug.log"/>
   
   <appender name="STDOUT"
    class="ch.qos.logback.core.ConsoleAppender">
   <appender name="FILE-hbbackend"
     class="ch.qos.logback.core.FileAppender">
     <file>${DEST_FILE-hbbackend}</file>
+    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
+      <level>INFO</level>
+    </filter>
+    <encoder>
+      <pattern>%d %-5level %logger{36} - %msg%n</pattern>
+    </encoder>
+  </appender>
+
+  <appender name="FILE-hbbackend-DEBUG"
+    class="ch.qos.logback.core.FileAppender">
+    <file>${DEST_FILE-hbbackend-DEBUG}</file>
     <encoder>
       <pattern>%d [%thread] %-5level %logger{36} - %msg%n</pattern>
     </encoder>
   </appender>
 
-  <appender name="FILE-hbbackend.submithttp"
+  <appender name="FILE-hbbackend.hpfeeds"
+    class="ch.qos.logback.core.FileAppender">
+    <file>${DEST_FILE-hbbackend.hpfeeds}</file>
+    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
+      <level>INFO</level>
+    </filter>
+    <encoder>
+      <pattern>%d %-5level %logger{1} - %msg%n</pattern>
+    </encoder>
+  </appender>
+
+  <appender name="FILE-hbbackend.hpfeeds-DEBUG"
     class="ch.qos.logback.core.FileAppender">
-    <file>${DEST_FILE-hbbackend.submithttp}</file>
+    <file>${DEST_FILE-hbbackend.hpfeeds-DEBUG}</file>
     <encoder>
       <pattern>%d [%thread] %-5level %logger{36} - %msg%n</pattern>
     </encoder>
   </appender>
 
   <logger name="org.honeynet.hbbackend" additivity="false">
-    <level value="trace"/>
+    <level value="info"/>
     <appender-ref ref="FILE-hbbackend"/>
+    <!-- <appender-ref ref="FILE-hbbackend-DEBUG"/> -->
   </logger>
 
-  <logger name="org.honeynet.hbbackend.submithttp" additivity="false">
-    <appender-ref ref="FILE-hbbackend.submithttp"/>
+  <logger name="org.honeynet.hbbackend.hpfeeds" additivity="false">
+    <level value="info"/>
+    <appender-ref ref="FILE-hbbackend.hpfeeds"/>
+    <!-- <appender-ref ref="FILE-hbbackend.hpfeeds-DEBUG"/> -->
   </logger>
   
   <root>
 
     <dependencies>
       <dependency org="javax" name="javaee-api" rev="6.0" conf="build->default" transitive="false"/>
-      <dependency org="org.osgi" name="org.osgi.core" rev="4.2.0" conf="build->default" transitive="false"/>
-      <dependency org="org.osgi" name="org.osgi.compendium" rev="4.2.0" conf="build->default" transitive="false"/>
-      <dependency org="org.osgi" name="org.osgi.enterprise" rev="4.2.0" conf="build->default" transitive="false"/>
 
-      <dependency org="org.slf4j" name="slf4j-api" rev="1.6.1" conf="build->default" transitive="false"/>
-      <dependency org="ch.qos.logback" name="logback-core" rev="0.9.28" conf="build->default" transitive="false"/>
-      <dependency org="ch.qos.logback" name="logback-classic" rev="0.9.28" conf="build->default" transitive="false"/>
-      <!-- <dependency org="org.slf4j" name="slf4j-jdk14" rev="1.6.1" conf="build->default" transitive="false"/> -->
+      <dependency org="org.slf4j" name="slf4j-api" rev="1.6.4" conf="build->default" transitive="false"/>
+      <dependency org="ch.qos.logback" name="logback-core" rev="1.0.3" conf="build->default" transitive="false"/>
+      <dependency org="ch.qos.logback" name="logback-classic" rev="1.0.3" conf="build->default" transitive="false"/>
+      <!-- <dependency org="org.slf4j" name="slf4j-jdk14" rev="1.6.4" conf="build->default" transitive="false"/> -->
 
-      <dependency org="postgresql" name="postgresql" rev="9.0-801.jdbc4" conf="build->default" transitive="false"/>
+      <!-- <dependency org="postgresql" name="postgresql" rev="9.0-801.jdbc4" conf="build->default" transitive="false"/> -->
 
-      <dependency org="net.java.xadisk" name="xadisk" rev="1.1" conf="build->default" transitive="false">
+      <dependency org="net.java.xadisk" name="xadisk" rev="1.2.1" conf="build->default" transitive="false">
 	<artifact name="xadisk" type="jar"/>
 	<artifact name="xadisk" type="rar"/>
       </dependency>
 
       <dependency org="com.googlecode.concurrentlinkedhashmap" name="concurrentlinkedhashmap-lru" rev="1.2" conf="build->default" transitive="false"/>
 
-      <dependency org="rhino" name="js" rev="1.7R2" conf="build->default" transitive="false"/>
-
-      <!-- <dependency org="org.codehaus.groovy" name="groovy-all" rev="1.8.0" conf="build->default" transitive="false"/> -->
-      <!-- <dependency org="org.fusesource.jansi" name="jansi" rev="1.2.1" conf="build->default" transitive="false"/> -->
-      <!-- <dependency org="jline" name="jline" rev="0.9.94" conf="build->default" transitive="false"/> -->
-      <!-- <dependency org="commons-fileupload" name="commons-fileupload" rev="1.1.1" conf="build->default" transitive="false"/> -->
-
-      <!-- <dependency org="org.glassfish.grizzly" name="grizzly-framework" rev="2.1.1" conf="build->default" transitive="false"> -->
-      <!-- 	<artifact name="grizzly-framework" type="jar"/> -->
-      <!-- 	<artifact name="grizzly-framework" type="sources" ext="jar"/> -->
-      <!-- </dependency> -->
-      <!-- <dependency org="org.glassfish.grizzly" name="grizzly-http" rev="2.1.1" conf="build->default" transitive="false"> -->
-      <!-- 	<artifact name="grizzly-http" type="jar"/> -->
-      <!-- 	<artifact name="grizzly-http" type="sources" ext="jar"/> -->
-      <!-- </dependency> -->
-      <!-- <dependency org="org.glassfish.grizzly" name="grizzly-http-server" rev="2.1.1" conf="build->default" transitive="false"> -->
-      <!-- 	<artifact name="grizzly-http-server" type="jar"/> -->
-      <!-- 	<artifact name="grizzly-http-server" type="sources" ext="jar"/> -->
-      <!-- </dependency> -->
-      <!-- <dependency org="org.glassfish.grizzly" name="grizzly-rcm" rev="2.1.1" conf="build->default" transitive="false"> -->
-      <!-- 	<artifact name="grizzly-rcm" type="jar"/> -->
-      <!-- </dependency> -->
-      <!-- <dependency org="org.glassfish.grizzly" name="grizzly-http-server-multipart" rev="2.1.1" conf="build->default" transitive="false"> -->
-      <!-- 	<artifact name="grizzly-http-server-multipart" type="jar"/> -->
-      <!-- 	<artifact name="grizzly-http-server-multipart" type="sources" ext="jar"/> -->
-      <!-- </dependency> -->
-      <!-- <dependency org="org.glassfish.gmbal" name="gmbal-api-only" rev="3.1.0-b001" conf="build->default" transitive="false"/> -->
-
-      <!-- <dependency org="org.springframework" name="org.springframework.core" rev="3.0.5.RELEASE" conf="build->runtime" transitive="false"/> -->
-      <!-- <dependency org="org.springframework" name="org.springframework.context" rev="3.0.5.RELEASE" conf="build->runtime" transitive="false"/> -->
-      <!-- <dependency org="org.springframework" name="org.springframework.beans" rev="3.0.5.RELEASE" conf="build->runtime" transitive="false"/> -->
-      <!-- <dependency org="org.springframework" name="org.springframework.expression" rev="3.0.5.RELEASE" conf="build->runtime" transitive="false"/> -->
-      <!-- <dependency org="org.springframework" name="org.springframework.asm" rev="3.0.5.RELEASE" conf="build->runtime" transitive="false"/> -->
-
-      <!-- <dependency org="org.springframework" name="org.springframework.aop" rev="3.0.5.RELEASE" conf="build->runtime" transitive="false"/> -->
-      <!-- <dependency org="org.aopalliance" name="com.springsource.org.aopalliance" rev="1.0.0" conf="build->runtime" transitive="false"/> -->
-      <!-- <dependency org="org.eclipse.gemini.blueprint" name="gemini-blueprint-core" rev="1.0.0.M1" conf="build->default" transitive="false"/> -->
-      <!-- <dependency org="org.eclipse.gemini.blueprint" name="gemini-blueprint-extender" rev="1.0.0.M1" conf="build->default" transitive="false"/> -->
-      <!-- <dependency org="org.eclipse.gemini.blueprint" name="gemini-blueprint-io" rev="1.0.0.M1" conf="build->default" transitive="false"/> -->
-
-      <!-- <dependency org="com.springsource.bundlor" name="com.springsource.bundlor.ant" rev="1.0.0.RELEASE" conf="buildlib->ant" transitive="true"/> -->
-
-      <!-- <dependency org="com.springsource.bundlor" name="com.springsource.bundlor" rev="1.0.0.RELEASE" conf="bundlor->default" transitive="false"/> -->
-      <!-- <dependency org="com.springsource.bundlor" name="com.springsource.bundlor.blint" rev="1.0.0.RELEASE" conf="bundlor->default" transitive="false"/> -->
-
-      <!-- <dependency org="org.codehaus.groovy" name="groovy" rev="1.8.0" conf="buildlib->default" transitive="false"/> -->
-      <!-- <dependency org="org.codehaus.gant" name="gant_groovy1.8" rev="1.9.5" conf="buildlib->default" transitive="false"/> -->
 
     </dependencies>
 </ivy-module>

File ivysettings.xml

   <resolvers>
     <chain name="custom">
 
-      <url name="glassfish" m2compatible="true">
-	<artifact pattern="http://download.java.net/maven/glassfish/[organisation]/[module]/[revision]/[artifact]-[revision](-[type]).[ext]"/>
-	<artifact pattern="http://download.java.net/maven/glassfish/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/>
-      </url>
+      <!-- <url name="glassfish" m2compatible="true"> -->
+      <!-- 	<artifact pattern="http://download.java.net/maven/glassfish/[organisation]/[module]/[revision]/[artifact]-[revision](-[type]).[ext]"/> -->
+      <!-- 	<artifact pattern="http://download.java.net/maven/glassfish/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/> -->
+      <!-- </url> -->
 
       <url name="javanet" m2compatible="true">
 	<artifact pattern="http://download.java.net/maven/2/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
 
       <ibiblio name="public" m2compatible="true"/>
 
-      <!-- <url name="com.springsource.repository.bundles.release"> -->
-      <!-- 	<ivy pattern="http://repository.springsource.com/ivy/bundles/release/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" /> -->
-      <!-- 	<artifact pattern="http://repository.springsource.com/ivy/bundles/release/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" /> -->
-      <!-- </url> -->
-      
-      <!-- <url name="com.springsource.repository.bundles.external"> -->
-      <!-- 	<ivy pattern="http://repository.springsource.com/ivy/bundles/external/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" /> -->
-      <!-- 	<artifact pattern="http://repository.springsource.com/ivy/bundles/external/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" /> -->
-      <!-- </url> -->
-
-      <!-- <url name="zodiac-repository" m2compatible="true"> -->
-      <!-- 	<artifact pattern="http://zodiac.springsource.com/maven/bundles/milestone/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" /> -->
-      <!-- </url> -->
 
     </chain>
   </resolvers>

File lib2/.gitignore

+*
+!.gitignore

File misc/mqpassfile

File contents unchanged.

File schema/hbbackend.erm

 	</page_setting>
 	<category_index>0</category_index>
 	<zoom>1.0</zoom>
-	<x>13</x>
-	<y>0</y>
+	<x>0</x>
+	<y>154</y>
 	<default_color>
 		<r>128</r>
 		<g>128</g>
 		<notation_expand_group>true</notation_expand_group>
 		<view_mode>1</view_mode>
 		<outline_view_mode>1</outline_view_mode>
+		<view_order_by>0</view_order_by>
 		<auto_ime_change>false</auto_ime_change>
 		<validate_physical_name>true</validate_physical_name>
 		<use_bezier_curve>false</use_bezier_curve>
 			<image_output></image_output>
 			<put_diagram_on_excel>false</put_diagram_on_excel>
 			<use_logical_name_as_sheet>false</use_logical_name_as_sheet>
+			<open_after_saved>false</open_after_saved>
 			<create_comment>true</create_comment>
 			<create_foreignKey>true</create_foreignKey>
 			<create_index>true</create_index>
 			<id></id>
 			<height>-1</height>
 			<width>-1</width>
-				<font_name>Lucida Grande</font_name>
+				<font_name>Sans</font_name>
 				<font_size>9</font_size>
 			<x>50</x>
 			<y>50</y>
 			</connections>
 			<display>false</display>
 			<creation_date>2011-06-10 22:42:09</creation_date>
-			<updated_date>2011-07-15 19:20:52</updated_date>
+			<updated_date>2012-05-27 00:01:18</updated_date>
 			<model_property>
 				<name>Project Name</name>
 				<value></value>
 		</word>
 		<word>
 			<id>17</id>
-			<length>null</length>
+			<length>16</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<array_dimension>null</array_dimension>
 			<unsigned>false</unsigned>
 			<args></args>
 			<description></description>
-			<logical_name>id</logical_name>
-			<physical_name>id</physical_name>
-			<type>bigserial</type>
+			<logical_name>ident</logical_name>
+			<physical_name>ident</physical_name>
+			<type>varchar(n)</type>
 		</word>
 		<word>
 			<id>18</id>
 			<unsigned>false</unsigned>
 			<args></args>
 			<description></description>
-			<logical_name>isp</logical_name>
-			<physical_name>isp</physical_name>
-			<type>long</type>
+			<logical_name>ip</logical_name>
+			<physical_name>ip</physical_name>
+			<type>inet</type>
 		</word>
 		<word>
 			<id>20</id>
 			<unsigned>false</unsigned>
 			<args></args>
 			<description></description>
-			<logical_name>label</logical_name>
-			<physical_name>label</physical_name>
-			<type>long</type>
+			<logical_name>ip</logical_name>
+			<physical_name>ip</physical_name>
+			<type>inet</type>
 		</word>
 		<word>
 			<id>21</id>
-			<length>32</length>
+			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<array_dimension>null</array_dimension>
 			<unsigned>false</unsigned>
 			<args></args>
 			<description></description>
-			<logical_name>md5</logical_name>
-			<physical_name>md5</physical_name>
-			<type>character(n)</type>
+			<logical_name>isp</logical_name>
+			<physical_name>isp</physical_name>
+			<type>long</type>
 		</word>
 		<word>
 			<id>22</id>
 			<unsigned>false</unsigned>
 			<args></args>
 			<description></description>
-			<logical_name>permalink</logical_name>
-			<physical_name>permalink</physical_name>
+			<logical_name>label</logical_name>
+			<physical_name>label</physical_name>
 			<type>long</type>
 		</word>
 		<word>
 			<id>23</id>
-			<length>null</length>
+			<length>32</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<array_dimension>null</array_dimension>
 			<unsigned>false</unsigned>
 			<args></args>
 			<description></description>
-			<logical_name>registered_time</logical_name>
-			<physical_name>registered_time</physical_name>
-			<type>timestamp with time zone</type>
+			<logical_name>md5</logical_name>
+			<physical_name>md5</physical_name>
+			<type>character(n)</type>
 		</word>
 		<word>
 			<id>24</id>
 			<unsigned>false</unsigned>
 			<args></args>
 			<description></description>
+			<logical_name>permalink</logical_name>
+			<physical_name>permalink</physical_name>
+			<type>long</type>
+		</word>
+		<word>
+			<id>25</id>
+			<length>null</length>
+			<decimal>null</decimal>
+			<array>false</array>
+			<array_dimension>null</array_dimension>
+			<unsigned>false</unsigned>
+			<args></args>
+			<description></description>
 			<logical_name>registry</logical_name>
 			<physical_name>registry</physical_name>
 			<type>long</type>
 		</word>
 		<word>
-			<id>25</id>
+			<id>26</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>datetime</type>
 		</word>
 		<word>
-			<id>26</id>
+			<id>27</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>integer</type>
 		</word>
 		<word>
-			<id>27</id>
+			<id>28</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>datetime</type>
 		</word>
 		<word>
-			<id>28</id>
+			<id>29</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>timestamp with time zone</type>
 		</word>
 		<word>
-			<id>29</id>
+			<id>30</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>timestamp with time zone</type>
 		</word>
 		<word>
-			<id>30</id>
+			<id>31</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>bigint</type>
 		</word>
 		<word>
-			<id>31</id>
+			<id>32</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>bigint</type>
 		</word>
 		<word>
-			<id>32</id>
+			<id>33</id>
+			<length>null</length>
+			<decimal>null</decimal>
+			<array>false</array>
+			<array_dimension>null</array_dimension>
+			<unsigned>false</unsigned>
+			<args></args>
+			<description></description>
+			<logical_name>seen_count</logical_name>
+			<physical_name>seen_count</physical_name>
+			<type>bigint</type>
+		</word>
+		<word>
+			<id>34</id>
+			<length>null</length>
+			<decimal>null</decimal>
+			<array>false</array>
+			<array_dimension>null</array_dimension>
+			<unsigned>false</unsigned>
+			<args></args>
+			<description></description>
+			<logical_name>seen_count</logical_name>
+			<physical_name>seen_count</physical_name>
+			<type>bigint</type>
+		</word>
+		<word>
+			<id>35</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>timestamp with time zone</type>
 		</word>
 		<word>
-			<id>33</id>
+			<id>36</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>timestamp with time zone</type>
 		</word>
 		<word>
-			<id>34</id>
+			<id>37</id>
+			<length>null</length>
+			<decimal>null</decimal>
+			<array>false</array>
+			<array_dimension>null</array_dimension>
+			<unsigned>false</unsigned>
+			<args></args>
+			<description></description>
+			<logical_name>seen_first</logical_name>
+			<physical_name>seen_first</physical_name>
+			<type>timestamp with time zone</type>
+		</word>
+		<word>
+			<id>38</id>
+			<length>null</length>
+			<decimal>null</decimal>
+			<array>false</array>
+			<array_dimension>null</array_dimension>
+			<unsigned>false</unsigned>
+			<args></args>
+			<description></description>
+			<logical_name>seen_first</logical_name>
+			<physical_name>seen_first</physical_name>
+			<type>timestamp with time zone</type>
+		</word>
+		<word>
+			<id>39</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>timestamp with time zone</type>
 		</word>
 		<word>
-			<id>35</id>
+			<id>40</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>timestamp with time zone</type>
 		</word>
 		<word>
-			<id>36</id>
+			<id>41</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<unsigned>false</unsigned>
 			<args></args>
 			<description></description>
-			<logical_name>sensor_ip</logical_name>
-			<physical_name>sensor_ip</physical_name>
-			<type>inet</type>
+			<logical_name>seen_last</logical_name>
+			<physical_name>seen_last</physical_name>
+			<type>timestamp with time zone</type>
 		</word>
 		<word>
-			<id>37</id>
+			<id>42</id>
+			<length>null</length>
+			<decimal>null</decimal>
+			<array>false</array>
+			<array_dimension>null</array_dimension>
+			<unsigned>false</unsigned>
+			<args></args>
+			<description></description>
+			<logical_name>seen_last</logical_name>
+			<physical_name>seen_last</physical_name>
+			<type>timestamp with time zone</type>
+		</word>
+		<word>
+			<id>43</id>
 			<length>128</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>character(n)</type>
 		</word>
 		<word>
-			<id>38</id>
+			<id>44</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>inet</type>
 		</word>
 		<word>
-			<id>39</id>
+			<id>45</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>integer</type>
 		</word>
 		<word>
-			<id>40</id>
+			<id>46</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>boolean</type>
 		</word>
 		<word>
-			<id>41</id>
+			<id>47</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>timestamp with time zone</type>
 		</word>
 		<word>
-			<id>42</id>
+			<id>48</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>inet</type>
 		</word>
 		<word>
-			<id>43</id>
+			<id>49</id>
 			<length>null</length>
 			<decimal>null</decimal>
 			<array>false</array>
 			<type>integer</type>
 		</word>
 		<word>
-			<id>44</id>
-			<length>128</length>
-			<decimal>null</decimal>
-			<array>false</array>
-			<array_dimension>null</array_dimension>
-			<unsigned>false</unsigned>
-			<args></args>
-			<description></description>
-			<logical_name>trig</logical_name>
-			<physical_name>trig</physical_name>
-			<type>varchar(n)</type>
-		</word>
-		<word>
-			<id>45</id>
+			<id>50</id>
 			<length>128</length>
 			<decimal>null</decimal>
 			<array>false</array>
 	<contents>
 		<table>
 			<id>0</id>
-			<height>112</height>
-			<width>235</width>
+			<height>107</height>
+			<width>222</width>
 				<font_name>Lucida Grande</font_name>
 				<font_size>9</font_size>
-			<x>47</x>
-			<y>260</y>
+			<x>1119</x>
+			<y>102</y>
 			<color>
 				<r>128</r>
 				<g>128</g>
 				<b>192</b>
 			</color>
 			<connections>
+				<relation>
+					<id>0</id>
+					<source>1</source>
+					<target>0</target>
+					<child_cardinality>1..n</child_cardinality>
+					<parent_cardinality>1</parent_cardinality>
+					<reference_for_pk>true</reference_for_pk>
+					<name></name>
+					<on_delete_action>RESTRICT</on_delete_action>
+					<on_update_action>RESTRICT</on_update_action>
+					<source_xp>-1</source_xp>
+					<source_yp>-1</source_yp>
+					<target_xp>-1</target_xp>
+					<target_yp>-1</target_yp>
+					<referenced_column>null</referenced_column>
+					<referenced_complex_unique_key>null</referenced_complex_unique_key>
+				</relation>
 			</connections>
-			<physical_name>ips</physical_name>
-			<logical_name>ips</logical_name>
+			<physical_name>virustotal_reports_data</physical_name>
+			<logical_name>virustotal_reports_data</logical_name>
 			<description></description>
 			<constraint></constraint>
 			<primary_key_name></primary_key_name>
+			<option></option>
 			<columns>
 				<normal_column>
-					<word_id>18</word_id>
 					<id>0</id>
+					<referenced_column>3</referenced_column>
+					<relation>0</relation>
 					<description></description>
 					<unique_key_name></unique_key_name>
-					<logical_name></logical_name>
-					<physical_name></physical_name>
-					<type>inet</type>
-					<constraint></constraint>
-					<default_value></default_value>
-					<auto_increment>false</auto_increment>
-					<foreign_key>false</foreign_key>
-					<not_null>true</not_null>
-					<primary_key>true</primary_key>
-					<unique_key>true</unique_key>
-					<sequence>
-						<name></name>
-						<schema></schema>
-						<increment></increment>
-						<min_value></min_value>
-						<max_value></max_value>
-						<start></start>
-						<cache></cache>
-						<cycle>false</cycle>
-						<description></description>
-					</sequence>
-				</normal_column>
-				<normal_column>
-					<word_id>30</word_id>
-					<id>1</id>
-					<description></description>
-					<unique_key_name></unique_key_name>
-					<logical_name></logical_name>
-					<physical_name></physical_name>
+					<logical_name>report_id</logical_name>
+					<physical_name>report_id</physical_name>
 					<type>bigint</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
-					<foreign_key>false</foreign_key>
-					<not_null>false</not_null>
+					<foreign_key>true</foreign_key>
+					<not_null>true</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>33</word_id>
-					<id>2</id>
+					<word_id>10</word_id>
+					<id>1</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>timestamp with time zone</type>
+					<type>long</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>34</word_id>
-					<id>3</id>
+					<word_id>22</word_id>
+					<id>2</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>timestamp with time zone</type>
+					<type>long</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 			</columns>
 		</table>
 		<table>
 			<id>1</id>
-			<height>95</height>
-			<width>297</width>
+			<height>170</height>
+			<width>334</width>
 				<font_name>Lucida Grande</font_name>
 				<font_size>9</font_size>
-			<x>444</x>
-			<y>257</y>
+			<x>733</x>
+			<y>71</y>
 			<color>
 				<r>128</r>
 				<g>128</g>
 			</color>
 			<connections>
 				<relation>
-					<id>0</id>
-					<source>5</source>
+					<id>1</id>
+					<source>8</source>
 					<target>1</target>
 					<child_cardinality>1..n</child_cardinality>
 					<parent_cardinality>1</parent_cardinality>
 					<referenced_column>null</referenced_column>
 					<referenced_complex_unique_key>null</referenced_complex_unique_key>
 				</relation>
-				<relation>
-					<id>1</id>
-					<source>5</source>
-					<target>1</target>
-					<child_cardinality>1..n</child_cardinality>
-					<parent_cardinality>1</parent_cardinality>
-					<reference_for_pk>false</reference_for_pk>
-					<name></name>
-					<on_delete_action>RESTRICT</on_delete_action>
-					<on_update_action>RESTRICT</on_update_action>
-					<source_xp>-1</source_xp>
-					<source_yp>-1</source_yp>
-					<target_xp>-1</target_xp>
-					<target_yp>-1</target_yp>
-					<referenced_column>27</referenced_column>
-					<referenced_complex_unique_key>null</referenced_complex_unique_key>
-				</relation>
 			</connections>
-			<physical_name>binaries_pending</physical_name>
-			<logical_name>binaries_pending</logical_name>
+			<physical_name>virustotal_reports</physical_name>
+			<logical_name>virustotal_reports</logical_name>
 			<description></description>
 			<constraint></constraint>
 			<primary_key_name></primary_key_name>
+			<option></option>
 			<columns>
 				<normal_column>
-					<id>4</id>
-					<referenced_column>26</referenced_column>
-					<relation>0</relation>
+					<word_id>14</word_id>
+					<id>3</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
-					<logical_name>binary_id</logical_name>
-					<physical_name>binary_id</physical_name>
+					<logical_name></logical_name>
+					<physical_name></physical_name>
+					<type>bigserial</type>
+					<constraint></constraint>
+					<default_value></default_value>
+					<auto_increment>false</auto_increment>
+					<foreign_key>false</foreign_key>
+					<not_null>true</not_null>
+					<primary_key>true</primary_key>
+					<unique_key>true</unique_key>
+					<character_set></character_set>
+					<collation></collation>
+					<sequence>
+						<name></name>
+						<schema></schema>
+						<increment></increment>
+						<min_value></min_value>
+						<max_value></max_value>
+						<start></start>
+						<cache></cache>
+						<cycle>false</cycle>
+						<order>false</order>
+						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
+					</sequence>
+				</normal_column>
+				<normal_column>
+					<id>4</id>
+					<referenced_column>47</referenced_column>
+					<relation>1</relation>
+					<description></description>
+					<unique_key_name></unique_key_name>
+					<logical_name>binary_id</logical_name>
+					<physical_name>binary_id</physical_name>
 					<type>bigint</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<not_null>true</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
+					<word_id>26</word_id>
 					<id>5</id>
-					<referenced_column>27</referenced_column>
-					<relation>1</relation>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>character(n)</type>
+					<type>datetime</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
-					<foreign_key>true</foreign_key>
-					<not_null>true</not_null>
+					<foreign_key>false</foreign_key>
+					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>23</word_id>
+					<word_id>27</word_id>
 					<id>6</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
+					<type>integer</type>
+					<constraint></constraint>
+					<default_value></default_value>
+					<auto_increment>false</auto_increment>
+					<foreign_key>false</foreign_key>
+					<not_null>false</not_null>
+					<primary_key>false</primary_key>
+					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
+					<sequence>
+						<name></name>
+						<schema></schema>
+						<increment></increment>
+						<min_value></min_value>
+						<max_value></max_value>
+						<start></start>
+						<cache></cache>
+						<cycle>false</cycle>
+						<order>false</order>
+						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
+					</sequence>
+				</normal_column>
+				<normal_column>
+					<word_id>24</word_id>
+					<id>7</id>
+					<description></description>
+					<unique_key_name></unique_key_name>
+					<logical_name></logical_name>
+					<physical_name></physical_name>
+					<type>long</type>
+					<constraint></constraint>
+					<default_value></default_value>
+					<auto_increment>false</auto_increment>
+					<foreign_key>false</foreign_key>
+					<not_null>false</not_null>
+					<primary_key>false</primary_key>
+					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
+					<sequence>
+						<name></name>
+						<schema></schema>
+						<increment></increment>
+						<min_value></min_value>
+						<max_value></max_value>
+						<start></start>
+						<cache></cache>
+						<cycle>false</cycle>
+						<order>false</order>
+						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
+					</sequence>
+				</normal_column>
+				<normal_column>
+					<word_id>29</word_id>
+					<id>8</id>
+					<description></description>
+					<unique_key_name></unique_key_name>
+					<logical_name></logical_name>
+					<physical_name></physical_name>
 					<type>timestamp with time zone</type>
 					<constraint></constraint>
 					<default_value>transaction_timestamp()</default_value>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 			</columns>
 			<indexes>
-				<inidex>
-					<full_text>false</full_text>
-					<non_unique>true</non_unique>
-					<name>binaries_pending_binary_id_idx</name>
-					<type></type>
-					<description></description>
-					<columns>
-						<column>
-							<id>4</id>
-							<desc>false</desc>
-						</column>
-					</columns>
-				</inidex>
-				<inidex>
-					<full_text>false</full_text>
-					<non_unique>true</non_unique>
-					<name>binaries_pending_md5_idx</name>
-					<type></type>
-					<description></description>
-					<columns>
-						<column>
-							<id>5</id>
-							<desc>false</desc>
-						</column>
-					</columns>
-				</inidex>
 			</indexes>
 			<complex_unique_key_list>
 			</complex_unique_key_list>
 		</table>
 		<table>
 			<id>2</id>
-			<height>134</height>
-			<width>216</width>
+			<height>233</height>
+			<width>334</width>
 				<font_name>Lucida Grande</font_name>
 				<font_size>9</font_size>
-			<x>624</x>
-			<y>67</y>
+			<x>194</x>
+			<y>567</y>
 			<color>
 				<r>128</r>
 				<g>128</g>
 				<b>192</b>
 			</color>
 			<connections>
-				<relation>
-					<id>2</id>
-					<source>5</source>
-					<target>2</target>
-					<child_cardinality>1..n</child_cardinality>
-					<parent_cardinality>1</parent_cardinality>
-					<reference_for_pk>true</reference_for_pk>
-					<name></name>
-					<on_delete_action>RESTRICT</on_delete_action>
-					<on_update_action>RESTRICT</on_update_action>
-					<source_xp>-1</source_xp>
-					<source_yp>-1</source_yp>
-					<target_xp>-1</target_xp>
-					<target_yp>-1</target_yp>
-					<referenced_column>null</referenced_column>
-					<referenced_complex_unique_key>null</referenced_complex_unique_key>
-				</relation>
 			</connections>
-			<physical_name>virustotal_reports</physical_name>
-			<logical_name>virustotal_reports</logical_name>
+			<physical_name>asn_cymru</physical_name>
+			<logical_name>asn_cymru</logical_name>
 			<description></description>
 			<constraint></constraint>
 			<primary_key_name></primary_key_name>
+			<option></option>
 			<columns>
 				<normal_column>
-					<word_id>13</word_id>
-					<id>7</id>
+					<word_id>20</word_id>
+					<id>9</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>bigserial</type>
+					<type>inet</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<not_null>true</not_null>
 					<primary_key>true</primary_key>
 					<unique_key>true</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<id>8</id>
-					<referenced_column>26</referenced_column>
-					<relation>2</relation>
+					<word_id>1</word_id>
+					<id>10</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
-					<logical_name>binary_id</logical_name>
-					<physical_name>binary_id</physical_name>
+					<logical_name></logical_name>
+					<physical_name></physical_name>
 					<type>bigint</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
-					<foreign_key>true</foreign_key>
-					<not_null>true</not_null>
+					<foreign_key>false</foreign_key>
+					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>25</word_id>
-					<id>9</id>
+					<word_id>4</word_id>
+					<id>11</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>datetime</type>
+					<type>long</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>26</word_id>
-					<id>10</id>
+					<word_id>8</word_id>
+					<id>12</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>integer</type>
+					<type>character(n)</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>22</word_id>
-					<id>11</id>
+					<word_id>25</word_id>
+					<id>13</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>29</word_id>
-					<id>12</id>
+					<word_id>0</word_id>
+					<id>14</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>timestamp with time zone</type>
+					<type>date</type>
 					<constraint></constraint>
-					<default_value>transaction_timestamp()</default_value>
+					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<foreign_key>false</foreign_key>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
-			</columns>
-			<indexes>
-			</indexes>
-			<complex_unique_key_list>
-			</complex_unique_key_list>
-			<table_properties>
-				<without_oids>true</without_oids>
-				<schema></schema>
-			</table_properties>
-		</table>
-		<table>
-			<id>3</id>
-			<height>95</height>
-			<width>185</width>
-				<font_name>Lucida Grande</font_name>
-				<font_size>9</font_size>
-			<x>919</x>
-			<y>88</y>
-			<color>
-				<r>128</r>
-				<g>128</g>
-				<b>192</b>
-			</color>
-			<connections>
-				<relation>
-					<id>3</id>
-					<source>2</source>
-					<target>3</target>
-					<child_cardinality>1..n</child_cardinality>
-					<parent_cardinality>1</parent_cardinality>
-					<reference_for_pk>true</reference_for_pk>
-					<name></name>
-					<on_delete_action>RESTRICT</on_delete_action>
-					<on_update_action>RESTRICT</on_update_action>
-					<source_xp>-1</source_xp>
-					<source_yp>-1</source_yp>
-					<target_xp>-1</target_xp>
-					<target_yp>-1</target_yp>
-					<referenced_column>null</referenced_column>
-					<referenced_complex_unique_key>null</referenced_complex_unique_key>
-				</relation>
-			</connections>
-			<physical_name>virustotal_reports_data</physical_name>
-			<logical_name>virustotal_reports_data</logical_name>
-			<description></description>
-			<constraint></constraint>
-			<primary_key_name></primary_key_name>
-			<columns>
 				<normal_column>
-					<id>13</id>
-					<referenced_column>7</referenced_column>
-					<relation>3</relation>
-					<description></description>
-					<unique_key_name></unique_key_name>
-					<logical_name>report_id</logical_name>
-					<physical_name>report_id</physical_name>
-					<type>bigint</type>
-					<constraint></constraint>
-					<default_value></default_value>
-					<auto_increment>false</auto_increment>
-					<foreign_key>true</foreign_key>
-					<not_null>true</not_null>
-					<primary_key>false</primary_key>
-					<unique_key>false</unique_key>
-					<sequence>
-						<name></name>
-						<schema></schema>
-						<increment></increment>
-						<min_value></min_value>
-						<max_value></max_value>
-						<start></start>
-						<cache></cache>
-						<cycle>false</cycle>
-						<description></description>
-					</sequence>
-				</normal_column>
-				<normal_column>
-					<word_id>10</word_id>
-					<id>14</id>
+					<word_id>5</word_id>
+					<id>15</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>long</type>
+					<type>inet</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>20</word_id>
-					<id>15</id>
+					<word_id>30</word_id>
+					<id>16</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>long</type>
+					<type>timestamp with time zone</type>
 					<constraint></constraint>
-					<default_value></default_value>
+					<default_value>transaction_timestamp()</default_value>
 					<auto_increment>false</auto_increment>
 					<foreign_key>false</foreign_key>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 			</columns>
 			</table_properties>
 		</table>
 		<table>
-			<id>4</id>
-			<height>-1</height>
-			<width>-1</width>
+			<id>3</id>
+			<height>233</height>
+			<width>229</width>
 				<font_name>Lucida Grande</font_name>
 				<font_size>9</font_size>
-			<x>27</x>
-			<y>29</y>
+			<x>553</x>
+			<y>566</y>
 			<color>
 				<r>128</r>
 				<g>128</g>
 				<b>192</b>
 			</color>
 			<connections>
-				<relation>
-					<id>4</id>
-					<source>5</source>
-					<target>4</target>
-					<child_cardinality>1..n</child_cardinality>
-					<parent_cardinality>1</parent_cardinality>
-					<reference_for_pk>true</reference_for_pk>
-					<name></name>
-					<on_delete_action>RESTRICT</on_delete_action>
-					<on_update_action>RESTRICT</on_update_action>
-					<source_xp>-1</source_xp>
-					<source_yp>-1</source_yp>
-					<target_xp>-1</target_xp>
-					<target_yp>-1</target_yp>
-					<referenced_column>null</referenced_column>
-					<referenced_complex_unique_key>null</referenced_complex_unique_key>
-				</relation>
 			</connections>
-			<physical_name>attacks_submithttp</physical_name>
-			<logical_name>attacks_submithttp</logical_name>
+			<physical_name>asn_shadowserver</physical_name>
+			<logical_name>asn_shadowserver</logical_name>
 			<description></description>
 			<constraint></constraint>
 			<primary_key_name></primary_key_name>
+			<option></option>
 			<columns>
 				<normal_column>
-					<word_id>17</word_id>
-					<id>16</id>
+					<word_id>20</word_id>
+					<id>17</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>bigserial</type>
+					<type>inet</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<not_null>true</not_null>
 					<primary_key>true</primary_key>
 					<unique_key>true</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<id>17</id>
-					<referenced_column>26</referenced_column>
-					<relation>4</relation>
-					<description></description>
-					<unique_key_name></unique_key_name>
-					<logical_name>binary_id</logical_name>
-					<physical_name>binary_id</physical_name>
-					<type>bigint</type>
-					<constraint></constraint>
-					<default_value></default_value>
-					<auto_increment>false</auto_increment>
-					<foreign_key>true</foreign_key>
-					<not_null>true</not_null>
-					<primary_key>false</primary_key>
-					<unique_key>false</unique_key>
-					<sequence>
-						<name></name>
-						<schema></schema>
-						<increment></increment>
-						<min_value></min_value>
-						<max_value></max_value>
-						<start></start>
-						<cache></cache>
-						<cycle>false</cycle>
-						<description></description>
-					</sequence>
-				</normal_column>
-				<normal_column>
-					<word_id>41</word_id>
+					<word_id>2</word_id>
 					<id>18</id>
 					<description></description>
 					<unique_key_name></unique_key_name>
 					<logical_name></logical_name>
 					<physical_name></physical_name>
-					<type>timestamp with time zone</type>
-					<constraint></constraint>
-					<default_value>transaction_timestamp()</default_value>
-					<auto_increment>false</auto_increment>
-					<foreign_key>false</foreign_key>
-					<not_null>false</not_null>
-					<primary_key>false</primary_key>
-					<unique_key>false</unique_key>
-					<sequence>
-						<name></name>
-						<schema></schema>
-						<increment></increment>
-						<min_value></min_value>
-						<max_value></max_value>
-						<start></start>
-						<cache></cache>
-						<cycle>false</cycle>
-						<description></description>
-					</sequence>
-				</normal_column>
-				<normal_column>
-					<word_id>36</word_id>
-					<id>19</id>
-					<description></description>
-					<unique_key_name></unique_key_name>
-					<logical_name></logical_name>
-					<physical_name></physical_name>
-					<type>inet</type>
+					<type>bigint</type>
 					<constraint></constraint>
 					<default_value></default_value>
 					<auto_increment>false</auto_increment>
 					<not_null>false</not_null>
 					<primary_key>false</primary_key>
 					<unique_key>false</unique_key>
+					<character_set></character_set>
+					<collation></collation>
 					<sequence>
 						<name></name>
 						<schema></schema>
 						<start></start>
 						<cache></cache>
 						<cycle>false</cycle>
+						<order>false</order>
 						<description></description>
+						<data_type></data_type>
+						<decimal_size>0</decimal_size>
 					</sequence>
 				</normal_column>
 				<normal_column>
-					<word_id>38</word_id>
-					<id>20</id>
+					<word_id>3</word_id>
+					<id>19</id>