Clone wiki

sparkseq / troubleshooting

Troubleshooting

* Fighting org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative exception.

Root cause:

You get this type of exception because your Hadoop distribution has native libraries compiled for a 32-bit architecture, but are using a 64-bit JVM ( or the other way round).

Solution:

Download Hadoop sources and compile them using mvn command (Guide):

mvn package -Pdist,native -DskipTests -Dtar

The package will be created under hadoop-dist/target/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz

* Possible errors while compiling Hadoop 2.2.0 :

  • Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1: Patch
  • Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc: Solution. Obviously some commands like apt and make install can be executed with sudo instead of using root account.
  • Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6: Intall cmake (e.g)
sudo apt-get install cmake

* Possible errors while running bedtools :

Bedtools2 accept tab-delimited input files - no other white spaces are allowed. You can run cat tool to see if '^I' characters are columns delimiters:

cat -A fileName

Updated