Variables, and add your settings in the User variables area.
Next, likewise set FOGBOW_DIR to be the top level directory where you
-unzipped the download. In Unix, type 'pwd' in the directory where
-this file is and use the path given to you by the shell as
+unzipped the Fogbow download. In Unix, type 'pwd' in the directory
+where this file is and use the path given to you by the shell as
FOGBOW_DIR. You can set this in the same manner as for JAVA_HOME
an environment variable with spaces in it, you need to put quotes around
the value when on Unix, but you must *NOT* do this when under Windows.
-It is assumed that you have Hadoop installed and in your path.
+It is assumed that you have Hadoop 0.20.2 installed and in your path,
+and that you have set HADOOP_DIR to be the location of your Hadoop
Building the system from source
-This will drop you into the SBT interface. The build targets that are
-supported are listeded here:
+This will drop you into the SBT interface. To see the actions that are
+possible, hit the TAB key. (In general, you can do auto-completion on
+any command prefix in SBT, hurrah!)
You can also see targets by typing "actions" on the SBT prompt.
-Note: if you have SBT already installed on your system, you can also
-just call it directly with "sbt".
+Note: if you have SBT 0.10.1 already installed on your system, you can
+also just call it directly with "sbt".
-Build the assembly jar (which contains all the dependencies).
-This will give you a jar in the target directory named
-Fogbow-assembly-0.1.jar which you can use with "hadoop jar"
-You can try it out on a single machine in non-distributed mode. As an
+Assuming you have completed all of the above steps, including running
+the "compile" action in SBT, you should now be able to try out the
+word count example on a single machine in non-distributed mode. As an
example, let's do word count on the Adventures of Sherlock Holmes.
To do Java word count, run:
hadoop jar $FOGBOW_DIR/target/Fogbow-assembly-0.1.jar tacc.hadoop.example.WordCount pg1661.txt wc_out_holmes_java
+$ .example.WordCount pg1661.txt wc_out_holmes_java
To do Scala word count, run:
-$ hadoop jar $FOGBOW_DIR/target/Fogbow-assembly-0.1.jar tacc.hadoop.scala.WordCount pg1661.txt wc_out_holmes_scala
+$ fogbow run fogbow.scala.WordCount pg1661.txt wc_out_holmes_scala
+Using the Fogbow assembly jar and calling it with Hadoop
+Using the 'fogbow' shell script will work for debugging your
+applications on a single machine in non-distributed mode (and without
+using HDFS). To deploy your application on HDFS, you need a jar file
+that you can call with the 'hadoop' exectuble. For this, Fogbow
+allows you to build an assembly jar that packages all the dependencies
+of Fogbow in a single jar file.
+To build the assembly jar, do the following:
+This will create fogbow-assembly.jar in the $FOGBOW_DIR/target
+As before, you can try it out on a single machine in non-distributed
+mode on Sherlock Holmes.
+To do Java word count, run:
+$ hadoop jar $FOGBOW_DIR/target/fogbow-assembly.jar fogbow.example.WordCount pg1661.txt wc_out_holmes_java_assembly
+To do Scala word count, run:
+$ hadoop jar $FOGBOW_DIR/target/fogbow-assembly.jar fogbow.scala.WordCount pg1661.txt wc_out_holmes_scala_assembly
* change the information in $FOGBOW_DIR/project/build.properties to
reflect your own project details
- * change $FOGBOW_DIR/project/build/FogbowProject.scala to be
- named for your project, and if you need to specify new managed
- dependencies, you can do so easily in that file (see SBT
- documentation for details). If you prefer to add dependencies
- manually, just add them to $FOGBOW_DIR/lib and they'll get picked
+ * change $FOGBOW_DIR/build.sbt configurations to be appropriate for
+ your project, and if you need to specify new managed dependencies,
+ you can do so easily in that file (see SBT documentation for
+ details). If you prefer to add dependencies manually, just add them
+ to $FOGBOW_DIR/lib and they'll get picked up without any fuss.
* change $FOGBOW_DIR/bin to be an executable of your choice, named
for your project, and adapt as necessary (including changing