1. Jason Baldridge
  2. fogbow

Commits

Jason Baldridge  committed c1e5858

Updated documentation for changes to exectuble and assembly.

  • Participants
  • Parent commits 9d3afb2
  • Branches default

Comments (0)

Files changed (2)

File README

View file
 Variables, and add your settings in the User variables area.
 
 Next, likewise set FOGBOW_DIR to be the top level directory where you
-unzipped the download. In Unix, type 'pwd' in the directory where
-this file is and use the path given to you by the shell as
+unzipped the Fogbow download. In Unix, type 'pwd' in the directory
+where this file is and use the path given to you by the shell as
 FOGBOW_DIR.  You can set this in the same manner as for JAVA_HOME
 above.
 
 an environment variable with spaces in it, you need to put quotes around
 the value when on Unix, but you must *NOT* do this when under Windows.
 
-It is assumed that you have Hadoop installed and in your path.
+It is assumed that you have Hadoop 0.20.2 installed and in your path,
+and that you have set HADOOP_DIR to be the location of your Hadoop
+0.20.2 installation.
 
 
 Building the system from source
 
 $ fogbow build
 
-This will drop you into the SBT interface.  The build targets that are
-supported are listeded here:
+This will drop you into the SBT interface. To see the actions that are
+possible, hit the TAB key. (In general, you can do auto-completion on
+any command prefix in SBT, hurrah!)
 
-http://code.google.com/p/simple-build-tool/wiki/RunningSbt
+Documentation for SBT is here:
 
-You can also see targets by typing "actions" on the SBT prompt.
+https://github.com/harrah/xsbt/wiki
 
-Note: if you have SBT already installed on your system, you can also
-just call it directly with "sbt".
+Note: if you have SBT 0.10.1 already installed on your system, you can
+also just call it directly with "sbt".
 
 
 Trying it out
 =============
 
-Build the assembly jar (which contains all the dependencies).
-
-$ fogbow build assembly
-
-This will give you a jar in the target directory named
-Fogbow-assembly-0.1.jar which you can use with "hadoop jar"
-commands. 
-
-You can try it out on a single machine in non-distributed mode. As an
+Assuming you have completed all of the above steps, including running
+the "compile" action in SBT, you should now be able to try out the
+word count example on a single machine in non-distributed mode. As an
 example, let's do word count on the Adventures of Sherlock Holmes.
 
 Obtain the text:
 
 To do Java word count, run:
 
-$ hadoop jar $FOGBOW_DIR/target/Fogbow-assembly-0.1.jar tacc.hadoop.example.WordCount pg1661.txt wc_out_holmes_java
+$ fogbow run fogbow.example.WordCount pg1661.txt wc_out_holmes_java
 
 To do Scala word count, run:
 
-$ hadoop jar $FOGBOW_DIR/target/Fogbow-assembly-0.1.jar tacc.hadoop.scala.WordCount pg1661.txt wc_out_holmes_scala
+$ fogbow run fogbow.scala.WordCount pg1661.txt wc_out_holmes_scala
+
+
+Using the Fogbow assembly jar and calling it with Hadoop
+========================================================
+
+Using the 'fogbow' shell script will work for debugging your
+applications on a single machine in non-distributed mode (and without
+using HDFS). To deploy your application on HDFS, you need a jar file
+that you can call with the 'hadoop' exectuble.  For this, Fogbow
+allows you to build an assembly jar that packages all the dependencies
+of Fogbow in a single jar file.
+
+To build the assembly jar, do the following:
+
+$ fogbow build assembly
+
+This will create fogbow-assembly.jar in the $FOGBOW_DIR/target
+directory.
+
+As before, you can try it out on a single machine in non-distributed
+mode on Sherlock Holmes.
+
+To do Java word count, run:
+
+$ hadoop jar $FOGBOW_DIR/target/fogbow-assembly.jar fogbow.example.WordCount pg1661.txt wc_out_holmes_java_assembly
+
+To do Scala word count, run:
+
+$ hadoop jar $FOGBOW_DIR/target/fogbow-assembly.jar fogbow.scala.WordCount pg1661.txt wc_out_holmes_scala_assembly
 
 
 Now what?
  * change the information in $FOGBOW_DIR/project/build.properties to
    reflect your own project details
 
- * change $FOGBOW_DIR/project/build/FogbowProject.scala to be
-   named for your project, and if you need to specify new managed
-   dependencies, you can do so easily in that file (see SBT
-   documentation for details). If you prefer to add dependencies
-   manually, just add them to $FOGBOW_DIR/lib and they'll get picked
-   up.
+ * change $FOGBOW_DIR/build.sbt configurations to be appropriate for
+   your project, and if you need to specify new managed dependencies,
+   you can do so easily in that file (see SBT documentation for
+   details). If you prefer to add dependencies manually, just add them
+   to $FOGBOW_DIR/lib and they'll get picked up without any fuss.
 
  * change $FOGBOW_DIR/bin to be an executable of your choice, named
    for your project, and adapt as necessary (including changing

File bin/fogbow

View file
 	. ~/.sbtconfig
     fi
     java -Dfile.encoding=UTF8 -Xmx1536M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256m ${SBT_OPTS} -jar $FOGBOW_DIR/bin/sbt-launch.jar "$@"
-    #java -jar $FOGBOW_DIR/project/build/sbt-launch-0.7.7.jar "$@"
 
 else