Clone wiki

Cistrome-Harvard / Tutorial_for_Cistrome_AP_installation

Environment

  1. Mercurial (apt-get install mercurial)
  2. Mysql or postgresql
  3. Python ver = 2.6
  4. gcc/g++ ver >= 3.2.2
  5. SWIG ver >= 1.3.24
  6. pybedtools
  7. bx-python

Install Cistrome Server

  • Run this command to get source code from our mercurial server.
$ hg clone https://bitbucket.org/cistrome/cistrome-harvard CistromeAP
  • Then go to CistromeAP folder and run once:
$ bash run.sh

After you see sonething like "serving on http://127.0.0.1:8080", then you can quit by pressing CTRL-C. (If CTRL-C doesn't work, try CTRL-Z and kill the process)

  • Copy the cistrome configurations:
$ cp datatypes_conf.xml.cistrome datatypes_conf.xml
$ cp tool_conf.xml.cistrome tool_conf.xml
$ cp universe_wsgi.ini.cistrome universe_wsgi.ini
  • Among the configuration files, universe_wsgi.ini file needs to be modified according to your server environment. You need to tweak the following parameters:

(1) port # set this and tweak your apache setting. I set port 10096 here as an example.

# The port on which to listen.
port = 10096

(2) host # typically it's 127.0.0.1

(3) prefix # set this and tweak your apache setting, here I use 'ap' as a example.

# Define the proxy-prefix filter.
[filter:proxy-prefix]
use = egg:PasteDeploy#prefix
prefix = /ap

(4) cookie_path # keep this same as prefix

cookie_path = /ap

(5) database_connection # use postgreseql or mysql, here I use mysql as example.

database_connection = mysql://cistrome:cistrome@localhost/cistromeap

The command need a user 'cistrome' with password 'cistrome', who has privilege on database 'cistromeap'. So you may need to login mysql with root and type this:

$ CREATE database cistromeap;
$ GRANT all privileges on cistromeap.* to 'cistrome'@'localhost' identified by 'cistrome' with grant option;

(6) file_path # typically it's database/files

(7) new_file_path # typically it's database/tmp

(8) error_email_to # complain emails will go to ...

(9) logo_url # set it to your own website

(10) wiki_url # your own wiki site

(11) bugs_email # complain emails will go to ...

(12) admin_users # add your admin users

(13) ftp_upload_dir # for FTP upload dir

(14) ftp_upload_site # for FTP website

(15) cistrome_static_library_path # where your Cistrome static library files are.

Install Cistrome Functions

  • Run this command to get source code of the applications.
$ hg clone https://username@bitbucket.org/cistrome/cistrome-applications-harvard cistrome-apps
  • Run installation script to enable tools

(1) Enable basic tools.

Go to "./cistrome-apps/cistrome-extra-apps", and run this to install basic tools.

$ python setup.py install

(2) Enable other packages, incluing MACS, CEAS, MA2C, MAT and seqpos.

go the "./cistrome-apps/mdseqpos", "./cistrome-apps/published-packages/CEAS", "./cistrome-apps/published-packages/MACS", "./cistrome-apps/published-packages/MA2C", "./cistrome-apps/published-packages/MAT" and follow the "INSTALL" file in each directory to install the packages.

(3) Enable expression tools. go to "./CistromeAP/cistromescripts" (the cistrome main folder), and run

$ Rscript bioc.R

You may need to install a higher version of R to run this. After this, it will install R dependencies Cistrome need.

Download Static data

Then, let's go to Cistrome server directory. All the static data will put in the tool-data folder, you can also change the position by edit the file "universe_wsgi.ini", change the value of "cistrome_static_library_path". (examples below assume you using default settings in this tutorial)

  • CISTROME STATIC DATA IS AVALIABLE NOW

If you don't want to read the instruction below, just go here and get all the static data: http://cistrome.org/aspera/user/?B=%2FCISTROME_STATIC_LIBRARIES send email to cistrome-bugs@jimmy.harvard.edu for a password to download.

  • MAT bpmaps

a) Go to http://liulab.dfci.harvard.edu/MAT/Download.htm , and download the "RepLib for new MAT".

b) Unzip each zip file and put them under each folder. Example: "./tool-data/MAT-lib/bpmaps/HumanTiling1.0_BPMAP_NCBIv36/" will contain 14 bpmap files.

  • CEAS library

a) Go to http://liulab.dfci.harvard.edu/CEAS/download.html , and download the annotation files. "ce4" and "ce6" for c. elegans, "dm2" and "dm3" for fruit fly, "mm8" and "mm9" for mouse, "hg18" and "hg19" for human.

b)Unzip them and put them under the folder. Example: "./tool-data/ceaslib/GeneTable/hg19", "./tool-data/ceaslib/GeneTable/mm9"

  • Liftover chain files

a) Download the raw data from UCSC, for example, the mm9 to mm8 chain file can be downloaded here: http://hgdownload.cse.ucsc.edu/goldenPath/mm9/liftOver/mm9ToMm8.over.chain.gz

b) Unzip it, and put the file under the folder. Example: "./tool-data/liftOver/chain_files/hg18ToHg19.over.chain", "./tool-data/liftOver/chain_files/mm7ToMm8.over.chain"

  • Conservation score files

As an example, I will show you how to build mm9

a) http://hgdownload.cse.ucsc.edu/goldenPath/mm9/phastCons30way/

b) *need to be edit later.

  • Seqpos assembly files

a) As a example, for human hg19, you can get data from http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/chromFa.tar.gz and http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/chromFaMasked.tar.gz

b) After unzip the files downloaded, put files like chr1.fa.masked into folder "./tool-data/assembly/humanhg19/masked/" and put files like chr1.fa into folder "./tool-data/assembly/humanhg19/rawgenome/" (one file for each chromosome)

Enable Functions

Here I show a example on how to enable samtools on Cistrome.

  • make sure you have install samtools on your server.
  • you need to write a tool config xml file. Each tool on Cistrome has it's own config file to tell where is the script, what to do, what to display on webpage and so on. Normally the config files are all under the folder "./tools/"
  • Since the samtool config files are already exist at "./tools/samtools/", what we need to do is edit "./tool_conf.xml" to make the tools show up at web page. Add these lines within "toolbox" tag.
  <section name="SAM Tools" id="samtools">
    <tool file="samtools/bam_to_sam.xml" />
    <tool file="samtools/sam_to_bam.xml" />
    <tool file="samtools/samtools_flagstat.xml" />
    <tool file="samtools/sam2interval.xml" />
    <tool file="samtools/sam_pileup.xml" />
    <tool file="samtools/sam_merge.xml" />
    <tool file="samtools/sam_bitwise_flag_filter.xml" />
    <tool file="samtools/pileup_interval.xml" />
    <tool file="samtools/pileup_parser.xml" />
  </section>

Start Server

1. run this command again, after that point your browser to http://localhost:10096/ap enjoy with it!

$ bash run.sh

Updated