First, Installation:

hg clone https://bitbucket.org/Alvin_Qin/dcman
cd dcman
sudo python setup.py install

Common Usage:

dcman meta -r dcman/dcman/meta/ -o . -d -u qinq
dcman  queue -l queue.txt -o . -d -u qinq


support absolid color fastq and Illumina fastq use crontab to run periodically input file example:

mm9 for mouse and hg19 for human

1. meta file
use ID as file name, e.g. 1910, with content
GSM691591,GSM691593   GSM691592   mm9     H3K27me3  Illumina

column two denotes treat files
column three denotes control files
**none** if no control, you can paste from excel, reserve a column for that **none**.
the name in column two and three should be consistent with GSM IDs or your defined names.
The last column denotes platform, Illumina, absolid
put a series of meta files in a directory, and input this directory
**the input file will be consistent with ID**, such as GSM691591.fastq

2. queue file -- multiple inputs
use any legal file name from excel zohos, internal dc usage
4672  GSM691591,GSM691592  GSM691593,GSM691593  mm9   H3K27me3  Illumina
4673  GSM691594    mm9   H3K27me3  absolid
the same rule as **1** except one more column for your experiment(or dataset) ID.

3. download file -- you want to analyze data manually, and only download data
use command
use any legal file name
GSM691591  GSM691592
GSM691593  GSM691592
a ID matrix with any length of rows or columns,
duplicate ones would be downloaded only once, data would be renamed to GSM ID
after downloaded.

For 1 and 2, if data is from geo platform and no exceptions occur during web extraction, the data would be downloaded and processed as normal. With exceptions or data from other platform, download manually, note:

data should be renamed to the id you specified in the meta file or queue file
such as GSM691591.fastq, GSM691592.fastq for 4672 treat files.


usage: dcman [-h] {test,down,meta,queue} ...

dcman : A tool for ChiLin

positional arguments:
{test,down,meta,queue} sub-command help
test test dcman using test data and config down generate a template of config file queue run with server meta run with meta info from other server
optional arguments:
-h, --help show this help message and exit


usage: dcman queue [-h] -l LIST -o OUTPUT -d -u USER

chilin run with meta files

optional arguments:
-h, --help show this help message and exit
-l LIST, --list LIST
 specify queue list for process
-o OUTPUT, --output OUTPUT
 specify queue output files
-d, --download specify meta output files
-u USER, --user USER
 specify user name


optional arguments:
-h, --help show this help message and exit
-r DIR, --dir DIR
 specify meta folder for process
-u USER, --user USER
 specify user name
-o OUTPUT, --output OUTPUT
 specify meta output files
-d, --download specify meta output files
optional arguments:
-h, --help show this help message and exit
-d DEST, --dest DEST
 specify download directory
-l LIST, --list LIST
 specify download list file, supporting meta


to run with -s option to test the 3 functions, not yet


  • For 1 and 2 input, output would be folder whose name is the dataset ID with all
    • chilin output
    • original fastq files
  • For 3, output would only be the original fastq files


How to recover from accident, such as shell top error, using qsub or snakemake?