Error when running tests

[Former user]

When running:

bash test_metannotate.sh

I get the following error:

.... Running FastTree. Running commands: FastTreeMP -out /home/programas/metannotate/databases/metannotate/test_output/rpoB_0_refseq_tree_aGkmAq49965386.newick -log /home/programas/metannotate/databases/metannotate/test_output/rpoB_0_refseq_fastree_log_LdopNo977189105.txt -mlnni 4 /home/programas/metannotate/databases/metannotate/test_output/rpoB_0_refseq_msa_9qRt_U968221187.fa Traceback (most recent call last): File "run_metannotate.py", line 200, in <module> main(sys.argv) File "run_metannotate.py", line 197, in main reference_tree, reference_log) File "/home/programas/metannotate/databases/metannotate/tasks.py", line 944, in RunPipelineReal log_file, '--tree-file', tree_file]) File "/home/programas/metannotate/databases/metannotate/tasks.py", line 33, in run_process the_process = subprocess.Popen(args, stdout=stdout, stderr=stderr) File "/usr/local/lib/python2.7/subprocess.py", line 710, in init errread, errwrite) File "/usr/local/lib/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

The FastTreeMP command does work (I have tried running it alone)

Any clue what it could be?

Thnak you in advance!

[Manas Bhatnagar]

Hi,

Could you post the output of 'whereis FastTreeMP' when run as the user you use for MetAnnotate?

That's the first thing I can think of, I'll update this post after some investigation.

'whereis FastTreeMP' might not produce any output (but if it does, please let me know what it says).

Please see what this produces as well:

$ find ~ -type f -iname "*fasttree*"

On my test system, this is what I see:

/home/metannotate-user/.local/bin/FastTreeMP
/home/metannotate-user/.local/lib/python2.7/site-packages/Bio/Phylo/Applications/_Fasttree.py
/home/metannotate-user/.local/lib/python2.7/site-packages/Bio/Phylo/Applications/_Fasttree.pyc
/home/metannotate-user/.local/lib/python2.7/site-packages/ete2/tools/phylobuild_lib/task/fasttree.pyc
/home/metannotate-user/.local/lib/python2.7/site-packages/ete2/tools/phylobuild_lib/task/fasttree.py
/home/metannotate-user/git/metannotate/bundled_software/FastTreeMP

The output you see might not be exactly like this, the important line is /home/metannotate-user/.local/bin/FastTreeMP If you do not see that line, the test could fail.

I have FastTreeMP on my fork if you need to download it:

https://bitbucket.org/manas_bhatnagar/metannotate/raw/c645de7f9d1432304c5e36dbd75f96c7baa9af5a/bundled_software/FastTreeMP

Manas

[BVG]

Hi Manas,

I do have FastTreeMp installed, so the result of 'whereis FastTreeMP' is '/usr/local/bin/FastTreeMP', plus I also have the last line of the find command. It does work when I try to run

FastTreeMP -out /home/programas/metannotate/databases/metannotate/test_output/rpoB_0_refseq_tree_aGkmAq49965386.newick -log /home/programas/metannotate/databases/metannotate/test_output/rpoB_0_refseq_fastree_log_LdopNo977189105.txt -mlnni 4 /home/programas/metannotate/databases/metannotate/test_output/rpoB_0_refseq_msa_9qRt_U968221187.fa

separately.

Blanca

[Manas Bhatnagar]

Hello Blanca,

The download link for FastTreeMP was changed by the provider a while back, the copy of FastTreeMP that the script downloaded is possibly just an HTML page and not an executable. MetAnnotate installs its own binary for FastTreeMP at ~/.local/bin/FastTreeMP. So, when you ran the provided installation script, FastTreeMP did not get downloaded properly.

You have 2 options, both of which should work equally well:

1

Copy FastTreeMP from /usr/local/bin/FastTreeMP to ~/.local/bin/

cp /usr/local/bin/FastTreeMP ~/.local/bin/

2

Download FastTreeMP to ~/.local/bin/

cd ~/.local/bin/ && wget https://bitbucket.org/manas_bhatnagar/metannotate/raw/c645de7f9d1432304c5e36dbd75f96c7baa9af5a/bundled_software/FastTreeMP

The difference between these two options is the version of FastTreeMP that gets installed for MetAnnotate to use, which could differ, but either option should work just as well.

Thanks, Manas

[BVG]

Hello Manas,

I have not been lucky, it doesn't work after trying both of your suggestions. Still having having the same error...

Blanca

[Manas Bhatnagar]

Okay, can you share your OS version & python version?

You could also check that the program is executable with

$ ls -al ~/.local/bin/FastTreeMP
-rwxrwxr-x 1 metannotate-user metannotate-user 373480 Mar 14 17:00 /home/metannotate-user/.local/bin/FastTreeMP

The contents of path.txt in your metannotate installation directory would also be useful.

[BVG]

OS version: CentOS 6.7 python version: 2.7.8

It seems that the program is executable.

This is the content of path.txt: /home/blanca/.local/bin/

[Manas Bhatnagar]

What filesize do you see for ~/.local/bin/FastTreeMP ?

And, does ~/.local/bin/FastTreeMP --version produce helpful output for FastTreeMP?

[Manas Bhatnagar]

We've been focussing on FastTreeMP so far but the issue could also be elsewhere. Let's verify that FastTreeMP is not causing the problem first and we can look at other things.

[BVG]

~/.local/bin/FastTreeMP is 365K.

And this is the output of ~/.local/bin/FastTreeMP --version:

Unknown or incorrect use of option -v FastTree protein_alignment > tree FastTree < protein_alignment > tree FastTree -out tree protein_alignment FastTree -nt nucleotide_alignment > tree FastTree -nt -gtr < nucleotide_alignment > tree FastTree < nucleotide_alignment > tree FastTree accepts alignments in fasta or phylip interleaved formats

Common options (must be before the alignment file): -quiet to suppress reporting information -nopr to suppress progress indicator -log logfile -- save intermediate trees, settings, and model details -fastest -- speed up the neighbor joining phase & reduce memory usage (recommended for >50,000 sequences) -n <number> to analyze multiple alignments (phylip format only) (use for global bootstrap, with seqboot and CompareToBootstrap.pl) -nosupport to not compute support values -intree newick_file to set the starting tree(s) -intree1 newick_file to use this starting tree for all the alignments (for faster global bootstrap on huge alignments) -pseudo to use pseudocounts (recommended for highly gapped sequences) -gtr -- generalized time-reversible model (nucleotide alignments only) -wag -- Whelan-And-Goldman 2001 model (amino acid alignments only) -quote -- allow spaces and other restricted characters (but not ' ) in sequence names and quote names in the output tree (fasta input only; FastTree will not be able to read these trees back in) -noml to turn off maximum-likelihood -nome to turn off minimum-evolution NNIs and SPRs (recommended if running additional ML NNIs with -intree) -nome -mllen with -intree to optimize branch lengths for a fixed topology -cat # to specify the number of rate categories of sites (default 20) or -nocat to use constant rates -gamma -- after optimizing the tree under the CAT approximation, rescale the lengths to optimize the Gamma20 likelihood -constraints constraintAlignment to constrain the topology search constraintAlignment should have 1s or 0s to indicates splits -expert -- see more options For more information, see http://www.microbesonline.org/fasttree/

[Manas Bhatnagar]

I'm not really sure what else to try. You could run bash test_metannotate.sh again and based on where it fails, you could run file on each file being called to verify that each one exists. If there are one or more files missing, we can look at replacing them.

[BVG]

It seems everything is there..

Thank you very much for your help anyways!

[Chris Z]

Hello all,

I seem to be having the same problem when running the test command. I tried changing the tasks.py file to include the parameter (shell=True) when calling the sub-process in line 35, but it didn't seem to work. I am running Ubuntu 12.04.5 LTS, Python 2.7.3 Thanks in advance.

[Andrew Doxey]

Hello Chris, thanks for the feedback.

We've recently switched to a docker install, which might resolve this issue. Have you tried it?

https://bitbucket.org/doxeylab/metannotateinstaller

[Chris Z]

Thank you, I will try it as soon as I can!