"Killed" error on every run using "jgi_summarize_bam_contig_depths"

[Former user]

I am attempting to run "jgi_summarize_bam_contig_depths" on multiple sorted .bam files, however each time I run the same command:

jgi_summarize_bam_contig_depths --outputDepth depth.txt D11P1_BS.bam D11P2_BS.bam D11P3_BS.bam D11P4_BS.bam D11P5_BS.bam D11P6_BS.bam D12P1_BS.bam D12P1_RS.bam D12P2_BS.bam D12P2_RS.bam D12P3_BS.bam D12P3_RS.bam D12P4_BS.bam D12P4_RS.bam D12P5_BS.bam D12P5_RS.bam D12P6_BS.bam D12P6_RS.bam D13P1_BS.bam D13P1_RS.bam D13P2_BS.bam D13P2_RS.bam D13P3_BS.bam D13P3_RS.bam D13P4_BS.bam D13P4_RS.bam D13P5_BS.bam D13P5_RS.bam D13P6_BS.bam D13P6_RS.bam D14P1_BS.bam D14P1_RS.bam D14P2_BS.bam D14P2_RS.bam D14P3_BS.bam D14P3_RS.bam D14P4_BS.bam D14P4_RS.bam D14P5_BS.bam D14P5_RS.bam D14P6_BS.bam D14P6_RS.bam D21P1_BS.bam D21P2_BS.bam D21P3_BS.bam D21P4_BS.bam D21P5_BS.bam D21P6_BS.bam D21P7_BS.bam D22P1_BS.bam D22P1_RS.bam D22P2_BS.bam D22P2_RS.bam D22P3_BS.bam D22P3_RS.bam D22P4_BS.bam D22P4_RS.bam D22P5_BS.bam D22P5_RS.bam D22P6_BS.bam D22P6_RS.bam D23P1_BS.bam D23P1_RS.bam D23P2_BS.bam D23P2_RS.bam D23P3_BS.bam D23P3_RS.bam D23P4_BS.bam D23P4_RS.bam D23P5_BS.bam D23P5_RS.bam D23P6_BS.bam D23P6_RS.bam D24P1_BS.bam D24P1_RS.bam D24P2_BS.bam D24P2_RS.bam D24P3_RS.bam D24P4_BS.bam D24P4_RS.bam D24P5_BS.bam D24P5_RS.bam D24P6_BS.bam D24P6_RS.bam D31P1_BS.bam D31P3_BS.bam D31P4_BS.bam D31P5_BS.bam D32P1_BS.bam D32P1_RS.bam D32P3_BS.bam D32P3_RS.bam D32P4_BS.bam D32P4_RS.bam D32P5_BS.bam D32P5_RS.bam D33P1_BS.bam D33P1_RS.bam D33P3_BS.bam D33P3_RS.bam D33P4_BS.bam D33P4_RS.bam D33P5_BS.bam D33P5_RS.bam D34P1_BS.bam D34P1_RS.bam D34P3_BS.bam D34P3_RS.bam D34P4_BS.bam D34P4_RS.bam D34P5_BS.bam D34P5_RS.bam

It ends with this, the same order as well when it resulted in "killed":

jgi_summarize_bam_contig_depths 2.15 (Bioconda) 2020-01-04T21:10:40

Output matrix to depth.txt

0: Opening bam: D11P1_BS.bam

15: Opening bam: D12P5_RS.bam

28: Opening bam: D13P6_BS.bam

40: Opening bam: D14P6_BS.bam

8: Opening bam: D12P2_BS.bam

39: Opening bam: D14P5_RS.bam

47: Opening bam: D21P6_BS.bam

32: Opening bam: D14P2_BS.bam10422436: Opening bam: D14P4_BS.bam

23: Opening bam: 31: Opening bam: D14P1_RS.bam

Opening bam: D21P1_BS.bam

18: Opening bam: D13P1_BS.bam

Opening bam: D13P4_BS.bam44: Opening bam: D21P3_BS.bam D13P3_RS.bam 34: Opening bam: D14P3_BS.bam 26: Opening bam: D13P5_BS.bam 20: Opening bam: D13P2_BS.bam 2: Opening bam: D11P3_BS.bam

Opening bam: D12P3_BS.bam

4: Opening bam: 4133132927: Opening bam: D13P5_RS.bam

D11P5_BS.bam

19: Opening bam: 11: Opening bam: D12P3_RS.bam

1: Opening bam: D11P2_BS.bam

Opening bam: D14P6_RS.bam

30: Opening bam: D14P1_BS.bam

38: Opening bam: D14P5_BS.bam14: Opening bam: D12P5_BS.bam

Opening bam: D14P2_RS.bam

4516: Opening bam: 22: Opening bam: D13P3_BS.bam

37: Opening bam: D14P4_RS.bam

12: Opening bam: 9: Opening bam: D12P2_RS.bam

43: Opening bam: D21P2_BS.bam

7: Opening bam: D12P1_RS.bam

5: Opening bam: D11P6_BS.bam

25: Opening bam: D13P4_RS.bam

6: Opening bam: D12P1_BS.bam

Opening bam: : Opening bam: 3: Opening bam: D12P4_BS.bam D13P1_RS.bam46: Opening bam: D21P5_BS.bam 21: Opening bam: D13P2_RS.bam D12P6_BS.bam D13P6_RS.bam D11P4_BS.bam35

Opening bam: D21P4_BS.bam

17: Opening bam: D12P6_RS.bam

Opening bam: D14P3_RS.bam D12P4_RS.bam 10: Opening bam: D22P5_RS.bam 5: Opening bam: D22P3_BS.bam 31: Opening bam: D24P4_RS.bam 24: Opening bam: D23P6_RS.bam 25: Opening bam: D24P1_BS.bam 13: Opening bam: D23P1_BS.bam 23: Opening bam: D23P6_BS.bam 20: Opening bam: D23P4_RS.bam 8: Opening bam: D22P4_RS.bam 16: Opening bam: D23P2_RS.bam 12: Opening bam: D22P6_RS.bam 4: Opening bam: D22P2_RS.bam 32: Opening bam: D24P5_BS.bam 6: Opening bam: D22P3_RS.bam 28: Opening bam: D24P2_RS.bam 18: Opening bam: D23P3_RS.bam 21: Opening bam: D23P5_BS.bam 19: Opening bam: D23P4_BS.bam 14: Opening bam: D23P1_RS.bam 26: Opening bam: D24P1_RS.bam 33: Opening bam: D24P5_RS.bam 2: Opening bam: D22P1_RS.bam 15: Opening bam: D23P2_BS.bam 29: Opening bam: D24P3_RS.bam 30: Opening bam: D24P4_BS.bam 9: Opening bam: D22P5_BS.bam 39: Opening bam: D31P5_BS.bam 37: Opening bam: D31P3_BS.bam 22: Opening bam: D23P5_RS.bam 27: Opening bam: D24P2_BS.bam 38: Opening bam: D31P4_BS.bam 3: Opening bam: D22P2_BS.bam 7: Opening bam: D22P4_BS.bam 1: Opening bam: D22P1_BS.bam 45: Opening bam: D32P4_RS.bam 40: Opening bam: D32P1_BS.bam 42: Opening bam: D32P3_BS.bam 43: Opening bam: D32P3_RS.bam 17: Opening bam: D23P3_BS.bam 34: Opening bam: D24P6_BS.bam 35: Opening bam: D24P6_RS.bam 11: Opening bam: D22P6_BS.bam 44: Opening bam: D32P4_BS.bam 41: Opening bam: D32P1_RS.bam 47: Opening bam: D32P5_RS.bam 46: Opening bam: D32P5_BS.bam 36: Opening bam: D31P1_BS.bam 0: Opening bam: D21P7_BS.bam 12: Opening bam: D34P4_BS.bam 10: Opening bam: D34P3_BS.bam 14: Opening bam: D34P5_BS.bam 8: Opening bam: D34P1_BS.bam 13: Opening bam: D34P4_RS.bam 5: Opening bam: D33P4_RS.bam 15: Opening bam: D34P5_RS.bam 9: Opening bam: D34P1_RS.bam 6: Opening bam: D33P5_BS.bam 11: Opening bam: D34P3_RS.bam 7: Opening bam: D33P5_RS.bam 4: Opening bam: D33P4_BS.bam 3: Opening bam: D33P3_RS.bam 2: Opening bam: D33P3_BS.bam 1: Opening bam: D33P1_RS.bam 0: Opening bam: D33P1_BS.bam Processing bam files Thread 44 finished: D21P3_BS.bam with 11525569 reads and 6851996 readsWellMapped Thread 34 finished: D14P3_BS.bam with 9239127 reads and 5798684 readsWellMapped Thread 30 finished: D14P1_BS.bam with 8514713 reads and 5149368 readsWellMapped Thread 20 finished: D13P2_BS.bam with 7968947 reads and 4588021 readsWellMapped Thread 32 finished: D14P2_BS.bam with 7607604 reads and 4426200 readsWellMapped Thread 18 finished: D13P1_BS.bam with 9212792 reads and 5728768 readsWellMapped Thread 47 finished: D21P6_BS.bam with 10966104 reads and 6503327 readsWellMapped Thread 28 finished: D13P6_BS.bam with 9782858 reads and 6404561 readsWellMapped Thread 38 finished: D14P5_BS.bam with 9905461 reads and 6270818 readsWellMapped Thread 24 finished: D13P4_BS.bam with 8968725 reads and 5619991 readsWellMapped Thread 31 finished: D14P1_RS.bam with 9575399 reads and 7175740 readsWellMapped Thread 46 finished: D21P5_BS.bam with 10237475 reads and 6468054 readsWellMapped Thread 26 finished: D13P5_BS.bam with 9807405 reads and 6194698 readsWellMapped Thread 43 finished: D21P2_BS.bam with 11489447 reads and 6969112 readsWellMapped Thread 36 finished: D14P4_BS.bam with 10761839 reads and 6529181 readsWellMapped Thread 40 finished: D14P6_BS.bam with 9922763 reads and 6031096 readsWellMapped Thread 25 finished: D13P4_RS.bam with 9388148 reads and 7120936 readsWellMapped Thread 0 finished: D11P1_BS.bam with 15772000 reads and 9766885 readsWellMapped Thread 22 finished: D13P3_BS.bam with 12693025 reads and 8081144 readsWellMapped Thread 42 finished: D21P1_BS.bam with 12188103 reads and 7275618 readsWellMapped Thread 21 finished: D13P2_RS.bam with 11091351 reads and 8387296 readsWellMapped Thread 35 finished: D14P3_RS.bam with 10451299 reads and 7805988 readsWellMapped Thread 39 finished: D14P5_RS.bam with 11498694 reads and 8782022 readsWellMapped Thread 33 finished: D14P2_RS.bam with 11573064 reads and 8692074 readsWellMapped Thread 37 finished: D14P4_RS.bam with 10571556 reads and 8021335 readsWellMapped Thread 23 finished: D13P3_RS.bam with 10987043 reads and 8241775 readsWellMapped Thread 41 finished: D14P6_RS.bam with 11079846 reads and 8571614 readsWellMapped Thread 45 finished: D21P4_BS.bam with 14317597 reads and 9149012 readsWellMapped Thread 29 finished: D13P6_RS.bam with 12489417 reads and 9623219 readsWellMapped Thread 2 finished: D11P3_BS.bam with 19120917 reads and 11682376 readsWellMapped Thread 27 finished: D13P5_RS.bam with 13431506 reads and 10465131 readsWellMapped Thread 19 finished: D13P1_RS.bam with 12740986 reads and 9516396 readsWellMapped Killed

[Rob Egan]

If it is being killed, then the machine you are running this on is likely running out of resources (most likely memory). You can try to run it with fewer threads or even a single thread

You can try setting this in the environment:

OMP_NUM_THREADS=16

or

OMP_NUM_THREADS=1

If you have a large assembly with what looks like a large number of samples, then the peak RAM for this will also be large.

If reducing the parallelism does not work, and it still runs out of memory then you may have to find a machine with more RAM.

You can also try running this in batches on a subset of bam files (try on 1/2, 1/4 all the way down to 1) and then you will have to manually build the single abundance file by a simple script to combine the columns and sum the total column.

[Rob Egan]

in anticipation that you will need to execute jgi_summarize_bam_contig_depths multiple times (and from other requests by other users for a similar workflow support) I just wrote a simple python script that can combine the resulting abundance files from multiple bam into 1:

https://bitbucket.org/berkeleylab/metabat/src/master/src/merge_abundances.py