Using par_ana will split my file into individual features

[yoshiki89]

I tried using par_ana to run my jobs in parallel, however whenever I ran it on my data, each of my features would get split into an individual file, and then when lsa_compute would run on this file, I would get an error saying that:

Error: one input file need more than two data row or use -e to specify another input file

Not sure if I'm using par_ana correctly, so here's the command that I was trying to run:

par_ana elsa-features.tsv elsa-output.tsv 'lsa_compute %s %s -m 25 -p perm -r 1 -s 29 -f linear -q scipy' /full/output/path/

[Charlie Xia]

Hi Yoshiki,

Thank you for using the software.

First, did you check your data with check_data?

Charlie

[yoshiki89]

Yes, check_data worked just fine in the input file i.e. elsa-features.tsv. In fact lsa_compute itself works fine when I don't use par_ana, though it would be better if I could take advantage of the parallelization capabilities.

[Charlie Xia]

Dear Yoshiki,

See the example here: Example: par_ana ARISA20.txt ARISA20.lsa 'lsa_compute %s %s -e ARISA20.txt -s 127 -r 1 -p theo' $PWD

To use par_ana -e is required, even if for a single input file. In your case, you need add "-e elsa-features.tsv" to your command

[yoshiki89]

Thanks @charade, this works as expected now!

Side question, is there anyway to specify how many jobs can be submitted at a time? As it is right now it only has two jobs running, but say I wanted to run 20 or 30, how would I modify that?

[Charlie Xia]

You are welcome. You can manually edit the ssa.py to increase maxcore. But I remembered the default is 30. I am not sure why you are only getting two. Are you using the torque pbs system and the main queue? You might need tweak the ssa.py script further if you are not using these defaults.

Best Charlie

[yoshiki89]

I have the number of cores set at 32, and I'm indeed using torque but not the main queue, I'm using the route queue, thus these are the settings I have:

core_max=32
mem_max=128000000000
uname="yovazquezbaeza"
qname="route"

[Charlie Xia]

Dear Yoshiki,

I think the mem_max is in GB; So yours would be 128.

Charlie

[Charlie Xia]

issue solved