Using par_ana will split my file into individual features
I tried using par_ana
to run my jobs in parallel, however whenever I ran it on my data, each of my features would get split into an individual file, and then when lsa_compute
would run on this file, I would get an error saying that:
Error: one input file need more than two data row or use -e to specify another input file
Not sure if I'm using par_ana
correctly, so here's the command that I was trying to run:
par_ana elsa-features.tsv elsa-output.tsv 'lsa_compute %s %s -m 25 -p perm -r 1 -s 29 -f linear -q scipy' /full/output/path/
Comments (9)
-
repo owner -
reporter Yes,
check_data
worked just fine in the input file i.e.elsa-features.tsv
. In factlsa_compute
itself works fine when I don't usepar_ana
, though it would be better if I could take advantage of the parallelization capabilities. -
repo owner Dear Yoshiki,
See the example here: Example: par_ana ARISA20.txt ARISA20.lsa 'lsa_compute %s %s -e ARISA20.txt -s 127 -r 1 -p theo' $PWD
To use par_ana -e is required, even if for a single input file. In your case, you need add "-e elsa-features.tsv" to your command
-
reporter Thanks @charade, this works as expected now!
Side question, is there anyway to specify how many jobs can be submitted at a time? As it is right now it only has two jobs running, but say I wanted to run 20 or 30, how would I modify that?
-
repo owner You are welcome. You can manually edit the ssa.py to increase maxcore. But I remembered the default is 30. I am not sure why you are only getting two. Are you using the torque pbs system and the main queue? You might need tweak the ssa.py script further if you are not using these defaults.
Best Charlie
-
reporter I have the number of cores set at 32, and I'm indeed using torque but not the main queue, I'm using the
route
queue, thus these are the settings I have:core_max=32 mem_max=128000000000 uname="yovazquezbaeza" qname="route"
-
repo owner Dear Yoshiki,
I think the mem_max is in GB; So yours would be 128.
Charlie
-
repo owner - changed status to closed
issue solved
-
repo owner - changed status to resolved
- Log in to comment
Hi Yoshiki,
Thank you for using the software.
First, did you check your data with check_data?
Charlie