- changed status to resolved
get_memory_usage is possibly unsafe on abel
Issue #19
resolved
Since it spawns a process (calls "ps ...") it depends on existence and correct behaviour of ps without corruption of the mpi process which there are warnings about.
I suggest we disable it by default to be on the safe side. The cluster job system reports memory usage anyway.
I got this error at the end of a couple of simulations (most are fine):
Timestep 10000 finished (t=1.00e+00, 100.0%) in 9.4s (solve: 4.0s). Time spent: 11h 55m 39s Time remaining: 0s
Traceback (most recent call last):
File "/cluster/home/martinal/studies/flow-reversal-study/cases/case_AugustForwardProblem.py", line 59, in <module>
Traceback (most recent call last):
File "/cluster/home/martinal/studies/flow-reversal-study/cases/case_AugustForwardProblem.py", line 59, in <module>
sweep.run(problem_configurations)
sweep.run(problem_configurations)
File "/cluster/home/martinal/studies/flow-reversal-study/cases/sweep.py", line 90, in run
File "/cluster/home/martinal/studies/flow-reversal-study/cases/sweep.py", line 90, in run
sys.exit(main(problem_configurations, sys.argv[1:]))
File "/cluster/home/martinal/studies/flow-reversal-study/cases/sweep.py", line 86, in main
namespace = solve_problem(problem)
sys.exit(main(problem_configurations, sys.argv[1:]))
File "/cluster/home/martinal/studies/flow-reversal-study/cases/sweep.py", line 64, in solve_problem
namespace = solver.solve()
File "/cluster/home/martinal/studies/flow-reversal-study/cases/sweep.py", line 86, in main
namespace = solve_problem(problem)
File "/usit/abel/u1/martinal/nobackup/local/lib/python2.7/site-packages/headflow/core/nssolver.py", line 59, in solve
File "/cluster/home/martinal/studies/flow-reversal-study/cases/sweep.py", line 64, in solve_problem
namespace = solver.solve()
File "/usit/abel/u1/martinal/nobackup/local/lib/python2.7/site-packages/headflow/core/nssolver.py", line 59, in solve
self._summarize()
self._summarize()
File "/usit/abel/u1/martinal/nobackup/local/lib/python2.7/site-packages/headflow/core/nssolver.py", line 72, in _summarize
self._final_memory = get_memory_usage()
File "/usit/abel/u1/martinal/nobackup/local/lib/python2.7/site-packages/headflow/core/nssolver.py", line 72, in _summarize
self._final_memory = get_memory_usage()
File "/usit/abel/u1/martinal/nobackup/local/lib/python2.7/site-packages/headflow/core/utils.py", line 60, in get_memory_usage
File "/usit/abel/u1/martinal/nobackup/local/lib/python2.7/site-packages/headflow/core/utils.py", line 60, in get_memory_usage
mymemory = getoutput("ps -o rss %s" % mypid).split()[1]
IndexError: list index out of range
mymemory = getoutput("ps -o rss %s" % mypid).split()[1]
IndexError: list index out of range
Comments (1)
-
reporter - Log in to comment