- edited description
Does MPI.info only support built-in keys?
Hi, I encountered some error when using mpi4py, as described in this code. When running the code, an exception raises like:
ERROR! While spawning workers through MPI: MPI_ERR_INFO_VALUE: invalid value argument for info object
ERROR! While spawning workers through MPI: MPI_ERR_INFO_VALUE: invalid value argument for info object
When set the info['env']
with info.Set('env', xxx)
, it doesn't work either. Not sure why it fails.
Thank you for your help.
Comments (10)
-
reporter -
reporter -
assigned issue to
-
assigned issue to
-
This is most likely an issue of your backend MPI. As long as you can set/get the key-value pairs in the info object, then mpi4py is fine. The problem is maybe related to MPI not understanding the value you pass for the "env" key. What MPI implementation are you using?
-
-
assigned issue to
- marked as minor
-
assigned issue to
-
reporter I’m using openmpi https://www.open-mpi.org/. I’m not sure why |env| cannot be recognized by it.
-
-
- changed status to invalid
-
reporter Hi Lisandro, I rewrite this line to
info.Set(key='env', value= '\n'.join('%s=%s' % (key, env[key].replace('\n', '')) for key in env))
, and it still doesn,t work. So please give me some advice if you will. And all the environmental variable is like the following, maybe the error is caused by the messy environmental variable?HISTTIMEFORMAT=%Y-%m-%d %H:%M:%S OMPI_MCA_orte_hnp_uri=2751725568.0;usock;tcp://10.141.104.22,10.141.250.86:48235;ud://11300.14.1 OMPI_FIRST_RANKS=0 THEANO_FLAGS=floatX=float32,device=cpu QTDIR=/usr/lib64/qt-3.3 OMPI_FILE_LOCATION=/tmp/openmpi-sessions-508@nmyjs_104_22_0/41988/1/0 LESSOPEN=||/usr/bin/lesspipe.sh %s PMIX_SERVER_URI=pmix-server:32022:/tmp/openmpi-sessions-508@nmyjs_104_22_0/41988/pmix-32022 SSH_CLIENT=10.130.14.91 49426 22 CPATH=/usr/local/cuda-7.5/include/ LOGNAME=chengshanbo USER=chengshanbo PATH=/search/odin/chengshanbo/anaconda2/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/cuda/bin:/opt/ibutils/bin HOME=/search/odin/chengshanbo OMPI_MCA_orte_num_nodes=2 OMPI_COMMAND=python LD_LIBRARY_PATH=/usr/local/lib:/usr/local/cuda/lib64:/usr/local/nccl::/usr/local/cuda/lib64 OMPI_MCA_orte_app_num=0 LANG=en_US QTLIB=/usr/lib64/qt-3.3/lib TERM=xterm-256color SHELL=/bin/bash CVS_RSH=ssh OPAL_OUTPUT_STDERR_FD=37 QTINC=/usr/lib64/qt-3.3/include OMPI_MCA_initial_wdir=/search/odin/chengshanbo/platoon/platoon-fix_attempt/platoon/tests/functional G_BROKEN_FILENAMES=1 PMIX_SECURITY_MODE=native HISTSIZE=1000 OMPI_MCA_orte_local_daemon_uri=2751725568.0;usock;tcp://10.141.104.22,10.141.250.86:48235;ud://11300.14.1 CC=/usr/local/gcc-4.9.2/bin/gcc OMPI_APP_CTX_NUM_PROCS=2 OMPI_MCA_orte_precondition_transports=7869745547cc8f02-d75751c95c24a5f4 LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36: OMPI_MCA_orte_bound_at_launch=1 MODULESHOME=/usr/share/Modules HFI_NO_BACKTRACE=1 OMPI_MCA_ess_base_jobid=2751725569 OMPI_MCA_mpi_warn_on_fork=0 OMPI_MCA_orte_tmpdir_base=/tmp OMPI_MCA_orte_launch=1 OMPI_MCA_orte_ess_node_rank=0 PLATOON_TEST_WORKERS_NUM=2 IPATH_NO_BACKTRACE=1 OMPI_COMM_WORLD_NODE_RANK=0 OMPI_COMM_WORLD_LOCAL_RANK=0 OMPI_COMM_WORLD_LOCAL_SIZE=1 OMPI_MCA_rmaps_base_oversubscribe=1 LC_ALL=zh_CN.UTF-8 OMPI_MCA_ess_base_vpid=0 OMPI_NUM_APP_CTX=1 _=./platoon-launcher MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles OMPI_MCA_pmix=^s1,s2,cray OMPI_UNIVERSE_SIZE=2 LSCOLORS=GxFxCxDxBxegedabagaced PMIX_ID=2751725569.0 OMPI_COMM_WORLD_RANK=0 OMPI_ARGV=-u -m platoon.channel.controller test_multi_node PLATOON_LOGS/test_multi_node --multi CXX=/usr/local/gcc-4.9.2/bin/g++ SSH_TTY=/dev/pts/12 OLDPWD=/search/odin/chengshanbo/platoon/platoon-fix_attempt/platoon/channel LOADEDMODULES= OMPI_MCA_orte_ess_num_procs=2 PMIX_RANK=0 OMPI_MCA_ess=pmi HISTCONTROL=ignoredups SHLVL=2 PWD=/search/odin/chengshanbo/platoon/platoon-fix_attempt/platoon/tests/functional HOSTNAME=nmyjs_104_22 OMPI_MCA_shmem_RUNTIME_QUERY_hint=mmap OMPI_COMM_WORLD_SIZE=2 MAIL=/var/spool/mail/chengshanbo CLICOLOR=1 SSH_CONNECTION=10.130.14.91 49426 10.141.104.22 22 PMIX_NAMESPACE=2751725569 OMPI_MCA_mpi_yield_when_idle=0 BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`}
-
I'm sorry, but I cannot help you. You should make these questions to Open MPI folks, this is not an mpi4py issue nor something that mpi4py is in position to change/improve. As a general advice, sending the whole environment is for sure the problem, though I have not clue why is that. I would start by limiting the contents of the environment you send to child processes to the minimum required for your application to work.
-
reporter Thank you for your reply, Lisandro! I will try to see what happens by contacting openmpi folks. And I will let you know if there's any solution if you want
Thank you. Shanbo
- Log in to comment