Does MPI.info only support built-in keys?

[Shanbo]

Hi, I encountered some error when using mpi4py, as described in this code. When running the code, an exception raises like:

ERROR! While spawning workers through MPI: MPI_ERR_INFO_VALUE: invalid value argument for info object
ERROR! While spawning workers through MPI: MPI_ERR_INFO_VALUE: invalid value argument for info object

When set the info['env'] with info.Set('env', xxx), it doesn't work either. Not sure why it fails.

Thank you for your help.

[Lisandro Dalcin]

This is most likely an issue of your backend MPI. As long as you can set/get the key-value pairs in the info object, then mpi4py is fine. The problem is maybe related to MPI not understanding the value you pass for the "env" key. What MPI implementation are you using?

[Shanbo]

I’m using openmpi https://www.open-mpi.org/. I’m not sure why |env| cannot be recognized by it.

[Lisandro Dalcin]

https://groups.google.com/forum/m/#!topic/mpi4py/cYyJKyqEQTY

[Shanbo]

Hi Lisandro, I rewrite this line to info.Set(key='env', value= '\n'.join('%s=%s' % (key, env[key].replace('\n', '')) for key in env)), and it still doesn,t work. So please give me some advice if you will. And all the environmental variable is like the following, maybe the error is caused by the messy environmental variable?

HISTTIMEFORMAT=%Y-%m-%d %H:%M:%S 
OMPI_MCA_orte_hnp_uri=2751725568.0;usock;tcp://10.141.104.22,10.141.250.86:48235;ud://11300.14.1
OMPI_FIRST_RANKS=0
THEANO_FLAGS=floatX=float32,device=cpu
QTDIR=/usr/lib64/qt-3.3
OMPI_FILE_LOCATION=/tmp/openmpi-sessions-508@nmyjs_104_22_0/41988/1/0
LESSOPEN=||/usr/bin/lesspipe.sh %s
PMIX_SERVER_URI=pmix-server:32022:/tmp/openmpi-sessions-508@nmyjs_104_22_0/41988/pmix-32022
SSH_CLIENT=10.130.14.91 49426 22
CPATH=/usr/local/cuda-7.5/include/
LOGNAME=chengshanbo
USER=chengshanbo
PATH=/search/odin/chengshanbo/anaconda2/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/cuda/bin:/opt/ibutils/bin
HOME=/search/odin/chengshanbo
OMPI_MCA_orte_num_nodes=2
OMPI_COMMAND=python
LD_LIBRARY_PATH=/usr/local/lib:/usr/local/cuda/lib64:/usr/local/nccl::/usr/local/cuda/lib64
OMPI_MCA_orte_app_num=0
LANG=en_US
QTLIB=/usr/lib64/qt-3.3/lib
TERM=xterm-256color
SHELL=/bin/bash
CVS_RSH=ssh
OPAL_OUTPUT_STDERR_FD=37
QTINC=/usr/lib64/qt-3.3/include
OMPI_MCA_initial_wdir=/search/odin/chengshanbo/platoon/platoon-fix_attempt/platoon/tests/functional
G_BROKEN_FILENAMES=1
PMIX_SECURITY_MODE=native
HISTSIZE=1000
OMPI_MCA_orte_local_daemon_uri=2751725568.0;usock;tcp://10.141.104.22,10.141.250.86:48235;ud://11300.14.1
CC=/usr/local/gcc-4.9.2/bin/gcc
OMPI_APP_CTX_NUM_PROCS=2
OMPI_MCA_orte_precondition_transports=7869745547cc8f02-d75751c95c24a5f4
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
OMPI_MCA_orte_bound_at_launch=1
MODULESHOME=/usr/share/Modules
HFI_NO_BACKTRACE=1
OMPI_MCA_ess_base_jobid=2751725569
OMPI_MCA_mpi_warn_on_fork=0
OMPI_MCA_orte_tmpdir_base=/tmp
OMPI_MCA_orte_launch=1
OMPI_MCA_orte_ess_node_rank=0
PLATOON_TEST_WORKERS_NUM=2
IPATH_NO_BACKTRACE=1
OMPI_COMM_WORLD_NODE_RANK=0
OMPI_COMM_WORLD_LOCAL_RANK=0
OMPI_COMM_WORLD_LOCAL_SIZE=1
OMPI_MCA_rmaps_base_oversubscribe=1
LC_ALL=zh_CN.UTF-8
OMPI_MCA_ess_base_vpid=0
OMPI_NUM_APP_CTX=1
_=./platoon-launcher
MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles
OMPI_MCA_pmix=^s1,s2,cray
OMPI_UNIVERSE_SIZE=2
LSCOLORS=GxFxCxDxBxegedabagaced
PMIX_ID=2751725569.0
OMPI_COMM_WORLD_RANK=0
OMPI_ARGV=-u -m platoon.channel.controller test_multi_node PLATOON_LOGS/test_multi_node --multi
CXX=/usr/local/gcc-4.9.2/bin/g++
SSH_TTY=/dev/pts/12
OLDPWD=/search/odin/chengshanbo/platoon/platoon-fix_attempt/platoon/channel
LOADEDMODULES=
OMPI_MCA_orte_ess_num_procs=2
PMIX_RANK=0
OMPI_MCA_ess=pmi
HISTCONTROL=ignoredups
SHLVL=2
PWD=/search/odin/chengshanbo/platoon/platoon-fix_attempt/platoon/tests/functional
HOSTNAME=nmyjs_104_22
OMPI_MCA_shmem_RUNTIME_QUERY_hint=mmap
OMPI_COMM_WORLD_SIZE=2
MAIL=/var/spool/mail/chengshanbo
CLICOLOR=1
SSH_CONNECTION=10.130.14.91 49426 10.141.104.22 22
PMIX_NAMESPACE=2751725569
OMPI_MCA_mpi_yield_when_idle=0
BASH_FUNC_module()=() {  eval `/usr/bin/modulecmd bash $*`}

[Lisandro Dalcin]

I'm sorry, but I cannot help you. You should make these questions to Open MPI folks, this is not an mpi4py issue nor something that mpi4py is in position to change/improve. As a general advice, sending the whole environment is for sure the problem, though I have not clue why is that. I would start by limiting the contents of the environment you send to child processes to the minimum required for your application to work.

[Shanbo]

Thank you for your reply, Lisandro! I will try to see what happens by contacting openmpi folks. And I will let you know if there's any solution if you want

Thank you. Shanbo