mss cannot start in root environment, PROJ_LIB Path, KeyError

[Joern Ungermann]

Starting mss with a fresh installation crashes with Anaconda3 and Windows7 with the following error message:

(mss-1.7.3) C:\Users\icg173>mss
Traceback (most recent call last):
  File "C:\Users\icg173\Anaconda3\envs\mss-1.7.3\Scripts\mss-script.py", line 6, in <module>
    from mslib.msui.mss_pyui import main
  File "C:\Users\icg173\Anaconda3\envs\mss-1.7.3\lib\site-packages\mslib\msui\mss_pyui.py", line 49, in <module>
    from mslib.msui.mss_qt import ui_mainwindow as ui
  File "C:\Users\icg173\Anaconda3\envs\mss-1.7.3\lib\site-packages\mslib\msui\mss_qt.py", line 40, in <module>
    from mslib.utils import config_loader, FatalUserError
  File "C:\Users\icg173\Anaconda3\envs\mss-1.7.3\lib\site-packages\mslib\utils.py", line 47, in <module>
    import mpl_toolkits.basemap.pyproj as pyproj
  File "C:\Users\icg173\Anaconda3\envs\mss-1.7.3\lib\site-packages\mpl_toolkits\basemap\__init__.py", line 146, in <modu
le>
    epsgf = open(os.path.join(pyproj.pyproj_datadir,'epsg'))
FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/icg173/Anaconda3/envs/mss-1.7.3\\share\\proj\\epsg'

The error occurs with pyproj package in version 1.9.5.1-py36hb98d9bb_1

Workaround: Install pyproj in version 1.9.5.1-py36_0 by

    conda install pyproj=1.9.5.1=py36_0

The error itself is triggered within basemap and is caused by pyproj storing the wrong directory location 'C:/Users/icg173/Anaconda3/envs/mss-1.7.3\\share\\proj' instead of 'C:/Users/icg173/Anaconda3/envs/mss-1.7.3\\Library\\share' (and/or it is stored wrongly, but that isn't important from our perspective).

I could alternatively fix it by patching the file <mssenv>/Lib/site-packages/pyproj/datadir.py with the proper path pyproj_datadir="<mssenv>/Library/share" with <mssenv> being the path to my mss environment (e.g. C:\Users\username\Anaconda3\envs\mss-1.7.3).

[Joern Ungermann]

Would https://github.com/ContinuumIO/anaconda-issues/issues be the proper place to report this?

[Reimar Bauer]

@joernu76 yes, there was also some older issue

I have installed it yesterday on Docker and on linux completly fresh. For Linux

pyproj 1.9.5.1 py36h7b21b82_1

[Reimar Bauer]

https://github.com/jswhit/pyproj/issues/17

[Reimar Bauer]

@joernu76 please try the PROJ_DIR env var, or check it.

If this solves it temporary, we should describe it.

Another option is adding environment variable 'PROJ_DIR', set to position of pyproj\data (where the 'epsg' file resides), i.e. \Anaconda2\Lib\site-packages\pyproj\data (and, possibly, restarting to update env. vars). After that running 'print pyproj.pyproj_datadir' shold give correct location, and 'os.path.exists(pyproj.pyproj_datadir + '/epsg')' should return true.

[Joern Ungermann]

I do not see how a PROJ_DIR variable would help things. Didn't do anything on Windows. The path in datadir.py is hard-coded by something. The bug report is kindof old and has been resolved. I think this is a new issue that we should report.

[Reimar Bauer]

We may be can add this rule for windows.

diff --git a/meta.yaml b/meta.yaml
index 2c5527c..4854190 100644
--- a/meta.yaml
+++ b/meta.yaml
@@ -47,6 +47,7 @@ requirements:
     - unicodecsv
     - fs_filepicker
     - future # [py2k]
+    - pyproj 1.9.5.1  # [win and (environ.get('CF_MAX_PY_VER', '') >= '36')]

 test:
   imports:

[Reimar Bauer]

this may be can also be done by doing this change for a new build on conda-forge for 1.7.3

[Reimar Bauer]

very strange I have a similiar problem only in pycharm 2018.2.2

# create dictionary that maps epsg codes to Basemap kwargs.
pyproj_datadir = os.environ['PROJ_LIB']

On terminal I have no issue with this environment variable, but in the IDE it seems not existing

[Reimar Bauer]

The snap installation of pycharm seems not to take care on the env vars.

If I manually add the env var to the run environment it works.

@joernu76 please verify if the problem on your system exists without pycharm

[Reimar Bauer]

after restart and env var removed also the newer pycharm works now. Hmm not reproducable.

[Joern Ungermann]

That PROJ_LIB variable doesn't seem to be used on Windows. I see no reference to that.

[Reimar Bauer]

please try that env var as it is also mentioned

https://github.com/conda-forge/pyproj-feedstock/issues/17

Filipe @ocefpaf 17:52 @ReimarBauer I answered you in the issue but I'll repeat here in case others find it on gitter.

pyproj uses a datadir variable to find proj.4's data, but that was broken due to the bundled proj.4 and some other oddities that module does at install time. We opted to use proj.4's environmental variable PROJ_LIB to find the data files instead. However, that requires:

• you either set that variable yourself pointing to the right place where the data is;
• use conda-env and get that for free during the activation.

I recommend the former.

[Reimar Bauer]

One can check with:

echo $PROJ_LIB  # *nix
echo %PROJ_LIB%  # windows

[Reimar Bauer]

We currently build a fix on that.

https://github.com/conda-forge/basemap-feedstock/issues/30#issuecomment-422453837

Text version:

If you are using Windows I just identified a bug that will be fixed in conda-forge/pyproj-feedstock 18. (Ping @ReimarBauer here who helped me figure that one out.)

However, if you are not on Windows you should use env b/c it is no longer possible to install basemap in the base env. (It is bad practice BTW and you should not install anything in the base env anyway.)

[Reimar Bauer]

@joernu76 can you verify that it works now for you (of course using an env)

[Reimar Bauer]

explanaition by ocefpaf:

note that the origin of all this PROJ_LIB headache is due to some bad practices in pyproj that we need to workaround. pyproj bundles its proj.4 and bake at build time the datadir. In conda-forge we need to build with newer proj.4 and, b/c the package must be relocatable, we need to make pyproj find the datadir anywhere the users installs it. Not using the bundled pyproj is relatively easy and both defaults and conda-forge are doing that now, but in order to make the datadir relocatable we need to rely on proj.4's own activation scripts that sets it. See https://github.com/conda-forge/proj.4-feedstock/blob/master/recipe/scripts/activate.sh#L10

[Reimar Bauer]

We will do a 1.7.3-1 with all known options to solve this for the base env. But 1.7.4 will be likly the last version to be run from base env.

[Reimar Bauer]

set PROJ_LIB env var for the root environment. This var is set by env activation, but not for the root env, fixes #360

→ <<cset e0ce45009f03>>