PIP Package Installer problem

Issue #382 resolved
Andreas Janz created an issue

Comments (22)

  1. Benjamin Jakimow

    Please provide a list your local pip installation, e.g. using this script:

    import pip
    import pkgutil
    for importer, modname, ispkg in pkgutil.walk_packages(path=pip.__path__,
                                                           prefix=pip.__name__+'.',
                                                           onerror=lambda x: None):
         print(modname)
    

  2. Andreas Janz reporter

    Here is the output. Note that I manually installed Scikit-Learn in the meantime:

    Python Console
    Use iface to access QGIS API interface or Type help(iface) for more info
    Security warning: typing commands from an untrusted source can lead to data loss and/or leak
    import pip
    import pkgutil
    for importer, modname, ispkg in pkgutil.walk_packages(path=pip.path,
    prefix=pip.name+'.',
    onerror=lambda x: None):
    print(modname)
    pip.main
    pip._internal
    pip._internal.build_env
    pip._internal.cache
    pip._internal.cli
    pip._internal.cli.autocompletion
    pip._internal.cli.base_command
    pip._internal.cli.cmdoptions
    pip._internal.cli.command_context
    pip._internal.cli.main_parser
    pip._internal.cli.parser
    pip._internal.cli.req_command
    pip._internal.cli.status_codes
    pip._internal.collector
    pip._internal.commands
    pip._internal.commands.check
    pip._internal.commands.completion
    pip._internal.commands.configuration
    pip._internal.commands.debug
    pip._internal.commands.download
    pip._internal.commands.freeze
    pip._internal.commands.hash
    pip._internal.commands.help
    pip._internal.commands.install
    pip._internal.commands.list
    pip._internal.commands.search
    pip._internal.commands.show
    pip._internal.commands.uninstall
    pip._internal.commands.wheel
    pip._internal.configuration
    pip._internal.distributions
    pip._internal.distributions.base
    pip._internal.distributions.installed
    pip._internal.distributions.source
    pip._internal.distributions.source.legacy
    pip._internal.distributions.wheel
    pip._internal.download
    pip._internal.exceptions
    pip._internal.index
    pip._internal.legacy_resolve
    pip._internal.locations
    pip._internal.main
    pip._internal.models
    pip._internal.models.candidate
    pip._internal.models.format_control
    pip._internal.models.index
    pip._internal.models.link
    pip._internal.models.search_scope
    pip._internal.models.selection_prefs
    pip._internal.models.target_python
    pip._internal.network
    pip._internal.network.auth
    pip._internal.network.cache
    pip._internal.network.session
    pip._internal.network.xmlrpc
    pip._internal.operations
    pip._internal.operations.check
    pip._internal.operations.freeze
    pip._internal.operations.generate_metadata
    pip._internal.operations.prepare
    pip._internal.pep425tags
    pip._internal.pyproject
    pip._internal.req
    pip._internal.req.constructors
    pip._internal.req.req_file
    pip._internal.req.req_install
    pip._internal.req.req_set
    pip._internal.req.req_tracker
    pip._internal.req.req_uninstall
    pip._internal.self_outdated_check
    pip._internal.utils
    pip._internal.utils.appdirs
    pip._internal.utils.compat
    pip._internal.utils.deprecation
    pip._internal.utils.encoding
    pip._internal.utils.filesystem
    pip._internal.utils.filetypes
    pip._internal.utils.glibc
    pip._internal.utils.hashes
    pip._internal.utils.inject_securetransport
    pip._internal.utils.logging
    pip._internal.utils.marker_files
    pip._internal.utils.misc
    pip._internal.utils.models
    pip._internal.utils.packaging
    pip._internal.utils.setuptools_build
    pip._internal.utils.subprocess
    pip._internal.utils.temp_dir
    pip._internal.utils.typing
    pip._internal.utils.ui
    pip._internal.utils.unpacking
    pip._internal.utils.urls
    pip._internal.utils.virtualenv
    pip._internal.vcs
    pip._internal.vcs.bazaar
    pip._internal.vcs.git
    pip._internal.vcs.mercurial
    pip._internal.vcs.subversion
    pip._internal.vcs.versioncontrol
    pip._internal.wheel
    pip._vendor
    pip._vendor.appdirs
    pip._vendor.cachecontrol
    pip._vendor.cachecontrol._cmd
    pip._vendor.cachecontrol.adapter
    pip._vendor.cachecontrol.cache
    pip._vendor.cachecontrol.caches
    pip._vendor.cachecontrol.caches.file_cache
    pip._vendor.cachecontrol.caches.redis_cache
    pip._vendor.cachecontrol.compat
    pip._vendor.cachecontrol.controller
    pip._vendor.cachecontrol.filewrapper
    pip._vendor.cachecontrol.heuristics
    pip._vendor.cachecontrol.serialize
    pip._vendor.cachecontrol.wrapper
    pip._vendor.certifi
    pip._vendor.certifi.main
    pip._vendor.certifi.core
    pip._vendor.chardet
    pip._vendor.chardet.big5freq
    pip._vendor.chardet.big5prober
    pip._vendor.chardet.chardistribution
    pip._vendor.chardet.charsetgroupprober
    pip._vendor.chardet.charsetprober
    pip._vendor.chardet.cli
    pip._vendor.chardet.cli.chardetect
    pip._vendor.chardet.codingstatemachine
    pip._vendor.chardet.compat
    pip._vendor.chardet.cp949prober
    pip._vendor.chardet.enums
    pip._vendor.chardet.escprober
    pip._vendor.chardet.escsm
    pip._vendor.chardet.eucjpprober
    pip._vendor.chardet.euckrfreq
    pip._vendor.chardet.euckrprober
    pip._vendor.chardet.euctwfreq
    pip._vendor.chardet.euctwprober
    pip._vendor.chardet.gb2312freq
    pip._vendor.chardet.gb2312prober
    pip._vendor.chardet.hebrewprober
    pip._vendor.chardet.jisfreq
    pip._vendor.chardet.jpcntx
    pip._vendor.chardet.langbulgarianmodel
    pip._vendor.chardet.langcyrillicmodel
    pip._vendor.chardet.langgreekmodel
    pip._vendor.chardet.langhebrewmodel
    pip._vendor.chardet.langhungarianmodel
    pip._vendor.chardet.langthaimodel
    pip._vendor.chardet.langturkishmodel
    pip._vendor.chardet.latin1prober
    pip._vendor.chardet.mbcharsetprober
    pip._vendor.chardet.mbcsgroupprober
    pip._vendor.chardet.mbcssm
    pip._vendor.chardet.sbcharsetprober
    pip._vendor.chardet.sbcsgroupprober
    pip._vendor.chardet.sjisprober
    pip._vendor.chardet.universaldetector
    pip._vendor.chardet.utf8prober
    pip._vendor.chardet.version
    pip._vendor.colorama
    pip._vendor.colorama.ansi
    pip._vendor.colorama.ansitowin32
    pip._vendor.colorama.initialise
    pip._vendor.colorama.win32
    pip._vendor.colorama.winterm
    pip._vendor.contextlib2
    pip._vendor.distlib
    pip._vendor.distlib._backport
    pip._vendor.distlib._backport.misc
    pip._vendor.distlib._backport.shutil
    pip._vendor.distlib._backport.sysconfig
    pip._vendor.distlib._backport.tarfile
    pip._vendor.distlib.compat
    pip._vendor.distlib.database
    pip._vendor.distlib.index
    pip._vendor.distlib.locators
    pip._vendor.distlib.manifest
    pip._vendor.distlib.markers
    pip._vendor.distlib.metadata
    pip._vendor.distlib.resources
    pip._vendor.distlib.scripts
    pip._vendor.distlib.util
    pip._vendor.distlib.version
    pip._vendor.distlib.wheel
    pip._vendor.distro
    pip._vendor.html5lib
    pip._vendor.html5lib._ihatexml
    pip._vendor.html5lib._inputstream
    pip._vendor.html5lib._tokenizer
    pip._vendor.html5lib._trie
    pip._vendor.html5lib._trie._base
    pip._vendor.html5lib._trie.datrie
    pip._vendor.html5lib._trie.py
    pip._vendor.html5lib._utils
    pip._vendor.html5lib.constants
    pip._vendor.html5lib.filters
    pip._vendor.html5lib.filters.alphabeticalattributes
    pip._vendor.html5lib.filters.base
    pip._vendor.html5lib.filters.inject_meta_charset
    pip._vendor.html5lib.filters.lint
    pip._vendor.html5lib.filters.optionaltags
    pip._vendor.html5lib.filters.sanitizer
    pip._vendor.html5lib.filters.whitespace
    pip._vendor.html5lib.html5parser
    pip._vendor.html5lib.serializer
    pip._vendor.html5lib.treeadapters
    pip._vendor.html5lib.treeadapters.genshi
    pip._vendor.html5lib.treeadapters.sax
    pip._vendor.html5lib.treebuilders
    pip._vendor.html5lib.treebuilders.base
    pip._vendor.html5lib.treebuilders.dom
    pip._vendor.html5lib.treebuilders.etree
    pip._vendor.html5lib.treebuilders.etree_lxml
    pip._vendor.html5lib.treewalkers
    pip._vendor.html5lib.treewalkers.base
    pip._vendor.html5lib.treewalkers.dom
    pip._vendor.html5lib.treewalkers.etree
    pip._vendor.html5lib.treewalkers.etree_lxml
    pip._vendor.html5lib.treewalkers.genshi
    pip._vendor.idna
    pip._vendor.idna.codec
    pip._vendor.idna.compat
    pip._vendor.idna.core
    pip._vendor.idna.idnadata
    pip._vendor.idna.intranges
    pip._vendor.idna.package_data
    pip._vendor.idna.uts46data
    pip._vendor.ipaddress
    pip._vendor.msgpack
    pip._vendor.msgpack._version
    pip._vendor.msgpack.exceptions
    pip._vendor.msgpack.fallback
    pip._vendor.packaging
    pip._vendor.packaging.about
    pip._vendor.packaging._compat
    pip._vendor.packaging._structures
    pip._vendor.packaging.markers
    pip._vendor.packaging.requirements
    pip._vendor.packaging.specifiers
    pip._vendor.packaging.tags
    pip._vendor.packaging.utils
    pip._vendor.packaging.version
    pip._vendor.pep517
    pip._vendor.pep517._in_process
    pip._vendor.pep517.build
    pip._vendor.pep517.check
    pip._vendor.pep517.colorlog
    pip._vendor.pep517.compat
    pip._vendor.pep517.dirtools
    pip._vendor.pep517.envbuild
    pip._vendor.pep517.meta
    pip._vendor.pep517.wrappers
    pip._vendor.pkg_resources
    pip._vendor.pkg_resources.py31compat
    pip._vendor.progress
    pip._vendor.progress.bar
    pip._vendor.progress.counter
    pip._vendor.progress.spinner
    pip._vendor.pyparsing
    pip._vendor.pytoml
    pip._vendor.pytoml.core
    pip._vendor.pytoml.parser
    pip._vendor.pytoml.test
    pip._vendor.pytoml.utils
    pip._vendor.pytoml.writer
    pip._vendor.requests
    pip._vendor.requests.version
    pip._vendor.requests._internal_utils
    pip._vendor.requests.adapters
    pip._vendor.requests.api
    pip._vendor.requests.auth
    pip._vendor.requests.certs
    pip._vendor.requests.compat
    pip._vendor.requests.cookies
    pip._vendor.requests.exceptions
    pip._vendor.requests.help
    pip._vendor.requests.hooks
    pip._vendor.requests.models
    pip._vendor.requests.packages
    pip._vendor.requests.sessions
    pip._vendor.requests.status_codes
    pip._vendor.requests.structures
    pip._vendor.requests.utils
    pip._vendor.retrying
    pip._vendor.six
    pip._vendor.urllib3
    pip._vendor.urllib3._collections
    pip._vendor.urllib3.connection
    pip._vendor.urllib3.connectionpool
    pip._vendor.urllib3.contrib
    pip._vendor.urllib3.contrib._appengine_environ
    pip._vendor.urllib3.contrib._securetransport
    pip._vendor.urllib3.contrib._securetransport.bindings
    pip._vendor.urllib3.contrib._securetransport.low_level
    pip._vendor.urllib3.contrib.appengine
    pip._vendor.urllib3.contrib.ntlmpool
    pip._vendor.urllib3.contrib.pyopenssl
    pip._vendor.urllib3.contrib.securetransport
    pip._vendor.urllib3.contrib.socks
    pip._vendor.urllib3.exceptions
    pip._vendor.urllib3.fields
    pip._vendor.urllib3.filepost
    pip._vendor.urllib3.packages
    pip._vendor.urllib3.packages.backports
    pip._vendor.urllib3.packages.backports.makefile
    pip._vendor.urllib3.packages.six
    pip._vendor.urllib3.packages.ssl_match_hostname
    pip._vendor.urllib3.packages.ssl_match_hostname._implementation
    pip._vendor.urllib3.poolmanager
    pip._vendor.urllib3.request
    pip._vendor.urllib3.response
    pip._vendor.urllib3.util
    pip._vendor.urllib3.util.connection
    pip._vendor.urllib3.util.queue
    pip._vendor.urllib3.util.request
    pip._vendor.urllib3.util.response
    pip._vendor.urllib3.util.retry
    pip.vendor.urllib3.util.ssl
    pip._vendor.urllib3.util.timeout
    pip._vendor.urllib3.util.url
    pip._vendor.urllib3.util.wait
    pip._vendor.webencodings
    pip._vendor.webencodings.labels
    pip._vendor.webencodings.mklabels
    pip._vendor.webencodings.tests
    pip._vendor.webencodings.x_user_defined

  3. Benjamin Jakimow

    your pip is outdated, please update to version >= 20.0, which is the default for QGIS 3.10 LTR (checked with OSGeo4W installer)

  4. Andreas Janz reporter
    • changed status to open

    If there is a minimal PIP version required, the PIP Package Installer should notice and handle the PIP update as well (if possible).

    Otherwise we leave the user with the same problem as before - he has to manually open the shell and execute a command, which would be user-unfriendly.

    Any ideas how to update PIP from within QGIS?

  5. Andreas Janz reporter

    I was able to update PIP from within QGIS Console using:

    cmd = r"c:\osgeo4~1\bin\python3.exe -m pip install --upgrade pip"
    
    import subprocess
    process = subprocess.run(cmd, check=True, stdout=subprocess.PIPE, universal_newlines=True)
    print(process.stdout)
    

  6. Benjamin Jakimow

    I don’t thinks that updating pip from inside QGIS is a good idea. It’s too much a core package that we shouldn’t touch without using the parent packaging system (OSGeo4W, hombebrew, apt-get, …) At least for windows it should get updated anyway just by running the standard OSGeo4W installer.

  7. Andreas Janz reporter

    At least for windows it should get updated anyway just by running the standard OSGeo4W installer.

    Running OSGeo4W from the shell after installing the EnMAP-Box is not very convinient for the user.

    If possible we shoud avoid this step. If it is too complicated, we should give instructions how to proceed. The current error message will confuse people:

  8. Benjamin Jakimow

    It is the convenient to update everything in the OSGeo4W system, e.g. to pull QGIS updates. Instead of hacking package the manager we should better add a link to our documents and just allow to install pip package to --user.

    By the way, your update command does not work without admin rights.

  9. Fabian Thiel

    On Ubuntu 18.04 LTR there is also a problem, as it comes (at least on my clean VM setup) with python 3.6.9 and pip 9.0.1

    No module named 'pip._internal'
    

  10. Benjamin Jakimow

    I try to make the pip usage independent of the internal pip package structure.

    But there remains the problem that at least parts of the hubflow routines require python >= 3.7 (see #374)

  11. Andreas Janz reporter

    I thought Python >= 3.7 is only a problem for the CI tests, or am I missing something here?

  12. Benjamin Jakimow

    As shown by the CI tests, it is a problem for each users who runs QGIS with python 3.6 - which is the default on most linux systems.

  13. Andreas Janz reporter

    🤔 that’s a problem. Perhaps I need to mock the future annotation imports for Python < 3.7

  14. Benjamin Jakimow

    @Andreas Janz @Fabian Thiel please test the recent version.

    The installation is now done in a parallel thread, errors are shown in the console and I was able to use it on windows 10 QGIS 3.10 LTM + 3.12 on OSGeo4W + conda-forge QGIS.

    How about linux?

  15. Andreas Janz reporter

    Ok, under Python 3.6 you can use Strings as annotation and PyCharm is still able to infere the correct type:

  16. Benjamin Jakimow

    As this now relates to the python version and not the package installer it’s probably better to close this issue and create a new one?

  17. Log in to comment