PIP Package Installer problem

Benjamin Jakimow

Which pip version do you have installed?

‌

2020-03-23T19:56:12+00:00

Benjamin Jakimow

changed status to open

2020-03-23T19:56:20+00:00

Andreas Janz reporter

The one that shipped with my QGIS: PIP v19.3.1

2020-03-23T19:58:21+00:00

Benjamin Jakimow

Please provide a list your local pip installation, e.g. using this script:

import pip
import pkgutil
for importer, modname, ispkg in pkgutil.walk_packages(path=pip.__path__,
                                                       prefix=pip.__name__+'.',
                                                       onerror=lambda x: None):
     print(modname)

‌

2020-03-23T20:02:26+00:00

Andreas Janz reporter

Here is the output. Note that I manually installed Scikit-Learn in the meantime:

Python Console
Use iface to access QGIS API interface or Type help(iface) for more info
Security warning: typing commands from an untrusted source can lead to data loss and/or leak
import pip
import pkgutil
for importer, modname, ispkg in pkgutil.walk_packages(path=pip.path,
prefix=pip.name+'.',
onerror=lambda x: None):
print(modname)
pip.main
pip._internal
pip._internal.build_env
pip._internal.cache
pip._internal.cli
pip._internal.cli.autocompletion
pip._internal.cli.base_command
pip._internal.cli.cmdoptions
pip._internal.cli.command_context
pip._internal.cli.main_parser
pip._internal.cli.parser
pip._internal.cli.req_command
pip._internal.cli.status_codes
pip._internal.collector
pip._internal.commands
pip._internal.commands.check
pip._internal.commands.completion
pip._internal.commands.configuration
pip._internal.commands.debug
pip._internal.commands.download
pip._internal.commands.freeze
pip._internal.commands.hash
pip._internal.commands.help
pip._internal.commands.install
pip._internal.commands.list
pip._internal.commands.search
pip._internal.commands.show
pip._internal.commands.uninstall
pip._internal.commands.wheel
pip._internal.configuration
pip._internal.distributions
pip._internal.distributions.base
pip._internal.distributions.installed
pip._internal.distributions.source
pip._internal.distributions.source.legacy
pip._internal.distributions.wheel
pip._internal.download
pip._internal.exceptions
pip._internal.index
pip._internal.legacy_resolve
pip._internal.locations
pip._internal.main
pip._internal.models
pip._internal.models.candidate
pip._internal.models.format_control
pip._internal.models.index
pip._internal.models.link
pip._internal.models.search_scope
pip._internal.models.selection_prefs
pip._internal.models.target_python
pip._internal.network
pip._internal.network.auth
pip._internal.network.cache
pip._internal.network.session
pip._internal.network.xmlrpc
pip._internal.operations
pip._internal.operations.check
pip._internal.operations.freeze
pip._internal.operations.generate_metadata
pip._internal.operations.prepare
pip._internal.pep425tags
pip._internal.pyproject
pip._internal.req
pip._internal.req.constructors
pip._internal.req.req_file
pip._internal.req.req_install
pip._internal.req.req_set
pip._internal.req.req_tracker
pip._internal.req.req_uninstall
pip._internal.self_outdated_check
pip._internal.utils
pip._internal.utils.appdirs
pip._internal.utils.compat
pip._internal.utils.deprecation
pip._internal.utils.encoding
pip._internal.utils.filesystem
pip._internal.utils.filetypes
pip._internal.utils.glibc
pip._internal.utils.hashes
pip._internal.utils.inject_securetransport
pip._internal.utils.logging
pip._internal.utils.marker_files
pip._internal.utils.misc
pip._internal.utils.models
pip._internal.utils.packaging
pip._internal.utils.setuptools_build
pip._internal.utils.subprocess
pip._internal.utils.temp_dir
pip._internal.utils.typing
pip._internal.utils.ui
pip._internal.utils.unpacking
pip._internal.utils.urls
pip._internal.utils.virtualenv
pip._internal.vcs
pip._internal.vcs.bazaar
pip._internal.vcs.git
pip._internal.vcs.mercurial
pip._internal.vcs.subversion
pip._internal.vcs.versioncontrol
pip._internal.wheel
pip._vendor
pip._vendor.appdirs
pip._vendor.cachecontrol
pip._vendor.cachecontrol._cmd
pip._vendor.cachecontrol.adapter
pip._vendor.cachecontrol.cache
pip._vendor.cachecontrol.caches
pip._vendor.cachecontrol.caches.file_cache
pip._vendor.cachecontrol.caches.redis_cache
pip._vendor.cachecontrol.compat
pip._vendor.cachecontrol.controller
pip._vendor.cachecontrol.filewrapper
pip._vendor.cachecontrol.heuristics
pip._vendor.cachecontrol.serialize
pip._vendor.cachecontrol.wrapper
pip._vendor.certifi
pip._vendor.certifi.main
pip._vendor.certifi.core
pip._vendor.chardet
pip._vendor.chardet.big5freq
pip._vendor.chardet.big5prober
pip._vendor.chardet.chardistribution
pip._vendor.chardet.charsetgroupprober
pip._vendor.chardet.charsetprober
pip._vendor.chardet.cli
pip._vendor.chardet.cli.chardetect
pip._vendor.chardet.codingstatemachine
pip._vendor.chardet.compat
pip._vendor.chardet.cp949prober
pip._vendor.chardet.enums
pip._vendor.chardet.escprober
pip._vendor.chardet.escsm
pip._vendor.chardet.eucjpprober
pip._vendor.chardet.euckrfreq
pip._vendor.chardet.euckrprober
pip._vendor.chardet.euctwfreq
pip._vendor.chardet.euctwprober
pip._vendor.chardet.gb2312freq
pip._vendor.chardet.gb2312prober
pip._vendor.chardet.hebrewprober
pip._vendor.chardet.jisfreq
pip._vendor.chardet.jpcntx
pip._vendor.chardet.langbulgarianmodel
pip._vendor.chardet.langcyrillicmodel
pip._vendor.chardet.langgreekmodel
pip._vendor.chardet.langhebrewmodel
pip._vendor.chardet.langhungarianmodel
pip._vendor.chardet.langthaimodel
pip._vendor.chardet.langturkishmodel
pip._vendor.chardet.latin1prober
pip._vendor.chardet.mbcharsetprober
pip._vendor.chardet.mbcsgroupprober
pip._vendor.chardet.mbcssm
pip._vendor.chardet.sbcharsetprober
pip._vendor.chardet.sbcsgroupprober
pip._vendor.chardet.sjisprober
pip._vendor.chardet.universaldetector
pip._vendor.chardet.utf8prober
pip._vendor.chardet.version
pip._vendor.colorama
pip._vendor.colorama.ansi
pip._vendor.colorama.ansitowin32
pip._vendor.colorama.initialise
pip._vendor.colorama.win32
pip._vendor.colorama.winterm
pip._vendor.contextlib2
pip._vendor.distlib
pip._vendor.distlib._backport
pip._vendor.distlib._backport.misc
pip._vendor.distlib._backport.shutil
pip._vendor.distlib._backport.sysconfig
pip._vendor.distlib._backport.tarfile
pip._vendor.distlib.compat
pip._vendor.distlib.database
pip._vendor.distlib.index
pip._vendor.distlib.locators
pip._vendor.distlib.manifest
pip._vendor.distlib.markers
pip._vendor.distlib.metadata
pip._vendor.distlib.resources
pip._vendor.distlib.scripts
pip._vendor.distlib.util
pip._vendor.distlib.version
pip._vendor.distlib.wheel
pip._vendor.distro
pip._vendor.html5lib
pip._vendor.html5lib._ihatexml
pip._vendor.html5lib._inputstream
pip._vendor.html5lib._tokenizer
pip._vendor.html5lib._trie
pip._vendor.html5lib._trie._base
pip._vendor.html5lib._trie.datrie
pip._vendor.html5lib._trie.py
pip._vendor.html5lib._utils
pip._vendor.html5lib.constants
pip._vendor.html5lib.filters
pip._vendor.html5lib.filters.alphabeticalattributes
pip._vendor.html5lib.filters.base
pip._vendor.html5lib.filters.inject_meta_charset
pip._vendor.html5lib.filters.lint
pip._vendor.html5lib.filters.optionaltags
pip._vendor.html5lib.filters.sanitizer
pip._vendor.html5lib.filters.whitespace
pip._vendor.html5lib.html5parser
pip._vendor.html5lib.serializer
pip._vendor.html5lib.treeadapters
pip._vendor.html5lib.treeadapters.genshi
pip._vendor.html5lib.treeadapters.sax
pip._vendor.html5lib.treebuilders
pip._vendor.html5lib.treebuilders.base
pip._vendor.html5lib.treebuilders.dom
pip._vendor.html5lib.treebuilders.etree
pip._vendor.html5lib.treebuilders.etree_lxml
pip._vendor.html5lib.treewalkers
pip._vendor.html5lib.treewalkers.base
pip._vendor.html5lib.treewalkers.dom
pip._vendor.html5lib.treewalkers.etree
pip._vendor.html5lib.treewalkers.etree_lxml
pip._vendor.html5lib.treewalkers.genshi
pip._vendor.idna
pip._vendor.idna.codec
pip._vendor.idna.compat
pip._vendor.idna.core
pip._vendor.idna.idnadata
pip._vendor.idna.intranges
pip._vendor.idna.package_data
pip._vendor.idna.uts46data
pip._vendor.ipaddress
pip._vendor.msgpack
pip._vendor.msgpack._version
pip._vendor.msgpack.exceptions
pip._vendor.msgpack.fallback
pip._vendor.packaging
pip._vendor.packaging.about
pip._vendor.packaging._compat
pip._vendor.packaging._structures
pip._vendor.packaging.markers
pip._vendor.packaging.requirements
pip._vendor.packaging.specifiers
pip._vendor.packaging.tags
pip._vendor.packaging.utils
pip._vendor.packaging.version
pip._vendor.pep517
pip._vendor.pep517._in_process
pip._vendor.pep517.build
pip._vendor.pep517.check
pip._vendor.pep517.colorlog
pip._vendor.pep517.compat
pip._vendor.pep517.dirtools
pip._vendor.pep517.envbuild
pip._vendor.pep517.meta
pip._vendor.pep517.wrappers
pip._vendor.pkg_resources
pip._vendor.pkg_resources.py31compat
pip._vendor.progress
pip._vendor.progress.bar
pip._vendor.progress.counter
pip._vendor.progress.spinner
pip._vendor.pyparsing
pip._vendor.pytoml
pip._vendor.pytoml.core
pip._vendor.pytoml.parser
pip._vendor.pytoml.test
pip._vendor.pytoml.utils
pip._vendor.pytoml.writer
pip._vendor.requests
pip._vendor.requests.version
pip._vendor.requests._internal_utils
pip._vendor.requests.adapters
pip._vendor.requests.api
pip._vendor.requests.auth
pip._vendor.requests.certs
pip._vendor.requests.compat
pip._vendor.requests.cookies
pip._vendor.requests.exceptions
pip._vendor.requests.help
pip._vendor.requests.hooks
pip._vendor.requests.models
pip._vendor.requests.packages
pip._vendor.requests.sessions
pip._vendor.requests.status_codes
pip._vendor.requests.structures
pip._vendor.requests.utils
pip._vendor.retrying
pip._vendor.six
pip._vendor.urllib3
pip._vendor.urllib3._collections
pip._vendor.urllib3.connection
pip._vendor.urllib3.connectionpool
pip._vendor.urllib3.contrib
pip._vendor.urllib3.contrib._appengine_environ
pip._vendor.urllib3.contrib._securetransport
pip._vendor.urllib3.contrib._securetransport.bindings
pip._vendor.urllib3.contrib._securetransport.low_level
pip._vendor.urllib3.contrib.appengine
pip._vendor.urllib3.contrib.ntlmpool
pip._vendor.urllib3.contrib.pyopenssl
pip._vendor.urllib3.contrib.securetransport
pip._vendor.urllib3.contrib.socks
pip._vendor.urllib3.exceptions
pip._vendor.urllib3.fields
pip._vendor.urllib3.filepost
pip._vendor.urllib3.packages
pip._vendor.urllib3.packages.backports
pip._vendor.urllib3.packages.backports.makefile
pip._vendor.urllib3.packages.six
pip._vendor.urllib3.packages.ssl_match_hostname
pip._vendor.urllib3.packages.ssl_match_hostname._implementation
pip._vendor.urllib3.poolmanager
pip._vendor.urllib3.request
pip._vendor.urllib3.response
pip._vendor.urllib3.util
pip._vendor.urllib3.util.connection
pip._vendor.urllib3.util.queue
pip._vendor.urllib3.util.request
pip._vendor.urllib3.util.response
pip._vendor.urllib3.util.retry
pip.vendor.urllib3.util.ssl
pip._vendor.urllib3.util.timeout
pip._vendor.urllib3.util.url
pip._vendor.urllib3.util.wait
pip._vendor.webencodings
pip._vendor.webencodings.labels
pip._vendor.webencodings.mklabels
pip._vendor.webencodings.tests
pip._vendor.webencodings.x_user_defined

2020-03-23T20:04:20+00:00

Benjamin Jakimow

changed status to closed

your pip is outdated, please update to version >= 20.0, which is the default for QGIS 3.10 LTR (checked with OSGeo4W installer)

2020-03-23T20:22:37+00:00

Andreas Janz reporter

changed status to open

If there is a minimal PIP version required, the PIP Package Installer should notice and handle the PIP update as well (if possible).

Otherwise we leave the user with the same problem as before - he has to manually open the shell and execute a command, which would be user-unfriendly.

Any ideas how to update PIP from within QGIS?

2020-03-24T06:54:03+00:00

Andreas Janz reporter

I was able to update PIP from within QGIS Console using:

‌

cmd = r"c:\osgeo4~1\bin\python3.exe -m pip install --upgrade pip"

import subprocess
process = subprocess.run(cmd, check=True, stdout=subprocess.PIPE, universal_newlines=True)
print(process.stdout)

‌

2020-03-24T07:13:14+00:00

Benjamin Jakimow

I don’t thinks that updating pip from inside QGIS is a good idea. It’s too much a core package that we shouldn’t touch without using the parent packaging system (OSGeo4W, hombebrew, apt-get, …) At least for windows it should get updated anyway just by running the standard OSGeo4W installer.

2020-03-24T07:41:56+00:00

Andreas Janz reporter

At least for windows it should get updated anyway just by running the standard OSGeo4W installer.

Running OSGeo4W from the shell after installing the EnMAP-Box is not very convinient for the user.

If possible we shoud avoid this step. If it is too complicated, we should give instructions how to proceed. The current error message will confuse people:

‌

2020-03-24T07:53:53+00:00

Benjamin Jakimow

It is the convenient to update everything in the OSGeo4W system, e.g. to pull QGIS updates. Instead of hacking package the manager we should better add a link to our documents and just allow to install pip package to --user.

By the way, your update command does not work without admin rights.

2020-03-24T08:01:23+00:00

Fabian Thiel

On Ubuntu 18.04 LTR there is also a problem, as it comes (at least on my clean VM setup) with python 3.6.9 and pip 9.0.1

‌

No module named 'pip._internal'

‌

2020-03-24T10:43:54+00:00

Benjamin Jakimow

I try to make the pip usage independent of the internal pip package structure.

But there remains the problem that at least parts of the hubflow routines require python >= 3.7 (see ~~#374~~)

2020-03-24T11:00:44+00:00

Andreas Janz reporter

I thought Python >= 3.7 is only a problem for the CI tests, or am I missing something here?

2020-03-24T16:14:06+00:00

Benjamin Jakimow

As shown by the CI tests, it is a problem for each users who runs QGIS with python 3.6 - which is the default on most linux systems.

2020-03-24T16:29:34+00:00

Andreas Janz reporter

that’s a problem. Perhaps I need to mock the future annotation imports for Python < 3.7

2020-03-24T17:37:57+00:00

Benjamin Jakimow

@Andreas Janz @Fabian Thiel please test the recent version.

The installation is now done in a parallel thread, errors are shown in the console and I was able to use it on windows 10 QGIS 3.10 LTM + 3.12 on OSGeo4W + conda-forge QGIS.

How about linux?