support tracking changes in requirements.txt without full rebuild

Issue #296 on hold
Maxim Kovgan created an issue

Use case

during the development, you often switch between new and old requirements. This currently forces the user to run tox -r. Sometimes it's not serious, but for automated tests in CI, using internet - it can be several minutes overhead of ~5 or even more minutes. This becomes problematic for short builds: test suite that runs 1 minutes is waiting for tox ~5 minutes.

scenarios I'd like to handle are:

  1. addition of new packages
  2. removal of currently installed packages
  3. upgrade of existing package(s)
  4. downgrade of existing package(s)

Scope

only handle 4 situations, nothing else

Implementation:

Use difflib module's functionality: it already has ndiff() seemingly very useful for this purpose. based on that diff we should generate 2 lists:

  1. Packages_to_remove
  2. Packages_to_install

Testing should be done first on the function that properly detects what needs to be removed and what needs to be installed.

  1. Initially - just remove anything that is changed, and install it from scratch
  2. later - we might want to optimize:
    1. packages to remove - to contain packages that are supposed to be removed or downgraded
    2. packages to install - should contain packages that are supposed to be installed or upgraded with -I option (i.e. older will be uninstalled)

A relatively easy to implement approach would be to compare: . current pip freeze output (on the left) requirements.txt (on the right)

As a result, if there are diffs (lines with "-" and lines with "+"), those with "-" can be uninstalled, and those with "+" - installed.

Comments (4)

  1. Florian Bruhin

    This looks like something to be done after #149, right? It also doesn't seem to be specific to requirements.txt, e.g. when having the dependencies in tox.ini the same would apply as well.

  2. Log in to comment