1. PyPA
  2. Python Packaging Authority Projects
  3. distlib
Issue #23 resolved

distil's noob-barrier is too high

C Anthony Risinger
created an issue

upon revisiting i think distil would work pretty well for my project, but i really want to install distil itself at the end of the day like a normal dist, ie. via a standard source url.

my users are other developers, as such they require an installation tool to try new libraries/etc.

while i understand the reasons for reluctance, would it be possible to make distil a first class citizen along with distlib?

i a push bugfixes/features upstream pretty diligently -- anything useful heads this way ASAP! alas, the lack of a repo is disrupting my workflow (tools!) enough to make for less-than-unfriendly debug/learning/introspection cycles...

distil can exist as a bundle, but should (IMO) first support a standard install like any other dist (eg, uses existing distlib)

in addition to above, i ask because distil.py appears months out of date, possibly back to early may...?

Comments (12)

  1. Vinay Sajip
    1. distil is not meant to be installed like other packages. If you installed it conventionally, it would go into an environment such as system site-packages, user site-packages or a venv. You would have multiple copies and require multiple installations, as with pip - one copy in each venv. One of the points of packaging distil as I have is precisely to avoid that duplication and requirement for multiple installations. I understand that the new and unconventional sometimes throws people, but I can't see why the current packaging mode makes distil harder to use.
    2. If you think that downloading distil.py to a location on the path and renaming it to distil (on POSIX) is too much to expect of users, I can provide a 'get-distil.py' which does those things, or provide a dummy package 'distil' which has a setup.py which doesn't use setuptools or distutils, but just installs the distil.py to a specified directory. (This is not the conventional behaviour of setup.py and so I haven't done it that way.)
    3. I'm still experimenting with various features of distil, so I haven't considered it ready for release in a form where I expect collaboration in the same way as with distlib. I want to get feedback from a user point of view (including developer-as-user). If that feedback comes sufficiently to make it clear that distil functionality is genuinely useful for people, than it would be time for it to settle down, stabilise and invite collaboration in the same way as distlib does.
    4. I appreciate that you have been giving very useful feedback, patches etc. for distlib. Are you saying that you have run into problems with distil where you have wanted to look into them further, but were held back by having no source repository to look at? Can you report those problems here regardless? (Whether bugs, proposals for missing features or changes to existing features.)
    5. distil and distlib evolve together (at the moment), as distil is a test-bed for distlib. That is why they are tightly coupled (at the moment).
    6. Are you concerned that the code is in some way obfuscated because it's not in plain view?
    7. distil itself seems old, but it should still be able to do useful work for the most part. I haven't released a new version recently, partly because there has been little or no feedback about its usefulness or lack of it. There are also a lot of changes in the metadata formats (PEP 426 is undergoing continual changes) and distil relies on stable metadata. The currently released versions of distlib and distil use the pre-PEP 426 metadata format, which I've still retained. Once PEP 426 stabilises, I would expect to release a new version of distlib and a corresponding new version of distil. Before doing that, I would need to update all the metadata on red-dove.com to reflect the stabilised PEP 426. You can see the change log for distil to see if any of the features there for planned 0.1.2 are of use to you, and let me know if that's the case.
  2. C Anthony Risinger reporter
    1. that's sort of the thing... my software creates a standalone bundle-thing with python itself and all it's deps "builtin", ie. i purposefully want a full copy, every time. it's not harder to use per se, just harder to interact-with/understand/mold-to-my-needs. FTR, waf does something similar to what you're looking for; they have waf-light, which will use $WAFLIB or ./waflib, and waf, which is the same as waf-light except it contains an embedded bz2 compressed (zip|tar)file and decompresses to ./waflib-<VER>-<HASH>

    2. im looking for something identical to waf: a light version that does not compress stuff, uses the "system" distlib, and uses ./distil/__init__.py

    3. it is useful ;) i just want to be able to run it easily with HEAD/unreleased distlib versions, etc

    4. no i have not hit any issues really, it's just not lending itself very well for learning... distil represents the most complete and extensive use of distlib available, and i want to follow/tweak it to learn more about distlib. i will probably still report stuff, but i can say it's more annoying, since i cant as easily find/understand a problem if i can't quickly/easily poke/prod the code.

    5. that's cool with me, and i think other users too, if the original source were available and annotated as such.

    6. oh no, nothing like that, more of a "i've built all my tools, process, expectations with the assumption that all code has a backing repo"

    7. not sure about others, but if it were available waf-style (waf-light/waf -> distil-light/distil?) i would already be using it, but i need the fixes/changes in distlib HEAD, and i want to be able to easily track any changes i make (which i can of course do with my VCS, but if upstream has no VCS merging is super-annoying!)

  3. Vinay Sajip

    I'll give some thought to a distil variant that uses a separate distlib, but I'm not sure that won't lead to more problems. For example, if I make some changes to distlib in response to PEP changes, I will generally push those changes as and when they occur (to take advantage of Travis for testing, for example). However, a distil version might suddenly break if it used that tip version of distlib, until I made corresponding changes to distil and released them, plus made any necessary changes to the metadata on red-dove.com. It's a bit of work to keep them synced-up with every push of distlib, though no problem to do when I release a new version of distlib.

  4. C Anthony Risinger reporter

    bah... ok. i think i'll just keep going with direct distlib integration. i want to use distil (especially post-install, so developers can add to their environment easily) but since i must use a current distlib (with custom patches!) to work at all, i'll have to revisit once distil is more accessible -- right now that requires me to unpack the embedded zip, replace distlib, repack and re-embed... a bit too rich for my blood :(

  5. Vinay Sajip

    I can just release a version of distil for your use, will that solve your problem? It will be an interim release which hasn't received as much testing as I'd like, but it would use the latest distlib (but still not using the resources patch - I'm not done integrating that yet, as I've expanded the scope to deal with EXPORTS, RECORD etc).

  6. Vinay Sajip

    A new version of distil is available as distil-f5b6e9-691948.py on the downloads page. This uses the latest distlib and incorporates the usage of resources rather than files for metadata, exports etc. Please feel free to try it out and let me know if you run into any problems.

  7. Vinay Sajip

    I spent a little time creating a 'light' version, which is available from the downloads page as distil-light.py. To use it, you just need to ensure that distlib is on the Python path. You can use either PYTHONPATH or a DISTLIB variable in the environment, whose value will be prepended to sys.path before trying to import distlib.

  8. C Anthony Risinger reporter

    this is all really fantastic stuff, thanks much!! i'd still argue a standard repo as the simplest/preferable, but since these updates were the last thing holding me back (at tech level) from completing my integration, and my initial impressions of distil have been pretty good... so i'll roll with it :) thx again

    i hope to see distil officially support usage as a library, and non-embedded/simple-layout, moving forward!

    will update ASAP! i really want to try this now, but i'm just not able, maybe a few days still...

  9. C Anthony Risinger reporter

    ok, things are moving well (most the packages i need are installing) but i'm currently stuck... appears all dists with a - (hyphen) in the name fail:

    [any@sh dir]$ mkdir -p cache
    [any@sh dir]$ python2.7 distil-f5b6e9-691948.py download -d cache django-extensions
    [any@sh dir]$ python2.7 distil-f5b6e9-691948.py package --format=wheel cache/django-extensions-1.1.1
    Invalid name or filename: u'django-extensions-1.1.1'
    

    ...tried to track it down proper, but wasn't able to in time allotted :( something to do with Wheel() and/or others not converting hyphens to underscores in name?

  10. Vinay Sajip

    The wheel naming problem should be fixed in distil-258fbf-32a4bc.py, available from the downloads page.

    Please open separate issues for these sorts of problems - thanks!

    Update: while the wheel naming is fixed, you still can't install from those wheels - should be fixed soon.

    Further update: I've pulled the older version, try with distil-258fbf-32a4bc.py which should build a wheel from django-extensions and install from that wheel without error.

  11. Log in to comment