Source

hhgp / source / creation.txt

==================
Creating a Package
==================

Basics: Creating and Distributing Distributions
===============================================

If you have some useful Python :term:`modules <module>` that you think
others might benefit from, but aren't sure how to go about packaging
them up and distributing them, then this short document is for you. By
the end of it, you'll be a contributor to the :ref:`pypi_info`.

For a more detailed look at packaging a larger project, see this
:doc:`example <example>`.

Let's begin.

Background
----------

Suppose you've written a couple modules to help you keep track of your
towel (``location.py`` and ``utils.py``), and you'd like to share
them.  First thing to do is come up with a CamelCase project name for
them. Let's go with "TowelStuff" since it seems appropriate and also
it has not yet been used on the :ref:`pypi_info`.

.. _directory_layout:

Arranging your file and directory structure
-------------------------------------------

"TowelStuff" will be the name of our project as well as the name of our
:term:`distribution`. We should also come up with a :term:`package`
name within which our modules will reside (to avoid naming conflicts
with other modules). For this example, there's only one package, so
let's reuse the project name and go with "towelstuff".  Make the
layout of your project directory (described below) look like this::

    TowelStuff/
        bin/
        CHANGES.txt
        docs/
        LICENSE.txt
        MANIFEST.in
        README.txt
        setup.py
        towelstuff/
            __init__.py
            location.py
            utils.py
            test/
                __init__.py
                test_location.py
                test_utils.py



Here's what you should do for each of those listed above:

.. _bin_description:

* Put into ``bin`` any scripts you've written that use your
  ``towelstuff`` package and which you think would be useful for your
  users. If you don't have any, then remove the ``bin`` directory.

.. _changes_txt_description:

* For now, the ``CHANGES.txt`` file should only contain::

      v<version>, <date> -- Initial release.

  since this is your very first version (version number will be
  described below) and there are no changes to report.

.. _docs_description:

* The ``docs`` dir should contain any design docs, implementation notes, a
  FAQ, or any other docs you've written. For now, stick to plain text
  files ending in ".txt". This author (JohnMG) likes to use `Pandoc's
  Markdown <http://johnmacfarlane.net/pandoc/>`_, but many Pythoneers
  use :term:`reStructuredText <reStructuredText>`.

.. _license_txt_description:

* The ``LICENSE.txt`` file is often just a copy/paste of your license
  of choice.  We recommend going with a commonly-used license, such as
  the GPL, BSD, or MIT.

.. _manifest_in_description:

* The ``MANIFEST.in`` file should contain this::

      include *.txt
      recursive-include docs *.txt

.. _readme_txt_description:

* The ``README.txt`` file should be written in :term:`reST <reStructuredText>`
  so that the PyPI can use it to generate your project's PyPI page. Here's a
  10-second intro to reST that you might use to start with::

      ===========
      Towel Stuff
      ===========

      Towel Stuff provides such and such and so and so. You might find
      it most useful for tasks involving <x> and also <y>. Typical usage
      often looks like this::

          #!/usr/bin/env python

          from towelstuff import location
          from towelstuff import utils

          if utils.has_towel():
              print "Your towel is located:", location.where_is_my_towel()

      (Note the double-colon and 4-space indent formatting above.)

      Paragraphs are separated by blank lines. *Italics*, **bold**,
      and ``monospace`` look like this.


      A Section
      =========

      Lists look like this:

      * First

      * Second. Can be multiple lines
        but must be indented properly.

      A Sub-Section
      -------------

      Numbered lists look like you'd expect:

      1. hi there

      2. must be going

      Urls are http://like.this and links can be
      written `like this <http://www.example.com/foo/bar>`_.

  You might also consider adding a "Contributors" section and/or a
  "Thanks also to" section to list the names of people who've helped.

  By the way, to see how the above ``README.txt`` looks rendered in
  html, see the `TowelStuff project
  <http://pypi.python.org/pypi/TowelStuff/>`_ at the PyPI.

.. _setup_py_description:

* ``setup.py`` -- Create this file and make it look like this::

      from distutils.core import setup

      setup(
          name='TowelStuff',
          version='0.1.0',
          author='J. Random Hacker',
          author_email='jrh@example.com',
          packages=['towelstuff', 'towelstuff.test'],
          scripts=['bin/stowe-towels.py','bin/wash-towels.py'],
          url='http://pypi.python.org/pypi/TowelStuff/',
          license='LICENSE.txt',
          description='Useful towel-related stuff.',
          long_description=open('README.txt').read(),
      )

  but, of course, replace the towel stuff with your own project and
  package names. For more details about picking version numbers, see
  :doc:`versioning`, but '0.1.0' will work just fine for a first
  release (this is using the common "major.minor.micro" numbering
  convention).

  If you have no scripts to distribute (and thus no ``bin`` dir), you
  can remove the above line which begins with "scripts".

.. _towelstuff_description:

* Inside the ``towelstuff`` directory, ``__init__.py`` can be
  empty. Likewise, inside ``towelstuff/test``, *that* ``__init__.py``
  can be empty as well. If you have no tests written yet, you can
  leave the two other module files in ``towelstuff/test`` empty for
  now too. When writing your tests, use the standard ``unittest``
  module.

For our example, TowelStuff does not depend upon any other
distributions (it only depends upon what's already in the Python
standard library). To specify dependencies upon other distributions,
see the more detailed :doc:`example`.


Creating your distribution file
-------------------------------

Create your distribution file like so::

    $ cd path/to/TowelStuff
    $ python setup.py sdist

Running that last command will create a ``MANIFEST`` file in your
project directory, and also a ``dist`` and ``build`` directory. Inside
that ``dist`` directory is the distribution that you'll be uploading
to the PyPI. In our case, the distribution file will be named
``TowelStuff-0.1.0.tgz``. Feel free to poke around in the ``dist``
directory to look at your distribution.


Uploading your distribution file
--------------------------------

Before uploading you first need to create an account at
http://pypi.python.org/pypi . Once that's complete, register your
distribution at the PyPI like so::

    $ cd path/to/TowelStuff
    $ python setup.py register

Use your existing login (choice #1). It will prompt you to save the
login info for future use (to which I agree). Then upload::

    $ python setup.py sdist upload

This builds the distribution one last time and then uploads it.

Thanks for your contribution!


Updating your distribution
--------------------------

Down the road, after you've made updates to your distribution and wish
to make a new release:

1. increment the version number in your ``setup.py`` file,
2. update your ``CHANGES.txt`` file,
3. if necessary, update the "Contributors" and "Thanks also to" sections
   of your ``README.txt`` file.
4. run ``python setup.py sdist upload`` again.



Entry points
------------

.. note:: XXX Ditch the example here and use the `parrot example
<http://bitbucket.org/carljm/sample-distutils2-project/>`_, developed by Carl
Meyer, to illustrate how this works.

Entry points are a Setuptools/Distribute feature that’s really handy in one
specific case: register something under a specific key in package A that
package B can query for.

Distribute itself uses it. If you’re packaging your project up properly, you've
probably used the ``console_scripts`` entry point::

   setup(name='zest.releaser',
         ...
         entry_points={
             'console_scripts':
                 ['release = zest.releaser.release:main',
                  'prerelease = zest.releaser.prerelease:main',
                  ]}
         )

``console_scripts`` is an entry point that Setuptools looks up. It looks up all
entry points registered under the name console_scripts and uses that
information to generate scripts. In the above example that’d be a bin/release
script that runs the main() method in zest/releaser/release.py.

You can use that for your own extension mechanism. For ``zest.releaser`` I
needed some extension mechanism. I wanted to be able to do extra things on
prerelease/release/postrelease time.

- Downloading an external javascript library into a package that cannot be
  stored in (zope's) svn repository directly due to licensing issues. Before
  packaging and releasing it, that is. Automatically so you don’t forget it.

- Uploading a version.cfg to
  ``scp://somewhere/kgs/ourmainproduct-version.cfg``
  after making a release to use it as a so-called “known good set” (KGS).

- Possibly modifying values (like a commit message) inside zest.releaser itself
  while doing a release. (I do get modification requests from time to time “hey,
  can you make x and y configurable”). So now every zest.releaser step
  (prerelease, release, postrelease) is splitted in two: a calculation phase
  and a “doing” phase. The results of the first phase are stored in a dict
  that gets used in the second phase. And you can register an entry point that
  gets passed that dict so you can modify it. See the entry point documentation
  of zest.releaser for details.

An entry point for zest.releaser is configured like this in your setup.py::

    entry_points={
        'console_scripts':
            ['myscript = my.package.scripts:main'],
        'zest.releaser.prereleaser.middle':
            ['dosomething = my.package.some:some_entrypoint, ]
    }

Replace prereleaser and middle in zest.releaser.prereleaser.middle with
prerelease/release/postrelease and before/middle/after where needed. (For this
specific zest.releaser example).

Now, how to use this in your program? The best way is to show a quick example
from zest.releaser where we query and use one of our entry points::

    import pkg_resources

    ...
    def run_entry_point(data):
        # Note: data is zest.releaser specific: we want to pass
        # something to the plugin group = 'zest.releaser.prerelease.middle'

        for entrypoint in pkg_resources.iter_entry_points(group=group):
            # Grab the function that is the actual plugin.
            plugin = entrypoint.load() # Call the plugin
            plugin(data)

So: pretty easy and simple way to allow other packages to register something
that you want to know. Extra plugins, extra render methods, extra functionality
you want to register in your web application, etcetera.

Packaging for a Particular Operating System (OS)
================================================

General Packaging Guidelines for Unix
-------------------------------------

General Packaging Guidelines for Windows
----------------------------------------