1. dirkbaechle
  2. SCons_experimental


dirkbaechle  committed 7f1ca36

- added original formic code

  • Participants
  • Parent commits 4dd2c2d
  • Branches ant_glob

Comments (0)

Files changed (7)

File src/engine/SCons/Node/FS.py

View file
  • Ignore whitespace
         Nodes if a corresponding entry exists in a Repository (either
         an in-memory Node or something on disk).
-        By defafult, the glob() function matches entries that exist
+        By default, the glob() function matches entries that exist
         on-disk, in addition to in-memory Nodes.  Setting the "ondisk"
         argument to False (or some other non-true value) causes the glob()
         function to only match in-memory Nodes.  The default behavior is

File src/engine/SCons/Node/formic/LICENSE.txt

View file
  • Ignore whitespace
+                   GNU GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+                            Preamble
+  The GNU General Public License is a free, copyleft license for
+software and other kinds of works.
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+the GNU General Public License is intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.  We, the Free Software Foundation, use the
+GNU General Public License for most of our software; it applies also to
+any other work released this way by its authors.  You can apply it to
+your programs, too.
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+  To protect your rights, we need to prevent others from denying you
+these rights or asking you to surrender the rights.  Therefore, you have
+certain responsibilities if you distribute copies of the software, or if
+you modify it: responsibilities to respect the freedom of others.
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must pass on to the recipients the same
+freedoms that you received.  You must make sure that they, too, receive
+or can get the source code.  And you must show them these terms so they
+know their rights.
+  Developers that use the GNU GPL protect your rights with two steps:
+(1) assert copyright on the software, and (2) offer you this License
+giving you legal permission to copy, distribute and/or modify it.
+  For the developers' and authors' protection, the GPL clearly explains
+that there is no warranty for this free software.  For both users' and
+authors' sake, the GPL requires that modified versions be marked as
+changed, so that their problems will not be attributed erroneously to
+authors of previous versions.
+  Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the manufacturer
+can do so.  This is fundamentally incompatible with the aim of
+protecting users' freedom to change the software.  The systematic
+pattern of such abuse occurs in the area of products for individuals to
+use, which is precisely where it is most unacceptable.  Therefore, we
+have designed this version of the GPL to prohibit the practice for those
+products.  If such problems arise substantially in other domains, we
+stand ready to extend this provision to those domains in future versions
+of the GPL, as needed to protect the freedom of users.
+  Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish to
+avoid the special danger that patents applied to a free program could
+make it effectively proprietary.  To prevent this, the GPL assures that
+patents cannot be used to render the program non-free.
+  The precise terms and conditions for copying, distribution and
+modification follow.
+                       TERMS AND CONDITIONS
+  0. Definitions.
+  "This License" refers to version 3 of the GNU General Public License.
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+  A "covered work" means either the unmodified Program or a work based
+on the Program.
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+  1. Source Code.
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+  The Corresponding Source for a work in source code form is that
+same work.
+  2. Basic Permissions.
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+  4. Conveying Verbatim Copies.
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+  5. Conveying Modified Source Versions.
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+  6. Conveying Non-Source Forms.
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+  7. Additional Terms.
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+  8. Termination.
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+  9. Acceptance Not Required for Having Copies.
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+  10. Automatic Licensing of Downstream Recipients.
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+  11. Patents.
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+  12. No Surrender of Others' Freedom.
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+  13. Use with the GNU Affero General Public License.
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU Affero General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the special requirements of the GNU Affero General Public License,
+section 13, concerning interaction through a network will apply to the
+combination as such.
+  14. Revised Versions of this License.
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU General Public License, you may choose any version ever published
+by the Free Software Foundation.
+  If the Program specifies that a proxy can decide which future
+versions of the GNU General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+  15. Disclaimer of Warranty.
+  16. Limitation of Liability.
+  17. Interpretation of Sections 15 and 16.
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.

File src/engine/SCons/Node/formic/VERSION.txt

View file
  • Ignore whitespace

File src/engine/SCons/Node/formic/__init__.py

View file
  • Ignore whitespace
+# Formic: An implementation of Apache Ant FileSet globs
+# Copyright (C) 2012, Aviser LLP, Singapore.
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# GNU General Public License for more details.
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+An implementation of Apache Ant globs.
+* The :mod:`formic.formic` module contains the main class
+  :class:`~formic.formic.FileSet`
+* The :mod:`formic.command` module contains the command-line interface.
+from formic import FileSet, Pattern, get_version

File src/engine/SCons/Node/formic/command.py

View file
  • Ignore whitespace
+# Formic: An implementation of Apache Ant FileSet globs
+# Copyright (C) 2012, Aviser LLP, Singapore.
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# GNU General Public License for more details.
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""The command-line glue-code for :command:`formic`. Call :func:`~formic.command.main()`
+with the command-line arguments.
+Full usage of the command is::
+  usage: formic [-i [INCLUDE [INCLUDE ...]]] [-e [EXCLUDE [EXCLUDE ...]]]
+               [--no-default-excludes] [--no-symlinks] [-r] [-h] [--usage]
+               [--version]
+               [directory]
+  Search the file system using Apache Ant globs
+  Directory:
+    directory             The directory from which to start the search (defaults
+                          to current working directory)
+  Globs:
+    -i [INCLUDE [INCLUDE ...]], --include [INCLUDE [INCLUDE ...]]
+                          One or more Ant-like globs in include in the search.If
+                          not specified, then all files are implied
+    -e [EXCLUDE [EXCLUDE ...]], --exclude [EXCLUDE [EXCLUDE ...]]
+                          One or more Ant-like globs in include in the search
+    --no-default-excludes
+                          Do not include the default excludes
+    --no-symlinks         Do not include symlinks
+  Output:
+     -r, --relative       Print file paths relative to directory.
+  Information:
+    -h, --help            Prints this help and exits
+    --usage               Prints additional help on globs and exits
+    --version             Prints the version of formic and exits
+from argparse import ArgumentParser, SUPPRESS, RawDescriptionHelpFormatter
+from sys import argv, stdout
+from os import path
+from pkg_resources import resource_string
+from formic import FileSet, FormicError, get_version
+DESCRIPTION = """Search the file system using Apache Ant globs"""
+"""For documentation, source code and other information, please visit:
+This program comes with ABSOLUTELY NO WARRANTY. See license for details.
+This is free software, and you are welcome to redistribute it
+under certain conditions; for details, run
+> formic --license
+Formic is Copyright (C) 2012, Aviser LLP, Singapore"""
+def create_parser():
+    """Creates and returns the command line parser, an
+     :class:`argparser.ArgumentParser` instance."""
+    parser = ArgumentParser(formatter_class=RawDescriptionHelpFormatter,
+                            description=DESCRIPTION,
+                            epilog=EPILOG,
+                            add_help=False)
+    directory = parser.add_argument_group("Directory")
+    directory.add_argument(dest='directory', action="store", default=None, nargs="?",
+                           help="The directory from which to start the search "
+                                "(defaults to current working directory)")
+    globs = parser.add_argument_group("Globs")
+    globs.add_argument('-i', '--include', action="store", nargs="*",
+                           help="One or more Ant-like globs in include in the search."
+                                "If not specified, then all files are implied")
+    globs.add_argument('-e', '--exclude', action="store", nargs="*",
+                           help="One or more Ant-like globs in include in the search")
+    globs.add_argument('--no-default-excludes', dest="default_excludes",
+                       action="store_false", default=True,
+                       help="Do not include the default excludes")
+    globs.add_argument('--no-symlinks', action="store_true", default=False,
+                       help="Do not include symlinks")
+    output = parser.add_argument_group("Output")
+    output.add_argument('-r', '--relative', action="store_true", default=False,
+                        help="Print file paths relative to directory.")
+    info = parser.add_argument_group("Information")
+    info.add_argument('-h', '--help', action='store_true', default=False,
+                      help="Prints this help and exits")
+    info.add_argument('--usage', action='store_true', default=False,
+                      help="Prints additional help on globs and exits")
+    info.add_argument('--version', action='store_true', default=False,
+                      help="Prints the version of formic and exits")
+    info.add_argument('--license', action="store_true", help=SUPPRESS)
+    return parser
+def main(*kw):
+    """Command line entry point; arguments must match those defined in
+    in :meth:`create_parser()`; returns 0 for success, else 1.
+    Example::
+      command.main("-i", "**/*.py", "--no-default-excludes")
+    Runs formic printing out all .py files in the current working directory
+    and its children to ``sys.stdout``.
+    If *kw* is None, :func:`main()` will use ``sys.argv``."""
+    parser = create_parser()
+    args = parser.parse_args(kw if kw else None)
+    if args.help:
+        parser.print_help()
+    elif args.usage:
+        print """Ant Globs
+Apache Ant fileset is documented at the Apache Ant project:
+* http://ant.apache.org/manual/dirtasks.html#patterns
+Ant Globs are like simple file globs (they use ? and * in the same way), but
+include powerful ways for selecting directories. The examples below use the
+Ant glob naming, so a leading slash represents the top of the search, *not* the
+root of the file system.
+    *.py
+            Selects every matching file anywhere in the whole tree
+                Matches /foo.py and /bar/foo.py
+                but not /foo.pyc or /bar/foo.pyc/
+    /*.py
+            Selects every matching file in the root of the directory (but no
+            deeper).
+            Matches /foo.py but not /bar/foo.py
+    /myapp/**
+            Matches all files under /myapp and below.
+    /myapp/**/__init__.py
+            Matches all __init__.py files /myapp and below.
+    dir1/__init__.py
+            Selects every __init__.py in directory dir1. dir1
+            directory can be anywhere in the directory tree
+            Matches /dir1/file.py, /dir3/dir1/file.py and
+            /dir3/dir2/dir1/file.py but not /dir1/another/__init__.py.
+    **/dir1/__init__.py
+            Same as above.
+    /**/dir1/__init__.py
+            Same as above.
+    /myapp/**/dir1/__init__.py
+            Selects every __init__.py in dir1 in the directory tree
+            /myapp under the root.
+            Matches /myapp/dir1/__init__.py and /myapp/dir2/dir1/__init__.py
+            but not /myapp/file.txt and /dir1/file.txt
+Default excludes
+Ant FileSet (and Formic) has built-in patterns to screen out a lot of
+development 'noise', such as hidden VCS files and directories. The full list is
+    * http://www.aviser.asia/formic/api.html#formic.formic.get_initial_default_excludes
+Default excludes can be simply switched off on both the command line and the
+API, for example::
+    $ formic -i "*.py" -e "__init__.py" "**/*test*/" "test_*" --no-default-excludes
+    elif args.version:
+        print "formic", get_version(), "http://www.aviser.asia/formic"
+    elif args.license:
+        print resource_string(__name__, "LICENSE.txt")
+    else:
+        try:
+            fileset = FileSet(directory=args.directory,
+                              include=args.include if args.include else ["*"],
+                              exclude=args.exclude,
+                              default_excludes=args.default_excludes,
+                              symlinks=not args.no_symlinks)
+        except FormicError as exception:
+            parser.print_usage()
+            print exception.message
+            return 1
+        prefix = fileset.get_directory()
+        for directory, file_name in fileset.files():
+            if args.relative:
+                stdout.write(".")
+            else:
+                stdout.write(prefix)
+            if dir:
+                stdout.write(path.sep)
+                stdout.write(directory)
+            stdout.write(path.sep)
+            stdout.write(file_name)
+            stdout.write("\n")
+    return 0
+def entry_point():
+    """Entry point for command line; calls :meth:`~formic.command.main()` and then
+    :func:`sys.exit()` with the return value."""
+    result = main()
+    exit(result)
+if __name__ == "__main__":
+    main(*argv[1:])

File src/engine/SCons/Node/formic/formic.py

View file
  • Ignore whitespace
+# Formic: An implementation of Apache Ant FileSet globs
+# Copyright (C) 2012, Aviser LLP, Singapore.
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# GNU General Public License for more details.
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""An implementation of Ant Globs.
+The main entry points for this modules are:
+* :class:`FileSet`: A collection of include and exclude globs starting at a specific
+  directory.
+  * :meth:`FileSet.files()`: A generator returning the matched files as
+    directory/file tuples
+  * :meth:`FileSet.qualified_files()`: A generator returning the matched files
+    as qualified paths
+* :class:`Pattern`: An individual glob
+from os import path, getcwd, walk
+from fnmatch import fnmatch, filter as fnfilter
+from itertools import chain
+def get_version():
+    """Returns the version of formic.
+    This method retrieves the version from VERSION.txt, and it should be
+    exactly the same as the version retrieved from the package manager"""
+    try:
+        # Try with the package manager, if present
+        from pkg_resources import resource_string
+        return resource_string(__name__, "VERSION.txt")
+    except:
+        # If the package manager is not present, try reading the file
+        version = path.join(path.dirname(__file__), "VERSION.txt")
+        with open(version, "r") as f:
+            return f.readlines()[0]
+def get_path_components(directory):
+    """Breaks a path to a directory into a (drive, list-of-folders) tuple
+    :param directory:
+    :return: a tuple consisting of the drive (if any) and an ordered list of
+             folder names
+    """
+    drive, dirs = path.splitdrive(directory)
+    folders = []
+    while dirs != path.sep and dirs != "":
+        dirs, folder = path.split(dirs)
+        if folder != "":
+            folders.append(folder)
+    folders.reverse()
+    return drive, folders
+def reconstitute_path(drive, folders):
+    """Reverts a tuple from `get_path_components` into a path.
+    :param drive: A drive (eg 'c:'). Only applicable for NT systems
+    :param folders: A list of folder names
+    :return: A path comprising the drive and list of folder names. The path terminate
+             with a `os.path.sep` *only* if it is a root directory
+    """
+    reconstituted = path.join(drive, path.sep, *folders)
+    return reconstituted
+def is_root(directory):
+    """Returns true if the directory is root (eg / on UNIX or c:\\ on Windows)"""
+    _, folders = get_path_components(directory)
+    return len(folders) == 0
+class FormicError(Exception):
+    """Formic errors, such as misconfigured arguments and internal exceptions"""
+    def __init__(self, message=None):
+        super(FormicError, self).__init__(message)
+class Matcher(object):
+    """An abstract class that holds some pattern to be matched;
+    ``matcher.match(string)`` returns a boolean indicating whether the string
+    matches the pattern.
+    The :meth:`Matcher.create()` method is a Factory that creates instances of
+    various subclasses."""
+    @staticmethod
+    def create(pattern):
+        """Factory for :class:`Matcher` instances; returns a :class:`Matcher`
+        suitable for matching the supplied pattern"""
+        if "?" in pattern or "*" in pattern:
+            return FNMatcher(pattern)
+        else:
+            return ConstantMatcher(pattern)
+    def __init__(self, pattern):
+        self.pattern = path.normcase(pattern)
+        self.pp      = pattern
+    def match(self, _):
+        """:class:`Matcher` is an abstract class - this will raise a
+        :exc:`FormicError`"""
+        raise FormicError("Match should not be directly constructed")
+    def __eq__(self, other):
+        return (isinstance(other, type(self)) and
+                self.pattern == other.pattern)
+    def __ne__(self, other):
+        return (not isinstance(other, type(self)) or
+                self.pattern != other.pattern)
+    def __hash__(self):
+        return self.pattern.__hash__()
+    def __str__(self):
+        return self.pp
+    def __repr__(self):
+        return self.pp
+class FNMatcher(Matcher):
+    """A :class:`Matcher` that matches simple file/directory wildcards as per
+    DOS or Unix.
+    * ``FNMatcher("*.py")`` matches all Python files in a given directory.
+    * ``FNMatcher("?ed")`` matches bed, fed, wed but not failed
+    :class:`FNMatcher` internally uses :func:`fnmatch.fnmatch()` to implement
+    :meth:`Matcher.match`"""
+    def __init__(self, pattern):
+        super(FNMatcher, self).__init__(pattern)
+    def match(self, string):
+        """Returns True if the pattern matches the string"""
+        return fnmatch(string, self.pattern)
+class ConstantMatcher(Matcher):
+    """A :class:`Matcher` for matching the constant passed in the constructor.
+    This is used to more efficiently match path and file elements that
+    do not have a wild-card, eg ``__init__.py``"""
+    def __init__(self, pattern):
+        super(ConstantMatcher, self).__init__(pattern)
+    def match(self, string):
+        """Returns True if the argument matches the constant."""
+        return self.pattern == path.normcase(string)
+class Section(object):
+    """A minimal object that holds fragments of a :class:`Pattern` path.
+    Each :class:`Section` holds a list of pattern fragments matching some
+    contiguous portion of a full path, separated by ``/**/`` from other
+    :class:`Section` instances.
+    For example, the :class:`Pattern` ``/top/second/**/sub/**end/*`` is stored
+    as a list of three :class:`Section` objects:
+    1. ``Section(["top", "second"])``
+    2. ``Section(["sub"])``
+    3. ``Section(["end"])``
+    """
+    def __init__(self, elements):
+        assert elements
+        self.elements    = []
+        self.bound_start = False
+        self.bound_end   = False
+        for element in elements:
+            self.elements.append(Matcher.create(element))
+        self.length = len(self.elements)
+        self.str = "/".join(str(e) for e in self.elements)
+    def match_iter(self, path_elements, start_at):
+        """A generator that searches over *path_elements* (starting from the
+        index *start_at*), yielding for each match.
+        Each value yielded is the index into *path_elements* to the first element
+        *after* each match. In other words, the returned index has already
+        consumed the matching path elements of this :class:`Section`.
+        Matches work by finding a contiguous group of path elements that
+        match the list of :class:`Matcher` objects in this :class:`Section`
+        as they are naturally paired.
+        This method includes an implementation optimization that simplifies
+        the search for :class:`Section` instances containing a single path
+        element. This produces significant performance improvements.
+        """
+        if self.length == 1:
+            return self._match_iter_single(path_elements, start_at)
+        else:
+            return self._match_iter_generic(path_elements, start_at)
+    def _match_iter_generic(self, path_elements, start_at):
+        """Implementation of match_iter for >1 self.elements"""
+        length = len(path_elements)
+        # If bound to start, we stop searching at the first element
+        if self.bound_start:
+            end = 1
+        else:
+            end = length - self.length + 1
+        # If bound to end, we start searching as late as possible
+        if self.bound_end:
+            start = length - self.length
+        else:
+            start = start_at
+        if start > end or start < start_at or end > length - self.length + 1:
+            # It's impossible to match. Either
+            # 1) the search has a fixed start and end, and path_elements
+            #    does not have enough elements for a match, or
+            # 2) To match the bound_end, we have to start before the start_at,
+            #    which means the search is impossible
+            # 3) The end is after the last possible end point in path_elements
+            return
+        for index in range(start, end):
+            matched = True
+            i = index
+            for matcher in self.elements:
+                element = path_elements[i]
+                i += 1
+                if not matcher.match(element):
+                    matched = False
+                    break
+            if matched:
+                yield index + self.length
+    def _match_iter_single(self, path_elements, start_at):
+        """Implementation of match_iter optimized for self.elements of length 1"""
+        length = len(path_elements)
+        if length == 0:
+            return
+        # If bound to end, we start searching as late as possible
+        if self.bound_end:
+            start = length - 1
+            if start < start_at:
+                return
+        else:
+            start = start_at
+        # If bound to start, we stop searching at the first element
+        if self.bound_start:
+            end = 1
+        else:
+            end = length
+            if start > end:
+                # It's impossible to match
+                # the search has a fixed start and end, and path_elements
+                # does not have enough elements for a match, or
+                return
+        for index in range(start, end):
+            element = path_elements[index]
+            if self.elements[0].match(element):
+                yield index + 1
+    def __eq__(self, other):
+        return isinstance(other, Section) and self.str == other.str
+    def __ne__(self, other):
+        return not isinstance(other, Section) or self.str != other.str
+    def __hash__(self):
+        return self.str.__hash__()
+    def __str__(self):
+        return self.str
+class MatchType(object):
+    """An enumeration of different match/non-match types to optimize
+    the search algorithm.
+    There are two special considerations in match results that derive
+    from the fact that Ant globs can be 'bound' to the start of the path
+    being evaluated (eg bound start: ``/Documents/**``).
+    The various match possibilities are bitfields using the members
+    starting ``BIT_``."""
+    BIT_MATCH              = 1 # M
+    # The Match types             -BIT FIELDS-
+    #                             X  M   A   N
+    NO_MATCH                    = 0
+    MATCH                       =    1
+    NO_MATCH_NO_SUBDIRECTORIES  =            4
+class Pattern(object):
+    """Represents a single Ant Glob.
+    The :class:`Pattern` object compiles the pattern into several components:
+    * *file_pattern*: The a pattern for matching files (not directories)
+      eg, for ``test/*.py``, the file_pattern is ``*.py``. This is always
+      the text after the final ``/`` (if any). If the end of the pattern
+      is a ``/``, then an implicit ``**/*`` is added to the end of the pattern.
+    * *bound_start*: True if the start of the pattern is 'bound' to the
+      start of the path. If the pattern starts with a ``/``, the
+      start is bound.
+    * *bound_end*: True if the end of the pattern is bound to the immediate
+      parent directory where the file matching is occurring. This is True if
+      the pattern specifies a directory before the file pattern, eg
+      ``**/test/*``
+    * *sections*: A list of :class:`Section` instances. Each :class:`Section`
+      represents a contiguous series of path patterns, and :class:`Section`
+      instances are separated whenever there is a ``**`` in the glob.
+    :class:`Pattern` also normalises the glob, removing redundant path elements
+    (eg ``**/**/test/*`` resolves to ``**/test/*``) and normalises the case of
+    the path elements (resolving difficulties with case insensitive file
+    systems)
+    """
+    def __init__(self, glob):
+        glob = glob.replace('\\', '/').replace('//', '/')
+        self.sections = []
+        self.str      = []
+        elements = Pattern._simplify(glob.split('/'))
+        self.bound_start = elements[0]  != "**"
+        if elements[-1] != "**":
+            # Patterns like "constant", "cons*" or "c?nst?nt"
+            self.file_pattern = path.normcase(elements[-1])
+            del elements[-1]
+        else:
+            self.file_pattern = "*"
+        # Optimization: Set self.file_filter to be a specific pattern
+        # validating algorithm for the specific pattern
+        if self.file_pattern == "*":
+            # The pattern matches everything
+            self.file_filter = lambda files: files
+        elif "*" in self.file_pattern or "?" in self.file_pattern:
+            # The pattern is a glob. Use fnmatch.filter
+            self.file_filter = lambda files: fnfilter(files, self.file_pattern)
+        else:
+            # This is a 'constant' pattern - use comprehension
+            self.file_filter = lambda files: [ file for file in files if path.normcase(file) == self.file_pattern ]
+        if elements:
+            self.bound_end = elements[-1] != "**"
+        else:
+            self.bound_end = self.bound_start
+        fragment = []
+        for element in elements:
+            if element == '**':
+                if fragment:
+                    self.sections.append(Section(fragment))
+                fragment = []
+            else:
+                fragment.append(element)
+        if fragment:
+            self.sections.append(Section(fragment))
+        # Propagate the bound start/end to the sections
+        if self.bound_start and self.sections:
+            self.sections[0].bound_start = True
+        if self.bound_end and self.sections:
+            self.sections[-1].bound_end = True
+    @staticmethod
+    def _simplify(elements):
+        """Simplifies and normalizes the list of elements removing
+        redundant/repeated elements and normalising upper/lower case
+        so case sensitivity is resolved here."""
+        simplified = []
+        previous = None
+        for element in elements:
+            if element == "..":
+                raise FormicError("Invalid glob:"
+                                  " Cannot have '..' in a glob: {0}".
+                                    format("/".join(elements)))
+            elif element == ".":
+                # . in a path does not do anything
+                pass
+            elif element == "**" and previous == "**":
+                # Remove repeated "**"s
+                pass
+            else:
+                simplified.append(path.normcase(element))
+                previous = element
+        if simplified[-1] == "":
+            # Trailing slash shorthand for /**
+            simplified[-1] = "**"
+        # Ensure the pattern either:
+        #  * Starts with a "**", or
+        #  * Starts with the first real element of the glob
+        if simplified[0] == "":
+            # "" means the pattern started with a slash.
+            del simplified[0]
+        else:
+            if simplified[0] != "**":
+                simplified.insert(0, "**")
+        return simplified
+    def match_directory(self, path_elements):
+        """Returns a :class:`MatchType` for the directory, expressed as a list of path
+        elements, match for the :class:`Pattern`.
+        If ``self.bound_start`` is True, the first :class:`Section` must match
+        from the first directory element.
+        If ``self.bound_end`` is True, the last :class:`Section` must match
+        the last contiguous elements of *path_elements*.
+        """
+        def match_recurse(is_start, sections, path_elements, location):
+            """A private function for implementing the recursive search.
+            The function takes the first section from sections and tries to
+            match this against the elements in path_elements, starting from
+            the location'th element in that list.
+            If sections is empty, this is taken to mean all sections have
+            been previously matched, therefore a match has been found.
+            * is_start: True if this is the call starting the recursion. False if
+              this call is recursing
+            * sections: A list of the remaining sections (sections not yet matched)
+            * path_elements: A list of directory names, each element being a single directory
+            * location: index into path_elements for where the search should start
+            """
+            if sections:
+                section = sections[0]
+                any_match = False
+                for end in section.match_iter(path_elements, location):
+                    any_match = True
+                    match = match_recurse(False, sections[1:], path_elements, end)
+                    if match | MatchType.MATCH:
+                        return match
+                # No match found
+                if is_start and self.bound_start and not any_match:
+                    # This this is the start of the recursion AND the pattern
+                    # is bound to the start of the path ("/start/**") AND this
+                    # did not match, then no subdirectories are possible either
+                    if len(path_elements) >= len(section.elements):
+                        return MatchType.NO_MATCH_NO_SUBDIRECTORIES
+                    else:
+                        # Optimization: Don't search subdirectories when
+                        #  i) we have an fixed start to the pattern, eg "/Users/myuser/**"
+                        #  ii) We have a path not matching the first, anchored, section, eg "/usr" or "/Users/another"
+                        # Need to check whether the last path element matches the corresponding element in section
+                        # If it does, return NO_MATCH (it's incomplete)
+                        # If, however, the element's don't match, then no further match is possible,
+                        # So return NO_MATCH_NO_SUBDIRECTORIES
+                        if section.length > len(path_elements) > 0:
+                            if not section.elements[len(path_elements)-1].match(path_elements[-1]):
+                                return MatchType.NO_MATCH_NO_SUBDIRECTORIES
+                        return MatchType.NO_MATCH
+                else:
+                    return MatchType.NO_MATCH
+            else:
+                # Termination of the recursion after FINDING the match.
+                if len(self.sections) == 1 and self.bound_start and self.bound_end:
+                    # If this pattern is of the form "/test/*" it matches
+                    # just THIS directory and no subdirectories
+                    return MatchType.MATCH_BUT_NO_SUBDIRECTORIES
+                elif self.bound_end:
+                    # "**/test/*" matches just this directory
+                    # and allows subdirectories to also match
+                    return MatchType.MATCH
+                else:
+                    # If the pattern is not bound to the end of the path (eg
+                    # NOT "**/term/**") the pattern matches all subdirectories
+                    return MatchType.MATCH_ALL_SUBDIRECTORIES
+            # End of: def match_recurse(is_start, sections, path_elements, location):
+        if self.sections:
+            return match_recurse(True, self.sections, path_elements, 0)
+        else:
+            # Catches directory-less patterns like "*.py" and "/*.py".
+            if self.bound_start:
+                if len(path_elements) == 0:
+                    # Eg "*/*.py" in the root directory
+                    return MatchType.MATCH_BUT_NO_SUBDIRECTORIES
+                else:
+                    # Eg "/*.py" meets directory "/test/" - nothing happening
+                    return MatchType.NO_MATCH_NO_SUBDIRECTORIES
+            else:
+                # Eg "**/*.py" - match all directories
+                return MatchType.MATCH_ALL_SUBDIRECTORIES
+    def all_files(self):
+        """Returns True if the :class:`Pattern` matches all files (in a matched
+        directory).
+        The file pattern at the end of the glob was `/` or ``/*``"""
+        return self.file_pattern == "*"
+    def match_files(self, matched, unmatched):
+        """Moves all matching files from the set *unmatched* to the set
+        *matched*.
+        Both *matched* and *unmatched* are sets of string, the strings
+        being unqualified file names"""
+        this_match = set(self.file_filter(unmatched))
+        matched   |= this_match
+        unmatched -= this_match
+    def _to_string(self):
+        """Implemented a function for __str__ and __repr__ to use, but
+        which prevents infinite recursion when migrating to Python 3"""
+        if self.sections:
+            start    = "/" if self.bound_start else "**/"
+            sections = "/**/".join(str(section) for section in self.sections)
+            end      = "" if self.bound_end else "/**"
+        else:
+            start    = ""
+            sections = ""
+            end      = "" if self.bound_end else "**"
+        return "{0}{1}{2}/{3}".format(start, sections, end, str(self.file_pattern))
+    def __repr__(self):
+        return self._to_string()
+    def __str__(self):
+        return self._to_string()
+class PatternSet(object):
+    """A set of :class:`Pattern` instances; :class:`PatternSet` provides
+     a number of operations over the entire set.
+    :class:`PatternSet` contains a number of implementation optimizations and
+    is an integral part of various optimizations in :class:`FileSet`.
+    This class is *not* an implementation of Apache Ant PatternSet"""
+    def __init__(self):
+        self.patterns   = []
+        self._all_files = False
+    def _compute_all_files(self):
+        """Handles lazy evaluation of self.all_files"""
+        self._all_files = any(pat.all_files() for pat in self.patterns)
+    def all_files(self):
+        """Returns True if there is any :class:`Pattern` in the
+        :class:`PatternSet` that matches all files (see
+        :meth:`Pattern.all_files()`)
+        Note that this method is implemented using lazy evaluation so direct
+        access to the member ``_all_files`` is very likely to result in errors"""
+        if self._all_files is None:
+            self._compute_all_files()
+        return self._all_files
+    def append(self, pattern):
+        """Adds a :class:`Pattern` to the :class:`PatternSet`"""
+        assert isinstance(pattern, Pattern)
+        self.patterns.append(pattern)
+        if self._all_files is not None:
+            self._all_files = self._all_files or pattern.all_files()
+    def extend(self, patterns):
+        """Extend a :class:`PatternSet` with addition *patterns*
+        *patterns* can either be:
+        * Another :class:`PatternSet` or
+        * A list of :class:`Pattern` instances"""
+        assert patterns is not None
+        if isinstance(patterns, PatternSet):
+            patterns = patterns.patterns
+        else:
+            assert all(isinstance(pat, Pattern) for pat in patterns)
+        self.patterns.extend(patterns)
+        self._all_files = None
+    def remove(self, pattern):
+        """Remove a :class:`Pattern` from the :class:`PatternSet`"""
+        assert isinstance(pattern, Pattern)
+        self.patterns.remove(pattern)
+        self._all_files = None
+    def match_files(self, matched, unmatched):
+        """Apply the include and exclude filters to those files in *unmatched*,
+        moving those that are included, but not excluded, into the *matched*
+        set.
+        Both *matched* and *unmatched* are sets of unqualified file names."""
+        for pattern in self.iter():
+            pattern.match_files(matched, unmatched)
+            if not unmatched:
+                # Optimization: If we have matched all files already
+                # simply return at this point - nothing else to do
+                break
+    def empty(self):
+        """Returns True if the :class:`PatternSet` is empty"""
+        return len(self.patterns) == 0
+    def iter(self):
+        """An iteration generator that allows the loop to modify the
+        :class:`PatternSet` during the loop"""
+        if self.patterns:
+            patterns = list(self.patterns)
+            for pattern in patterns:
+                yield pattern
+    def __str__(self):
+        return ("PatternSet (All files? {0}) [{1}] ".
+                    format(self.all_files(),
+                           ", ".join(str(pat) for pat in self.patterns)))
+class FileSetState(object):
+    """FileSetState is an object encapsulating the :class:`FileSet` in a
+    particular directory, caching inheritable Pattern matches.
+    This is an internal implementation class and not meant for reuse
+    or to be accessed directly
+    **Implementation notes:**
+    As the FileSet traverses the directories using :func:`os.walk()`, it
+    builds two graphs of FileSetState instances mirroring the graph of
+    directories - one graph of FileSetState instances is for the **include**
+    globs and the other graph of FileSetState instances for the **exclude**.
+    FileSetState embodies logic to decide whether to prune whole
+    directories from the search, either by detecting the include patterns
+    cannot match any file within, or by detecting that an exclude
+    matches all files in this directory and sub-directories.
+    The constructor has the following arguments:
+    1. *label*: A string used only in the :meth:`__str__` method (for debugging)
+    2. *directory*: The point in the graph that this FileSetState represents.
+       *directory* is relative to the starting node of the graph
+    3. *based_on*: A FileSetState from the previous directory traversed by
+       :func:`os.walk()`. This is used as the start point in the graph of
+       FileSetStates to search for the correct parent of this. This is None
+       to create the root node.
+    4. *unmatched*: Used only when *based_on* is None  - the set of initial
+       :class:`Pattern` instances. This is either the original include
+       or exclude globs.
+    During the construction of the instance, the instance will evaluate the
+    directory patterns in :class:`PatternSet` ``self.unmatched`` and, for
+    each :class:`Pattern`, perform of of the following actions:
+    1. If a pattern matches, it will be moved into one of the 'matched'
+    :class:`PatternSet` instances:
+       a. ``self.matched_inherit``: the directory pattern matches all sub
+          subdirectories as well, eg ``/test/**``
+       b. ``self.matched_and_subdir``: the directory matches this directory
+          and *may* match subdirectories as well, eg ``/test/**/more/**``
+       c. ``self.matched_no_subdir``: the directory matches this directory,
+          **but** cannot match any subdirectory, eg ``/test/*``. This pattern
+          will thus not be evaluated in any subdirectory.
+    2. If the pattern does not match, either:
+       a. It may be valid in subdirectories, so it stays in ``self.unmatched``,
+          eg ``**/nomatch/*``
+       b. It cannot evaluate to true in any subdirectory, eg ``/nomatch/**``.
+          In this case it is removed from all :class:`PatternSet` members
+          in this instance.
+    """
+    def __init__(self, label, directory, based_on=None, unmatched=None):
+        self.label = label
+        if directory:
+            self.path_elements = directory.split(path.sep)
+        else:
+            self.path_elements = []
+        # First find the real parent of this node (the based_on is really the
+        # previous return from os.walk, and may be a peer, cousin or completely
+        # unrelated to the directory in the argument
+        if based_on:
+            self.parent = based_on._find_parent(self.path_elements)
+        else:
+            self.parent = None
+        # If we have found a parent, copy the parent's computations here
+        # as the start. This is a significant optimization by caching
+        # as many directory matches as possible
+        self.matched_inherit    = PatternSet() # Matches this directory and all sub
+        self.matched_and_subdir = PatternSet() # Matches this directory and poss. sub
+        self.matched_no_subdir  = PatternSet() # Matches this directory, discard for sub
+        self.unmatched          = PatternSet() # Does no match this directory. but poss. sub
+        if self.parent:
+            self.unmatched.extend(self.parent.matched_and_subdir)
+            self.unmatched.extend(self.parent.unmatched)
+            # parent_has_patterns is True if _any_ parent up to root has
+            # cached a pattern in matched_inherit
+            self.parent_has_patterns = (self.parent.parent_has_patterns or
+                                        not self.parent.matched_inherit.empty())
+        else:
+            # This branch exercised only when constructing the root
+            self.parent_has_patterns = False
+            if unmatched:
+                self.unmatched.extend(unmatched)
+        # For this branch, check which patterns match, and the type of the
+        # match and thereby move the Patterns to the correct buckets
+        for pattern in self.unmatched.iter():
+            match = pattern.match_directory(self.path_elements)
+            if match & MatchType.BIT_MATCH:
+                self.unmatched.remove(pattern)
+                if match & MatchType.BIT_ALL_SUBDIRECTORIES:
+                    # don't re-evaluate this pattern
+                    self.matched_inherit.append(pattern)
+                elif match & MatchType.BIT_NO_SUBDIRECTORIES:
+                    self.matched_no_subdir.append(pattern)
+                else:
+                    # mark this pattern as a match, re-evaluate for subdirs
+                    self.matched_and_subdir.append(pattern)
+            else:
+                if match & MatchType.BIT_NO_SUBDIRECTORIES:
+                    self.unmatched.remove(pattern)
+    def _find_parent(self, path_elements):
+        """Recurse up the tree of FileSetStates until we find a parent, i.e.
+        one whose path_elements member is the start of the path_element
+        argument"""
+        if not self.path_elements:
+            # Automatically terminate on root
+            return self
+        elif self.path_elements == path_elements[0:len(self.path_elements)]:
+            return self
+        else:
+            return self.parent._find_parent(path_elements)
+    def _matching_pattern_sets(self):
+        """Returns an iterator containing all PatternSets that match this
+        directory.
+        This is build by chaining the this-directory specific PatternSet
+        (self.matched_and_subdir), the local (non-inheriting) PatternSet
+        (self.matched_no_subdir) with all the inherited PatternSets
+        that match this directory and all its parents (self.match_inherit)."""
+        gather = []
+        if self.matched_and_subdir:
+            gather.append(self.matched_and_subdir.iter())
+            gather.append(self.matched_no_subdir.iter())
+        ref = self
+        while ref is not None:
+            if ref.matched_inherit:
+                gather.append(ref.matched_inherit.iter())
+            if ref.parent_has_patterns:
+                ref = ref.parent
+            else:
+                ref = None
+        return chain.from_iterable(gather)
+    def match(self, files):
+        """Given a set of files in this directory, returns all the files that
+        match the :class:`Pattern` instances which match this directory."""
+        if not files:
+            return set()
+        if (self.matched_inherit.all_files() or
+           self.matched_and_subdir.all_files() or
+           self.matched_no_subdir.all_files()):
+            # Optimization: one of the matched patterns matches everything
+            # So simply return it
+            return set(files)
+        unmatched = set(files)
+        matched   = set()
+        for pattern_set in self._matching_pattern_sets():
+            pattern_set.match_files(matched, unmatched)
+            if not unmatched:
+                # Optimization: If we have matched all files already
+                # simply return at this point - nothing else to do
+                break
+        return matched
+    def matches_all_files_all_subdirs(self):
+        """Returns True if there is a pattern that:
+        * Matches this directory, and
+        * Matches all sub-directories, and
+        * Matches all files (eg ends with "*")
+        This acts as a terminator for :class:`FileSetState` instances in the
+        excludes graph."""
+        return any(pat.all_files() for pat in self.matched_inherit.iter())
+    def no_possible_matches_in_subdirs(self):
+        """Returns True if there are no possible matches for any
+        subdirectories of this :class:`FileSetState`.
+        When this :class:FileSetState is used for an 'include', a return of
+        `True` means we can exclude all subdirectories."""
+        return (not self.parent_has_patterns and
+                   self.matched_inherit.empty() and
+                   self.matched_and_subdir.empty() and
+                   self.unmatched.empty())
+    def __str__(self):
+        return ("FileSetState {0} in {1}/:\n"
+               "\tInherit: {2}\n"
+               "\tthis&subdir: {3}\n"
+               "\tthis-only: {4}\n"
+               "\tunmatched: {5}".format(
+                    self.label,
+                    "/".join(self.path_elements),
+                    self.matched_inherit,
+                    self.matched_and_subdir,
+                    self.matched_no_subdir,
+                    self.unmatched
+                ))
+def get_initial_default_excludes():
+    """Returns a the default excludes as a list of Patterns.
+     This will be the initial value of :attr:`FileSet.DEFAULT_EXCLUDES`.
+     It is defined in the `Ant documentation
+     <http://ant.apache.org/manual/dirtasks.html#defaultexcludes>`_.
+     Formic adds ``**/__pycache__/**``, with the resulting list being:
+        * \*\*/pycache/\*\*
+        * \*\*/\*~
+        * \*\*/#\*#
+        * \*\*/.#\*
+        * \*\*/%*%
+        * \*\*/._\*
+        * \*\*/CVS
+        * \*\*/CVS/\*\*
+        * \*\*/.cvsignore
+        * \*\*/SCCS
+        * \*\*/SCCS/\*\*
+        * \*\*/vssver.scc
+        * \*\*/.svn
+        * \*\*/.svn/\*\*
+        * \*\*/.DS_Store
+        * \*\*/.git
+        * \*\*/.git/\*\*
+        * \*\*/.gitattributes
+        * \*\*/.gitignore
+        * \*\*/.gitmodules
+        * \*\*/.hg
+        * \*\*/.hg/\*\*
+        * \*\*/.hgignore
+        * \*\*/.hgsub
+        * \*\*/.hgsubstate
+        * \*\*/.hgtags
+        * \*\*/.bzr
+        * \*\*/.bzr/\*\*
+        * \*\*/.bzrignore
+     """
+    return [ Pattern(exclude) for exclude in
+'''.splitlines() ]
+class FileSet(object):
+    """An implementation of the Ant FileSet class.
+    Arguments to the constructor:
+    1. *include*: An Ant glob or list of Ant globs for matching files to include
+       in the response. Ant globs can be specified either:
+       a. As a string, eg ``"*.py"``, or
+       b. As a :class:`Pattern` object
+    2. *exclude*: Specified in the same was as *include*, but any file that
+       matches an exclude glob will be excluded from the result.
+    3. *directory*: The directory from which to start the search; if None,
+       the current working directory is used
+    4. *default_excludes*: A boolean; if True (or omitted) the
+       :attr:`DEFAULT_EXCLUDES` will be combined with the *exclude*.
+       If False, the only excludes used are those in the excludes argument
+    5. *symlinks*: Sets whether symbolic links are included in the results or not.
+    **Usage**
+    First, construct a ``FileSet``::
+            from formic import FileSet
+            fileset = FileSet(directory="/some/where/interesting",
+                              include="*.py",
+                              exclude=["**/*test*/**", "test*"]
+                              )
+    There are three APIs for retrieving matches:
+    1. FileSet is itself an iterator and returns absolute file names::
+            for filename in fileset:
+                print filename
+    2. For more control, use ``fileset.qualified_files()``. The following
+       prints filenames *relative* to the directory::
+            for filename in fileset.qualified_files(absolute=False):
+                print filename
+    3. For absolute control, use the ``fileset.files()`` method and handle the
+       returned tuple yourself::
+            prefix = fileset.get_directory()
+            for directory, file_name in fileset.files():
+                sys.stdout.write(prefix)
+                if dir:
+                    sys.stdout.write(path.sep)
+                    sys.stdout.write(directory)
+                sys.stdout.write(path.sep)
+                sys.stdout.write(file_name)
+                sys.stdout.write("\\n")
+    Implementation notes:
+    * :class:`FileSet` is lazy: The files in the :class:`FileSet` are resolved
+      at the time the iterator is looped over. This means that it is very fast
+      to set up and (can be) computationally expensive only when results are
+      obtained.
+    * You can iterate over the same :class:`FileSet` instance as many times
+      as you want. Because the results are computed as you iterate over the
+      object, each separate iteration can return different results, eg if the
+      file system has changed.
+    * *include* and *exclude* arguments to the constructor can be given in
+      several ways:
+      * A string: This will be automatically turned into a :class:`Pattern`
+      * A :class:`Pattern`: If you prefer to construct the pattern yourself
+      * A list of strings and/or :class:`Pattern` instances (as above)
+    * In addition to Apache Ant's default excludes, :class:`FileSet` excludes:
+      * ``__pycache__``
+    * You can modify the :attr:`DEFAULT_EXCLUDES` class member (it is a list of
+      :class:`Pattern` instances). Doing so will modify the behaviour of all
+      instances of :class:`FileSet` using default excludes.
+    """
+    #: Default excludes shared by all instances. The member is a list of
+    #: :class:`Pattern` instances. You may modify this member at run time to
+    #: modify the behaviour of all instances.
+    DEFAULT_EXCLUDES = get_initial_default_excludes()
+    def __init__(self,
+                 include,
+                 exclude=None,
+                 directory=None,
+                 default_excludes=True,
+                 symlinks=True):
+        self.include  = FileSet._preprocess(include)
+        if not self.include:
+            raise FormicError("No include globs have been specified"
+                              "- nothing to find")
+        self.exclude  = FileSet._preprocess(exclude)
+        self.symlinks = symlinks
+        if default_excludes:
+            self.exclude.extend(FileSet.DEFAULT_EXCLUDES)
+        if directory is None:
+            self.directory = None
+        else:
+            self.directory = path.abspath(directory)
+        self._received = 0 # Used for testing
+    @staticmethod
+    def _preprocess(argument):
+        """Receives the argument (from the constructor), and normalizes it
+        into a list of Pattern objects."""
+        pattern_set = PatternSet()
+        if argument is not None:
+            if not hasattr(argument, "__iter__"):
+                argument = [ argument ]
+            argument = [ Pattern(pattern) if isinstance(pattern, basestring) else pattern for pattern in argument ]
+            pattern_set.extend(argument)
+        return pattern_set
+    def get_directory(self):
+        """Returns the directory in which the :class:`FileSet` will be run.
+        If the directory was set with None in the constructor, get_directory()
+        will return the current working directory.
+        The returned result is normalized so it never contains a trailing
+        path separator"""
+        directory = self.directory if self.directory else getcwd()