1. Eric Snow
  2. pylt

Overview

PyLT

pylt ("Python Language Transformations") is a system for source-to-source transformations...and more.

Once one or more transforms is registered, you can simply import any module for which the source should be transformed. An import hook

You can also use transform_raw() and transform_by_inspection() to transform source directly.

See https://bitbucket.org/ericsnowcurrently/pylt.

Transforms

A transform is a function that takes the source and the triggering tag/suffix, as well as any other keyword arguments:

some_transform(source, [tag, [suffix]], **kwargs)

Each transform will return the transformed source.

A variety of sample transforms are located in pylt.transforms.

Registration

Use register() to register a transform. It may be used as a decorator. Once a transform is registered, it will be used during import to transform the module's source if the tag and file suffix match.

Tags and Suffixes

Importing

Transform Notes

Targetable artifacts

  • source
  • token stream (from tokenization)
  • AST (from parsing)
  • bytecode (from compiling)

Transform strategies

  • do string replace on source before tokenization
  • do regex replace on source before tokenization
  • do token replace on token stream before parsing
  • do AST replace on AST before compilation
  • do bytecode replacement

possible future stuff

  • function to activate/deactivate pylt import hook
  • other plugin systems
  • register_for_pyc(transform)
  • import_engine
  • transforms (catalog of "built in" transforms)
  • transforms['inline_pylt'] (pragma or fake context manager, a la pwang)
  • Policy (for registration collisions, etc.)
  • Transform (base class) or helper functions
  • warning when pylt actually changes the source
  • transform type based on transform strategy (handle each differently)
  • Existing AST hacks and peephole optimizer in CPython...

examples

  • Python optimizer

  • 2 to 3

  • 3 to 2

  • DSL (domain-specific languages)

  • __moduleclass__

  • __build_function__

  • other languages to Python:
    • javascript
    • perl
    • ruby
    • C
  • new syntax:
    • def-from
    • given statement (PEP 3150)
    • function locals constants
    • attr wrapper - obj.(name)
    • implicit namespace binding - "def x.y(...):"
    • conditional except clause
    • inline
  • others:
    • python to C-ext module
    • python optimizer

"policy"

registration collisions/duplicate keys order constraints

Hook API

activate() deactivate() is_activated() activated() (context manager) deactivated() (context manager)

Registry API

register() unregister() is_registered() registered() (context manager) unregistered() (context manager)

Other stuff

possible approaches for "inlined" transforms

applies to the block (with the "with" clause removed and proper indent)
with pylt.transformed(tag, sufficx, ...):
...
applies to the block (with proper indentation)
# pylt: tag, suffix, ...
...
applies just to the statement:
# pylt: tag, suffix, ... <statement>

To-Do

  1. clean up docs/wiki/source/etc. and make them consistent
  2. create the sample transforms (with tests)
  3. initial release
  4. make compatible with Python 3.2
  5. backport to 2.7?