Python 3 compatibility from Tapple Gao

#11 Merged at 1031ad9
Repository
Deleted repository
Branch
default (1031ad9a8372)
Repository
llbase
Branch
default
Author
  1. Nat Linden
Reviewers
Description
  • futurize --stage1

  • switch from distutils to setuptools, so that we can use setup.py nosetest, rather than setup.py test. nosetest allows the cllsd tests to pass on mac, not just linux

  • futurize missed one relative import; fixed

  • added a setup.cfg that matches the setup.py nosetests and setup.py tests default configuration

  • add some setup.py and futurize temp files to .hgignore

  • made llsd.py parse bytes rather than strings

  • LLSDNotationUnitTest now passes on python 3.5

  • LLSDXMLUnitTest now passes on python 3.5

  • LLSDBinaryUnitTest now passes on python 3.5

  • add dependency on future for python 2/3 compatability

  • LLSDPythonXMLUnitTest now passes on python 3.5

  • all tests in llsd_test.py now pass on python 3.5

  • all tests in llsd_fuzz_test.py now pass on python 3.5

  • futurized

  • all tests in llidl_test.py now pass on python 3.5

  • llidl_speed_test now runs on python3.5

  • moved the typechecking utility functions from llsd_fuzz_test to llsd, since they are most useful there, and also accessible to all other modules

  • all tests in config_test.py now pass on python 3.5

  • centralize the platform specifics to the llsd module

  • futurize

  • all tests now pass on python 2.7 again

  • removed dependency on future; I wasn't using it for anything except the PY2 constant

  • mark as python 3 compatable

  • set up tox to run the unit tests under all supported python versions

  • full version compatability declaration

  • SL-764: Tweaks to Tapple Gao's pull request for Python 3 compat.

    (Many of these changes are to pre-existing llbase code; the Python 3 compatibility code review simply rubbed my nose in some issues.)

    With these changes, unit tests run cleanly on Mac on both Python 2 and 3.

    Rename is_str_or_unicode() to is_string(), is_int_or_long() to is_integer().

    Resolve type problems at module load time by introducing StringTypes, LongType, IntTypes, UnicodeType, BytesType. This allows simple isinstance() calls in is_integer(), is_unicode(), is_string(), is_bytes(). Since dict literals permit duplicate keys, it also allows us to avoid conditional dict construction by stating both int and LongType, str and UnicodeType.

    Also streamline _str_to_bytes() by observing that the only version-sensitive behavior is already encapsulated in is_unicode().

    Extract LLSDBaseFormatter base class for LLSDXMLFormatter and LLSDNotationFormatter, whose constructors used to be identical, creating a dispatch dict from types to methods. The method definitions are subclass specific, but the method names are the same in both cases.

    Extract LLSDBaseParser base class for LLSDBinaryParser and LLSDNotationParser to provide common _error(), _peek() and _getc() methods. Defend _peek() against garbage binary length values. Hoist _parse_string_delim() up into base class: the two subclass implementations were almost identical. Change LLSDNotationParser's _parse_string_delim() calls accordingly. Use a dict lookup instead of a chain of if/elif for escape-char processing.

    LLSDBinaryParser._parse() and LLSDNotationParser._parse() each used a long chain of if/elif to decide how to process the current character. Instead, make each parser subclass construct a dispatch list indexed by ord(char), which seems likely to improve performance.

    Introduce LLSDNotationParser._skip_then(): since the new dispatch mechanism requires a function or lambda, we need a helper method to eat the current character before returning some value.

    Change signature and semantics of LLSDNotationParser._get_re(), which used to return None on match failure. But since every existing call raises an error in that case, make _get_re() itself raise the error, streamlining every call site. Add a 'desc' parameter to clarify the error message. Also add an optional 'override' parameter: when successfully matching 'true' or 'false', we want _get_re() to return the corresponding bool value instead of the matched string.

    Defend LLSDBinaryParser._parse_date() against garbage date data. In Python 3, attempting to convert from binary seconds to a datetime value can raise OverflowError.

    Eliminate redundant base checking in LLSDNotationParser._parse_binary().

    Since iterating over a dict implicitly produces its keys, avoid calling dict.keys(), especially list(dict.keys()), which (in Python 2) instantiates and copies the list of keys. Add comments clarifying that's what we're doing.

    Explain the use of latin-1 encoding for UUID values.

    In tests, skip importing _cllsd module in Python 3 as well as on Windows.

Comments (3)