1. brendanarnold
  2. py-fortranformat
Issue #18 new

Reading performance

Jason Swails
created an issue

This looks like a great project that I would very readily use in my own codes (I work with a lot of Fortran codes and Fortran outputs).

The one thing that stops me from using the FortranRecordReader/Writer classes here in most of my own projects is that they're simply too slow for the size files that I'm trying to parse. As an example, in one of my projects I have a very simplistic parser that supports a tiny subset of permissible formats (but currently covers all of my use cases for that project). [https://github.com/swails/ParmEd/blob/master/chemistry/amber/amberformat.py#L22-L168 if you're interested]

Replacing my primitive code with the classes here results in a 5x-6x slowdown (file reading goes from 4 seconds to 17 seconds on a medium-sized example file). I understand the challenges in making codes both flexible and fast. I filed this issue in case you wanted to hear feedback like this. Feel free to close this issue as wontfix or invalid if either time constraints or project scope preclude working on performance enhancements.

Comments (1)

  1. brendanarnold repo owner

    Not at all, it's good to hear feedback.

    So correctness was prioritised over speed in this project. Much of the internals are almost direct conversions of C code from the GFortran project into Python. This means that a lot of the code is not idiomatic Python and therefore almost certainly could be sped up by re-writing into code that Python interpreters are more likely to be able to optimise.

    Secondly, the code is pure Python and so would likely be easily runnable on something like PyPy which could provide some speed-up there.

    Finally there is code which provides compatibility from 2.3 through to 3.X which is processed at runtime. Removing the 2.3 compatibility might speed some things up a little.

    It's unlikely I'll get much time to make these changes since I no longer work with Fortran much but I've included them in-case anyone would like to improve the speeds.

  2. Log in to comment