Pre-compiling patterns

Create issue
Issue #277 closed
Former user created an issue


I have relatively many patterns that i try to search in a text. I discovered that compiling the patterns takes quite some time. Therefore I have pre-compiled them when starting the program. However this takes quite some memory. I've compared it with the memory used by re module and it is at least triple. Does anybody have some experience with this? Any better suggestions?

Thanks, Flori

Comments (3)

  1. Matthew Barnett repo owner

    The module provides more functionality than the re module, and extra checks to reduce the chance of catastrophic backtracking, so it's not surprising that it also uses more memory. As you haven't said how many patterns you're using (how many is "relatively many"?), what they look like, or how you're using them (for example, are there any that are being tried one after the other, that could be combined?), I can't really make any suggestions. It's also not clear whether the extra memory they're using is actually causing you a problem.

  2. Matthew Barnett repo owner

    It's not clear how the speed of compilation or memory usage could be reduced without major work.

  3. Log in to comment