Issue #371 resolved

ANDMAYBE @Year:[2000 TO] crashes Whoosh

Helmut Jarausch avatarHelmut Jarausch created an issue

This is Whoosh HG 2013/10/27 version. My Schema contains a 'Year' field Using the standard query parser on Dahmen ANDMAYBE year :[2000 TO] gives

  File "./Bib_Whoosh_Test.py", line 87, in <module>
    Query= QP.parse('Dahmen ANDMAYBE @Year:[2000 TO]',debug=True)
  File "/usr/lib64/python3.3/site-packages/whoosh/qparser/default.py", line 373, in parse
    q = q.normalize()
  File "/usr/lib64/python3.3/site-packages/whoosh/query/compound.py", line 126, in normalize
    s = s.normalize()
  File "/usr/lib64/python3.3/site-packages/whoosh/query/compound.py", line 637, in normalize
    b = self.b.normalize()
AttributeError: 'NoneType' object has no attribute 'normalize'

The Query debug output is

Taggers: [<OpTagger '(?<=\\s)ANDNOT(?=\\s)' (anot)>, <OpTagger '(?<=\\s)ANDMAYBE(?=\\s)' (amaybe)>, <whoosh.qparser.plugins.EveryPlugin object at 0x7f8225049090>, <whoosh.qparser.plugins.SingleQuotePlugin object at 0x7f82254cb3d0>, <whoosh.qparser.plugins.FieldsPlugin.FieldnameTagger object at 0x7f82254c4950>, <whoosh.qparser.plugins.WildcardPlugin object at 0x7f82254cb410>, <whoosh.qparser.plugins.PhrasePlugin.PhraseTagger object at 0x7f82254cb850>, <FnTagger <_sre.SRE_Pattern object at 0x7f82254bd110> (openB)>, <FnTagger <_sre.SRE_Pattern object at 0x7f82254bd1f0> (closeB)>, <OpTagger '(^|(?<=(\\s|[()])))NOT(?=\\s)' (not)>, <OpTagger '(?<=\\s)AND(?=\\s)' (and)>, <OpTagger '(?<=\\s)OR(?=\\s)' (or)>, <OpTagger '(^|(?<=\\s))REQUIRE(?=\\s)' (req)>, <whoosh.qparser.plugins.BoostPlugin object at 0x7f8225049210>, <FnTagger <_sre.SRE_Pattern object at 0x7f8225c12c90> (quote)>, <whoosh.qparser.plugins.FuzzyTermPlugin object at 0x7f8225049310>, <whoosh.qparser.plugins.RangePlugin.RangeTagger object at 0x7f8225049150>, <whoosh.qparser.plugins.WhitespacePlugin object at 0x7f8225049050>, <whoosh.qparser.plugins.WhitespacePlugin object at 0x7f82254cb710>]
Tween: <None:'Dahmen'>
Tagger: <whoosh.qparser.plugins.WhitespacePlugin object at 0x7f8225049050> at 6: < >
Tagger: <OpTagger '(?<=\\s)ANDMAYBE(?=\\s)' (amaybe)> at 7: <OP 'ANDMAYBE'>
Tagger: <whoosh.qparser.plugins.WhitespacePlugin object at 0x7f8225049050> at 15: < >
Tween: <None:'@'>
Tagger: <whoosh.qparser.plugins.FieldsPlugin.FieldnameTagger object at 0x7f82254c4950> at 17: <'Year':>
Tagger: <whoosh.qparser.plugins.RangePlugin.RangeTagger object at 0x7f8225049150> at 22: <None:['2000' None]>
Tagged group: <AndGroup <None:'Dahmen'>, < >, <OP 'ANDMAYBE'>, < >, <None:'@'>, <'Year':>, <None:['2000' None]>>
Pre-filtered group: <AndGroup <None:'Dahmen'>, < >, <OP 'ANDMAYBE'>, < >, <None:'@'>, <'Year':>, <None:['2000' None]>>
..Applying: <bound method GroupPlugin.do_groups of <whoosh.qparser.plugins.GroupPlugin object at 0x7f82254cbf90>>
..Result: <AndGroup <None:'Dahmen'>, < >, <OP 'ANDMAYBE'>, < >, <None:'@'>, <'Year':>, <None:['2000' None]>>
..Applying: <bound method BoostPlugin.clean_boost of <whoosh.qparser.plugins.BoostPlugin object at 0x7f8225049210>>
..Result: <AndGroup <None:'Dahmen'>, < >, <OP 'ANDMAYBE'>, < >, <None:'@'>, <'Year':>, <None:['2000' None]>>
..Applying: <bound method FuzzyTermPlugin.do_fuzzyterms of <whoosh.qparser.plugins.FuzzyTermPlugin object at 0x7f8225049310>>
..Result: <AndGroup <None:'Dahmen'>, < >, <OP 'ANDMAYBE'>, < >, <None:'@'>, <'Year':>, <None:['2000' None]>>
..Applying: <bound method WildcardPlugin.do_wildcards of <whoosh.qparser.plugins.WildcardPlugin object at 0x7f82254cb410>>
..Result: <AndGroup <None:'Dahmen'>, < >, <OP 'ANDMAYBE'>, < >, <None:'@'>, <'Year':>, <None:['2000' None]>>
..Applying: <bound method FieldsPlugin.do_fieldnames of <whoosh.qparser.plugins.FieldsPlugin object at 0x7f82254cb750>>
..Result: <AndGroup <None:'Dahmen'>, < >, <OP 'ANDMAYBE'>, < >, <None:'@'>, <'Year':['2000' None]>>
..Applying: <bound method WhitespacePlugin.remove_whitespace of <whoosh.qparser.plugins.WhitespacePlugin object at 0x7f8225049050>>
..Result: <AndGroup <None:'Dahmen'>, <OP 'ANDMAYBE'>, <None:'@'>, <'Year':['2000' None]>>
..Applying: <bound method WhitespacePlugin.remove_whitespace of <whoosh.qparser.plugins.WhitespacePlugin object at 0x7f82254cb710>>
..Result: <AndGroup <None:'Dahmen'>, <OP 'ANDMAYBE'>, <None:'@'>, <'Year':['2000' None]>>
..Applying: <bound method BoostPlugin.do_boost of <whoosh.qparser.plugins.BoostPlugin object at 0x7f8225049210>>
..Result: <AndGroup <None:'Dahmen'>, <OP 'ANDMAYBE'>, <None:'@'>, <'Year':['2000' None]>>
..Applying: <bound method SequencePlugin.do_quotes of <whoosh.qparser.plugins.SequencePlugin object at 0x7f82250492d0>>
..Result: <AndGroup <None:'Dahmen'>, <OP 'ANDMAYBE'>, <None:'@'>, <'Year':['2000' None]>>
..Applying: <bound method OperatorsPlugin.do_operators of <whoosh.qparser.plugins.OperatorsPlugin object at 0x7f82254cbfd0>>
..Result: <AndGroup <AndMaybeGroup <None:'Dahmen'>, <None:'@'>>, <'Year':['2000' None]>>
Syntax tree: <AndGroup <AndMaybeGroup <None:'Dahmen'>, <None:'@'>>, <'Year':['2000' None]>>
Pre-normalized query: And([AndMaybe((Term('TOC', 'dahmen'), None)), NumericRange('Year', 2000, None, False, False, boost=1.0, constantscore=True)])

Comments (3)

  1. Matt Chaput

    The error is because the @ symbol gets parsed into a None (because the default field can't parse it), and the BinaryGroup syntax tree node couldn't handle one of its sub-queries being None.

    For now I've fixed BinaryGroup so that, when one of its sub-queries is None, it just returns the other. This results in the query Dahmen AND Year:[2000 TO].

    It would be nicer to re-architect the parser so it could ignore things like the @, and give the more correct Dahmen ANDMAYBE Year:[2000 TO], but that's a much bigger job.

  2. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.