1. Eric Larson
  2. mgoquery


Eric Larson  committed 2fa0162

Updated readme with notes on usage and parsing rules.

  • Participants
  • Parent commits cebe19d
  • Branches default

Comments (0)

Files changed (1)


View file
 MgoQuery (MongoDB Query Parser)
-A search-like interface that constructs a valid MongoDB query.
+A simple query language that returns a valid MongoDB query.
-MongoDB provides a flexible query model that is powerful, yet somewhat
-intimidating to write for non-technical users. Mqueryparser aims to
-create a concise search-like parser.
+MongoDB provides a flexible query model that is powerful, yet verbose
+at times. MgoQuery provides a simple query langauge to create a
+concise, search-like syntax for constructing MongoDB queries.
-This type of parser has two main goals:
  1. Provide a safe and limited interface for querying MongoDB.
  2. Provide a query language that is URL friendly
-MQueryParser allows an API developer to create a thin layer above a
-raw query that can be validated and tested for injection like attacks
-while providing a simple interface via the URL.
+Query Syntax
-Query Grammar
-The query grammar is inspired by tools such as Xapian, Lucene and
+The MgoQuery syntax is inspired by tools such as Xapian, Lucene and
 GMail's advance query search.
 Here is an example of the basic format: ::
                      {'z': True}]}]}
+Spaces are optional which means the above query could be rewritten as: ::
+  "x>3,x<5"|"y>10,z:True"
+Or: ::
+  "x>3,x<5"|"y>10,z:True"
+An expression defines a single requirement for a single key in a
+MongoDB query. For example ::
+  {'x': 1}
+An expression in a MgoQuery is as follows: ::
+  $key <-> operator <-> $value
 The operators are as follows:
   equals = ":"
   greater than or equal to >= ">"
   less than or equal to <= "<"
-The "," acts as an AND operator meaning the following
-field/operator/value will used in conjunction with the
+It should be noted that we only use greater/less than or equal to as
+the theory is it will be easier for users to understand the value they
+use will be included in the results.
-The "|" acts as an OR operator such that any match of the
-field/operator/value combinations should match.
+Here are some examples: ::
+  x:3     => {'x': '3'}
+  foo > 4 => {'foo': {'$gte': '4'}}
+  y < x   => {'y': {'$lte': 'x'}}
+One thing to note in the above examples is that the values are all
+strings. I will explain how to help the parser know when you want to
+use different types in the parsed output. 
+Expressions can be combined in order to create more complex
+expressions. There are two ways to combine expressions, grouping and
+combination operators.
+Combination Operators
+Similar to the operators in expressions, combination operators act
+upon two expressions. ::
+  expression <-> operator <-> expression
+Here are two examples using the two combination operators: ::
+  x:1 , y:2 => {'$and': [{'x': '1'}, {'y': '2'}]}
+  x:1 | y:2 => {'$or': [{'x': '1'}, {'y': '2'}]}
+The "," acts as an AND operator meaning both expressions would need to
+match in the document for it to be returned. 
+The "|" acts as an OR operator such that either expression can match
+in order for the document to be returned. 
+I should be noted that you may only use combination operator at a
+time. There is no precendence that takes place in order to clarify
+how the expression should be applied. For example: ::
+  x:1, y:2 | foo:bar
+There is no way for the parser to know whether you intended the 'y:2'
+expression to be compared first to the 'x:1' with AND or with the
+'foo:bar' using OR.
+It is possible to use different combination operators by using groups.
+Groups allow you to use more than one combination operator in a
+query. Here is the format for a group: ::
+  "expression [<-> operator]"
+Quotes are used to wrap the expressions and the combination operator
+such that you can use more than one. Here is a more complex example to
+see how this works: ::
+  "x>1, x<5" | "y>2, x:None"
+In this example the groups are surrounded in the quotes and use the
+AND operator. Both groups are then used in an OR operation. In english
+the example would read as: 
+  Select all documents if:
+    The key 'x' is greater than or equal to 1 AND
+    The key 'x' is less than or equal to 5
+  OR
+    The key 'y' is greater than or equal to 2 AND
+    The key 'x' does not exist or is None.
+Currently groups cannot be nested as we do not have the use case for
+this complex of queries.
+Using the Parser
+Here is a small session showing how to use the parser in order to
+construct queries: ::
+  Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
+  [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
+  Type "help", "copyright", "credits" or "license" for more information.
+  >>> from pprint import pprint
+  >>> from mgoquery import Parser, Query
+  >>> p = Parser()
+  >>> result = p.parse('x > 5, y < 3')
+  >>> print result
+  [<mgoquery.AndOr object at 0x10d75f790>]
+  >>> query = Query(result)
+  >>> pprint(query.as_dict())
+  {'$and': [{'x': {'$gte': '5'}}, {'y': {'$lte': '3'}}]}
+Converting Values in Queries
+As you can see from the examples, the parser default does not make an
+effort to understand the type of value for each expression. In order
+to convert the value to the correct type you can pass a conversion
+function to the Parser constructor. 
+Here is a simple session using the same example from above: ::
+  >>> p = Parser(conversion=lambda key, value: int(value))
+  >>> pprint(Query(p.parse('x:1, y:2')).as_dict())
+  {'$and': [{'x': 1}, {'y': 2}]}
+The conversion function should take two arguments, a "key" and
+"value". The key is the name of the key used by the documents you want
+to query. As MongoDB doesn't support forcing a type on a specific key
+in a collection of documents, we use the name of the key to provide a
+suggestion as to what type to use. 
+Here is an example using a potential date parsing function: ::
+  from mylibs import parse_date
+  from mgoquery import Parser, Query
+  def value_conversion(key, value):
+      if 'date' in key or 'time' in key:
+          return parse_date(value)
+      return value
+  p = Parser(conversion=value_conversion)
+  print(Query(p.parse('startdate:2012-02-03')).as_dict())
+  # prints -> {'starttdate': datetime(2012, 2, 3)}
+The return value of the conversion function should be the converted
+value. It is also appropriate to validate the input and throw an error
+if it is invalid.