1. Ludia
  2. Untitled project
  3. dynamodb-mapper


Jean-Tiare Le Bigot  committed 33914bf

addressed all comments by mnoel

  • Participants
  • Parent commits 42b7644
  • Branches default

Comments (0)

Files changed (7)

File docs/api/alter.rst

View file
  • Ignore whitespace
 Amazon's DynamoDB offers the ability to both update and insert data with a single
 :py:meth:`~.DynamoDBModel.save` method that is mostly exposed by Dynamodb-mapper.
+.. _saving:
 When saving an Item with an :py:class:`~.autoincrement_int` ``hash_key``, the
 :py:meth:`~.DynamoDBModel.save` method will automatically add checks to prevent
-accidental overwrite of the "magic item". The magic item holds the last allocated
+accidental overwrite of the u"magic item". The magic item holds the last allocated
 ID and is saved at ``hash_key=-1``. If ``hash_key == 0`` then a new ID is
 automatically and atomically allocated meaning that no collision can occure even
 if the database connection is lost. Additionaly, a check is performed to make

File docs/api/model.rst

View file
  • Ignore whitespace
+.. _data-models:
 Data models
     conn = ConnectionBorg()
     conn.create_table(MyModel, read_units=10, write_units=10, wait_for_active=True)
+Important note: Unlike most databases, table creation may take up to 1 minute.
+during this time, the table is *not* usable. Also, you can not have more than 10
+tables in ``CREATING`` or ``DELETING`` state any given time for your whole Amazon
+account. This is an Amazon's DynamoDB limitation.
 The connection manager automatically reads your credentials from either:
 - ``/etc/boto.cfg``
 specified ``hash_key``.
 Before using this feature, make sure you *really need it*. In most cases another
-field can be used in place. A good hint is "witch field would I have marked
+field can be used in place. A good hint is u"witch field would I have marked
 UNIQUE in SQL ?".
 - for users, ``email`` or ``login`` field shoud do it.
     # Create a new ticket and auto-generate an ID
     ticket = Ticket()
-    ticket.title = "Chuck Norris is the reason why Waldo hides"
+    ticket.title = u"Chuck Norris is the reason why Waldo hides"
     ticket.tags = set([u'priority:critical', u'version:yesterday'])
     ticket.description = u"Ludia needs to create a new social game to help people all around the world find him again. Where is Waldo?"
     # Create a new ticket and force the ID
     ticket = Ticket()
-    ticket.ticket_number = "42"
-    ticket.payload = "foo/bar"
+    ticket.ticket_number = u"42"
+    ticket.payload = u"foo/bar"
     ticket.save() # create or replace item #42
     print ticket.ticket_number # id has not changed
-.. TODO: lien vers save pour eviter d'ecraser
-.. Autre idee a la con: lever la limitation du sur range key, surtout pour stocker des revisions
+To prevent accidental data overwrite when saving to an arbitrary location, please
+see the detailed presentation of :ref:`saving`.
+.. Suggestion: remove the range_key limitation  when using `autoincrement_int`. might be useful to store revisions for ex
 Please note that ``hash_key=-1`` is currently reserved and nothing can be stored
 at this index.
         __hash_key__ = u"login"
         __schema__ = {
             u"login": unicode,
-            u"orders": set,
+            u"order_ids": set,
+    # Get all orders for customer u"John Doe"
+    customer = Customer(u"John Doe")
+    order_generator = Order.get_batch(customer.order_ids)
 But this approach has many drawbacks.
 - It is expensive:
     - An update to generate a new autoinc ID
     - An insertion for the new order item
     - An update to add the new order id to the customer
+- It is risky:
+    - Items are limited to 64KB but the ``order_ids`` set has no growth limit
 - To get all orders from a giver customer, you need to read the customer first
     and use a :py:meth:`~.DynamoDBModel.get_batch` request
         __hash_key__ = u"login"
         __schema__ = {
             u"login": unicode,
-            u"orders": set,
+            #u"orders": set, => This field is not needed anymore
-    # Get all orders for customer "John Doe"
+    # Get all orders for customer u"John Doe"
     Order.query(u"John Doe")
-    # Get all orders for customer "John Doe" after agiven date
-    Order.query(u"John Doe")
-.. TODO: lien vers query high level
-.. TODO link to query high level doc
+Not only is this approach u"better", it is also much more powerful. We could
+easily limit the result count, sort them in reverse order or filter them by
+creation date if needed. For more background on the querying system, please see
+the :ref:`accessing data <accessing-data>` section of this manual.
 Default values
-When instanciating a model, all fields are initialised to "neutral" values. For
+When instanciating a model, all fields are initialised to u"neutral" values. For
 containers (``dict``, ``set``, ``list``, ...) it is the empty container, for
 ``unicode``, it's the empty string, for numbers, 0...
-    from dynamodb_mapper.model import DynamoDBModel
+    from dynamodb_mapper.model import DynamoDBModel, utc_tz
     from datetime import datetime
     # define a model with defaults
     class PlayerStrength(DynamoDBModel):
-        __table__ = "player_strength"
-        __hash_key__ = "player_id"
+        __table__ = u"player_strength"
+        __hash_key__ = u"player_id"
         __schema__ = {
-            "player_id": int,
-            "strength": unicode,
-            "lastUpdate": datetime,
+            u"player_id": int,
+            u"strength": unicode,
+            u"last_update": datetime,
         __defaults__ = {
-            "strength": u'weak', # scalar default value
-            "lastUpdate": lambda: datetime.now(), # callable default value
+            u"strength": u'weak', # scalar default value
+            u"last_update": lambda: datetime.now(utc_tz), # callable default value
 >>> player = PlayerStrength(strength=u"chuck norris") # overload one of the defaults

File docs/api/query.rst

View file
  • Ignore whitespace
+.. _accessing-data:
 Accessing data
 slower too. So that keeping this aspect in mind is important.
 'Eventual consistency' is the default behavior in all requests. It also the only
-available option for ``query``.
+available option for ``scan`` and ``get_batch``.
 .. todo: get with update
     # Example model
     class MyUserModel(DynamoDBModel):
-        __table__ = "..."
-        __hash_key__ = "fullname"
+        __table__ = u"..."
+        __hash_key__ = u"fullname"
         __schema__ = {
             # This is probably a good key in a real world application because of homonynes
-            "fullname": unicode,
+            u"fullname": unicode,
             # [...]
     myuser = MyUserModel.get("Chuck Norris")
     # Do some work
-    print "myuser({})".format(data.h_key)
+    print "myuser({})".format(myuser.fullname)
 Use case: Get only objects after ``2012-12-21 13:37``
     # Example model
     class MyDataModel(DynamoDBModel):
-        __table__ = "..."
-        __hash_key__ = "h_key"
-        __range_key__ = "r_key"
+        __table__ = u"..."
+        __hash_key__ = u"h_key"
+        __range_key__ = u"r_key"
         __schema__ = {
-            "h_key": int,
-            "r_key": datetime,
+            u"h_key": int,
+            u"r_key": datetime,
             # [...]
     # Example model
     class MyBlogPosts(DynamoDBModel):
-        __table__ = "..."
-        __hash_key__ = "post_id"
-        __range_key__ = "revision"
+        __table__ = u"..."
+        __hash_key__ = u"post_id"
+        __range_key__ = u"revision"
         __schema__ = {
-            "post_id": int,
-            "revision": int,
-            "title": unicode,
-            "tags": set,
-            "content": unicode,
+            u"post_id": int,
+            u"revision": int,
+            u"title": unicode,
+            u"tags": set,
+            u"content": unicode,
             # [...]
     # Get the actual blog post to render
         mypost = mypost_last_revision_generator.next()
-    except:
+    except StopIteration:
         mypost = None # Not Found
 This example could easily be adapted to get the first revision, the ``n`` first

File docs/api/transaction.rst

View file
  • Ignore whitespace
 The :ref:`save use case <save-use-case>` demonstrates the use of
 ``expected_values`` argument. What it does is actually implement by hand a
-transaction. Amazon's DynamoDB has no "out of the box" transaction engines but
+transaction. Amazon's DynamoDB has no u"out of the box" transaction engines but
 provides this parameter as an elementary block for this purpose.
 Transaction concepts
 the lost update syndrome.
 Optionally, transactions may define a :py:meth:`~.Transactions._setup` method
-which will be called before any transactors.
+witch will be called before any transactors.
 Unless the transaction is explicitely marked ``transient``, its state will be
 persisted to a dedicated table. ``Transaction`` base class embeds a minimal
     bundle = {
-        "cost": 150,
-        "items": [
+        u"cost": 150,
+        u"items": [
     class BundleTransaction(Transaction):
         transient = False # Make it explicit. This is anyway the default.
-        __table__ = "mygame-dev-bundletransactions"
+        __table__ = u"mygame-dev-bundletransactions"
         def __init__(self, user_id, bundle):
+            super(BundleTransaction, self).__init__()
             self.requester_id = user_id
             self.bundle = bundle

File docs/pages/getting_started.rst

View file
  • Ignore whitespace
 Our maps also have an ``name`` and a set of ``cheats`` codes. In DynamoDB, all strings
 are stored as ``unicode`` hence the type. Lastly, we want each maps to recognize
-by ``__defaults__`` the famous "Konami" cheat code.
+by ``__defaults__`` the famous u"Konami" cheat code.
 DoomMap Model
 Note that they are defined on the class level. Any accidental override in the
 instances will be ignored.
-- ``__table__`` Table name in dynamoDB
+- ``__table__`` Table name in DynamoDB
 - ``__hash_key__`` Name of the the hash key field
 - ``__range_key__`` Name of the (optional) range key field
 - ``__schema__`` Dict mapping of ``{"field_name": type}``. Must at least contain
     the keys
 - ``__defaults__`` Define an optional default value for each field used by ``__init__``
-.. TODO: link to more doc.
-.. TODO: + details sur defaults
-.. TODO: limitation de schema et pb d'export de date
+For more informations on the models and defaults, please see the :ref:`data models
+<data-models>` section of this manual.
+.. TODO: schema limitation and dates export issues
 Initial Table creation
 First, create and :py:meth:`~.DynamoDBModel.save` new map in episode 1 and call
-it "Hangar". Let's also register a couple a cheats.
+it u"Hangar". Let's also register a couple a cheats.
 It is now possible to :py:meth:`~.DynamoDBModel.get` it from the database using
 a conpound index that is to say, both a ``hash_key`` and a ``range_key``. By
-default, ``get`` uses "eventual consistence" for data access but it is possible
+default, ``get`` uses u"eventual consistence" for data access but it is possible
 to ask for strongly consistent data using ``consistent_read=True``.
 .. TODO: plus d'info sur eventually consistent
     e1m1 = DoomMap.get((1, 1))
 What if I want to get all the maps in a given episode? This is the purpose of the
-:py:meth:`~.DynamoDBModel.query` methode which also allows to filter the results
+:py:meth:`~.DynamoDBModel.query` methode witch also allows to filter the results
 based on the ``range_key`` value.

File docs/pages/overview.rst

View file
  • Ignore whitespace
-Dynamodb-mapper uses 3 "logging" loggers:
+Dynamodb-mapper uses 3 u"logging" loggers:
 - model
 - model.database-access

File dynamodb_mapper/model.py

View file
  • Ignore whitespace
         get_batch *always* performs eventually consistent reads.
         Please not that a batch can *not* read more than 100 items at once.
+        :param keys: iterable of keys. ex ``[(hash1, range1), (hash2, range2)]``
         if len(keys) > 100:
             raise ValueError("Too many items to read in a single batch. Maximum is 100.")
     def _save_autoincrement_hash_key(self, item):
         """Compute an autoincremented hash_key for an item and save it to the DB.
-        To achieve this goal, we keep a special object at ``hash_key=0`` to keep track
-        of the counter status. We then issue an atomic inc to the counter field.
+        To achieve this goal, we keep a special object at ``hash_key=MAGIC_KEY``
+        to keep track of the counter status. We then issue an atomic inc to the
+        counter field.
         We do not need to read it befor as we know its hesh_key yet.
         The new value is send back to us and used as the hash_key for elem