Issue #61 open
Dimitris Glezos
created an issue

Sometimes the result set is just too large to handle. Out-of-the-box pagination support in would be hawt!

Comments (12)

  1. Joe Banafato

    I'm interested in tackling this one. I've written an implementation of partial get on 2.2. If you were to accept a patch, what would you like it to be against?

  2. Anonymous

    Thanks for the info. Agreed. I have a much different implementation. I suppose I'll fork from tip and you can review it if you're interested.

  3. Joe Banafato

    Please take a look at my fork at I'm extending RFC 2616 sec. 14.35 to add a "records" range unit. Test cases are provided. To try it, add a request header in the form "Range: records=start-end" where start and end are in the naturals. The response will be a 206 with the Content-Range header set. An unparseable header will result in a 400 response, and an unsatifiable request will result in a 416. To experiment with the blog application, try:

    $ curl -i  -H 'Range: records=0-1'  http://localhost:8000/api/posts/
    $ curl -i  -H 'Range: records=1-'  http://localhost:8000/api/posts/
    $ curl -i  -H 'Range: records=-1'  http://localhost:8000/api/posts/
    $ curl -i  -H 'Range: records=5-7'  http://localhost:8000/api/posts/
    $ curl -i  -H 'Range: records=abc'  http://localhost:8000/api/posts/

    I welcome comments and suggestions. I'm seriously considering adding request parameters support for ease of use when setting headers is more cumbersome, e.g., modifying the underlying XHR in jQuery, but I hesitate for two reasons:

    1. The spec is designed to allow extension via range-units, so I feel this is in line with the design intention.
    2. I don't want to worry about possibly interfering with existing applications' request namespaces, i.e., parameter name collisions.
  4. Jesper Noehr repo owner

    Thanks Joe.

    The records= extension is fine, I think.

    My gripe is also with your second point, specifying this in the clients (setting headers is cumbersome.)

    I'm not entirely sure how we can get around that. The other pagination fork uses GET parameters to specify beginning/count, but that pollutes the namespace, like you said.

    Hm. Just a random idea: What if we had a configuration parameter, a two-tuple, containing the names of said GET parameters? E.g. PISTON_PAGINATION_PARAMS = ('start', 'count') by default, so that ?start=0&count=20 would translate into Range: records=0-20? It would still piggyback on your implementation, getting all the HTTP conforming goodies (416 etc.).

    I don't know. I think we need to standardize pagination in Piston in a way that extends beyond the headers. As long as it's properly documented and configurable, I think we can get away with something like the above.

    I'd appreciate your thoughts on this.

  5. Joe Banafato

    Done with a basic implementation. Add ?offset=n&limit=m to your query string. You can override the parameters in per your example, e.g.. PISTON_PAGINATION_PARAMS = ('start', 'count'). The nuances of the parameters are documented in the code and demonstrated in the test cases.

    I suspect params support will be most useful in ajax contexts, so I think it's probably best to return 200 instead of 206 to support tools that only accept 200 as success. Also, I have the frequent need to consume my services with data-aware toolkits like YUI and Ext, so I intend to create specialized JSON emitters that construct the expected schemas out of the box.

  6. Jesper Noehr repo owner

    Very well.

    Offset and limit is what I call it in my APIs too, so good job on the nomenclature.

    If anyone wants to object, or add anything to this discussion about standardizing pagination in Piston, now's the time to speak up.

    I'm going to send a link to this thread to the mailinglist, so we can get some more feedback.

  7. Anonymous

    I'd love to see this get into piston, it's basically the only thing stopping me from using it right now. What is the status of the adoption right now?

  8. Anonymous

    Hi all

    +1 - Any timeframe on when pagination is going into Piston? We are using it today (it's great!) but now have an urgent need for pagination.


  9. Log in to comment