Issue #3 resolved

Table from list helper

Mike Orr
created an issue

Ergo has a helper to render an HTML table from a sequence (including SQLAlchemy RowProxy). Waiting for the code.

Comments (52)

  1. Marcin Lulek

    Im,a beginner programmer in python so this may need some improving ;-)

    what this class can do:

    • ordering support for columns with get params
    • customizable row and column rendering via lambdas, support for adding new columns that dont exist in passed lists like row with links to perform actions,
    • any default column rendering style can be overwritten with something custom
    • rows can be overwritten so user can create something like <tr>columns_row</tr><tr>some_other_info</tr>
    • builtin support for even/odd class styling of rows
    • auto generation of nice headers from keys of sa rows, so something_label becoms Something Label
    • headers have nice styling support with smart class usage, for both ordered and unordered fields
    • support for overwriting any header with something else
    • support to specify what columns are rendered and in what order by columns argument
    • every custom row or column markup is specified by lambda/function that returns formattable string

    the simpliest example usage for mako:

    <%
    from webreactor.lib.grid import Grid
    news_grid = Grid(c.news_page,columns=['title','date_of_creation'])
    %>
    

    <table class="stylized"> ${news_grid.render()|n} </table>

    more complex examples:

    <%
    from webreactor.lib.grid import Grid
    news_grid = Grid(c.news_page,columns=['title','date_of_creation','actions'])
    news_grid.exclude_ordering = news_grid.columns
    news_grid.label = {'title':'Some other title'}
    news_grid.format = {
    'actions':lambda i,item: '<td><a href="%s"><span class="icon newsUpdate"></span></a></td>' % (url())
    }
    %>
    

    <table class="stylized"> ${news_grid.render()|n} </table>

  2. Mike Orr reporter

    I have put it in the 'unfinished' directory in the WebHelpers source, along with my notes reproduced below. I have some other WebHelpers things I planned to do today, so I'll get back to this next week. In the meantime if you want an exercise, you can work on converting the HTML generation. I might also make my own implementation and we can compare.

    Grid integration plan for WebHelpers

    The main problem with table generators is that different people want different kinds of tables, and you want to try to accommodate them as much as possible, without making the simple case too cumbersome. I'd suggest putting the row-number feature in a subclass because many tables won't need it.

    I'd also suggest a subclass or option for a horizontal table: one with the headers in the left row. These are often two-column tables, with the labels implied by the first column in the data. E.g., dumping a dict.

    Generate all HTML using webhelpers.html.builder (examples in webhelpers.html.tags). This will also mark the HTML as a preformatted literal, so Mako's "|n" filter will no longer be necessary.

    Replace .render method with .str . This will make it automatically render when stringified, as Mako does with ${my_grid}.

    Write complete module docstring with usage and tests. (See examples in webhelpers.html.tags.) Document that this generates only <tr> and below, not <table> or <call> etc.

    There doesn't seem to be much use for ``*args`` and ``\*\*kwargs`` since you accept a small number of arguments with fixed meanings. I would replace those with positional args, with default values for those that are optional. 'columns' looks like it should be the second arg and required. Does ``.make_header_columns`` work if 'columns' is missing?

    Using methods to generate the header columns and regular columns looks interesting; I haven't seen another table class do that, but it may make it more flexible for subclassing. Perhaps even more flexible would be a hook method that returns: {tr_attributes}, [ (is_th, {th_or_td_attributes}, cell_content) ] This might be general enough to support all kinds of tables.

    Double-underscore method names are unnecessary, because this is unlikely to be subclassed except to customize the formatting, and the subclass would not have conflicting names.

    I don't understand the ordering code, so I'm not sure what it's doing.

    'labels' is ambiguous: is it a column header, form element <label>, row or cell label, etc? Consider 'header' instead.

    Interesting that ``.labels`` is a dict rather than a list of headers. I suppose that makes it easier to move columns around.

  3. Marcin Lulek

    dont make your own implementation before i provide you with examples and use cases, the class i developed was tested in all kinds of wacky situations ;-)

    1. "The main problem with table generators is that different people want different kinds of tables, and you want to try to accommodate them as much as possible, without making the simple case too cumbersome. I'd suggest putting the row-number feature in a subclass because many tables won't need it. "

    this is already done, because you have to add columns=["_numbered","foo","bar"], as the column that holds ordering, you dont add it , it doesnt order, so i consider this a non issue.

    2. I'd also suggest a subclass or option for a horizontal table: one with the headers in the left row. These are often two-column tables, with the labels implied by the first column in the data. E.g., dumping a dict.

    can i see an example of what you are talking about? screenshot would do.

    3. "There doesn't seem to be much use for ``*args`` and ``\*\*kwargs`` since you accept a small number of arguments with fixed meanings. I would replace those with positional args, with default values for those that are optional. 'columns' looks like it should be the second arg and required. Does ``.make_header_columns`` work if 'columns' is missing"

    right, this will get changed.

    4. Using methods to generate the header columns and regular columns looks interesting; I haven't seen another table class do that, but it may make it more flexible for subclassing. Perhaps even more flexible would be a hook method that returns: {tr_attributes}, [ (is_th, {th_or_td_attributes}, cell_content) ] This might be general enough to support all kinds of tables.

    it should already be able to do all kinds of stuff, i dont want any is_th etc. because this class could allow to generate something other than table, it could do div's, li's, etc. thats why we use methods, functions to format any element of our grid, anything should be possible to do. and the easiest case is the use of lambdas, ill provide you some cases tommorow if i find the time.

    4. I don't understand the ordering code, so I'm not sure what it's doing.

    in nutshell, the header cells become a tags, that preserve all params and add another get param : order_by=column_name, so you can click the header to set some get params you can handle in controller or model. it allso add some additional css so user can style headers that can be used or ordering, or those that are "on" in current moment (see screenshot).

    5. 'labels' is ambiguous: is it a column header, form element <label>, row or cell label, etc? Consider 'header' instead.

    yes, i will do that.

    6. Interesting that ``.labels`` is a dict rather than a list of headers. I suppose that makes it easier to move columns around.

    correct, you can change order of "columns" list elems and that should always work.

    7. Generate all HTML using webhelpers.html.builder (examples in webhelpers.html.tags). This will also mark the HTML as a preformatted literal, so Mako's "|n" filter will no longer be necessary.

    not sure if that would work too good, because you can make customizations like 1 item is actually 2 rows right now etc. ill try to supply you some more code examples with complicated cases, this will lighten things up for you, i think had a good purpose to do things like they are now, and keep balance between simplicity of code and retain full flexibility of grid. i think that right now i have the simpliest solution possible for users. Will supply code examples later

    8. Write complete module docstring with usage and tests. (See examples in webhelpers.html.tags.)

    i may need a bit help on that, if possible, never did any tests so far.

  4. Marcin Lulek

    code from screenshot example

    <%

    tickets_grid = Grid(c.tickets, columns=['_numbered','subject','category','status','date'])
    #lets override how rows look like
    #subject is link
    #categories and status hold text based on param of item text , the translations are dicts holding translation strings correlated with integers from db, in this example
    tickets_grid.format = {
    'subject':lambda i,item: '\n\t<td><a href="%s">%s</a></td>' % (url(controller='tickets',action='view_ticket',ticket_id=item['id']),item['subject']),
    'category':lambda i,item: '\n\t<td>%s</td>' % (app_globals.ticket_category_translations[item['category']]),
    'status':lambda i,item: '\n\t<td>%s</td>' % (app_globals.ticket_status_translations[item['status']]),
    }
    %>
    

    <table class="stylized tickets"> <caption>Tickets</caption> ${tickets_grid.render()|n} </table>

  5. Marcin Lulek

    this is a more complicated example where one row from db is actually 2 rows in table display:

    	<%
    	from points2shop.lib.grid import Grid
    	
    	def custum_row_markup(class_name,i, row, columns):
    		data = '<a class="button" href="%s" name="claim" title="Post a helpful comment to earn a lottery ticket">Post&nbsp;Comment</a>' % url(controller='offers', action='view_information', offer_id=row.id, name=row.name)
    		data += '<a class="button" href="%s" name="claim" title="More info regarding this offer">More&nbsp;Offer&nbsp;Information</a>' % url(controller='offers', action='view_offer', offer_id=row.id, name=row.name)
    		if row.monthly_retry == 1 and row.status < (datetime.date() - timedelta(months=1)):
    			data += '<a class="button" href="%s" title="Unignore offer">Add offer back to offer list</a>' % url(controller='offers', action='delete_claimed_offer', offer_id=row.id)
    		if row.status == app_globals.signup_status['STATUS_SIGNUP_IGNORE']:
    			data += '<a class="button" href="%s" title="Unignore offer">Unignore</a>' % url(controller='offers', action='unignore_offer', offer_id=row.id)
    		if row.status == app_globals.signup_status['STATUS_SIGNUP_DO_LATER']:
    			data += '<a class="button" href="%s" title="Unignore offer">Add offer back to offer list</a>' % url(controller='offers', action='add_offer_back_to_offer_list', offer_id=row.id)
    		if row.status == app_globals.signup_status['STATUS_SIGNUP_LOCKED']:
    			data += '<a class="button" href="%s" name="claim" title="Let us know if there is something wrong with this offer">Report&nbsp;Broken&nbsp;Offer</a>' % url(controller='support_tickets', action='add_ticket', category=app_globals.ticket_category['CATEGORY_TICKET_OFFER_NOT_WORKING'], subject='Error%20in%20offer%20'+row.name+'('+unicode(row.id)+')')
    		return '\n\t<tr class="%s">%s</tr><tr class="%s"><td colspan="7">%s</td></tr>' % (class_name,columns,class_name,data)
    
    	completed_offers_grid = Grid(c.completed_offers, columns=['name','date','status','siteid','value','reply','message'])
    	completed_offers_grid.custom_row_format = custum_row_markup
    	completed_offers_grid.format = {	
    	'name':lambda i,item: '\n\t<td><a href="%s">%s</a></td>' % (url(controller='offers',action='view_offer',offer_id=item['pts_id']),item['name']),
    	'status':lambda i,item: '\n\t<td>%s</td>' % (app_globals.signup_status_translations[item['status']]),
    	'message':lambda i,item: '\n\t<td>%s</td></tr>' % (item['message'])
    	}
    	%>
    

    as you can see the first row is displayed normally, it holds the columns, the second row is one cell spanning on all columns holding some links to performa actions.

    yea the code is ugly but itd not mine ;-)

  6. Mike Orr reporter

    "I'd also suggest a subclass or option for a horizontal table: one with the headers in the left row. --can i see an example of what you are talking about?"

    Well, the site is private so I can't make a screenshot, but just what it says. You want to display several attributes about something, and the values are too long to have columns going across, so instead you put each header in the left column and its value in the right column.

    "i dont want any is_th etc. because this class could allow to generate something other than table, it could do div's, li's, etc"

    Well, it's not what you want, but what would be most helpful to all WebHelpers users. :) Having a is_th flag might make it easier to do both a vertical and horizontal table. But maybe not. I do agree with the principle of div's and li's and stuff, but we'd need at least some preliminary code to show that this is feasable. For instance, how would you have multiple columns in a div or li. You can use embedded divs with float, but then everybody would want different attributes. Or if it's limited to divs without columns, why does it have to be the same class that renders it?

    "the easiest case is the use of lambdas"

    Lambdas are hard to read, so I'd rather leave them out of the code or examples unless the

    "Generate all HTML using webhelpers.html.builder --not sure if that would work too good, because you can make customizations like 1 item is actually 2 rows right now etc."

    This was one of the main features in WebHelpers 0.6, and something we've promised users that all HTML generation helpers will do from hereonout. It increases security by automatically escaping values that may not have been intended to be markup.

    "i may need a bit help on that, if possible, never did any tests so far."

    I can help with that.

  7. Marcin Lulek

    in this example our table will consist of 3 columns, first one will contain standard no. integers, second will be standard row that basicly displays value of the key, the third one "options" is a column that doesnt exist at all in the list/sa_proxy we passed in, instead its an artificial column with icons that links to perdorm sone actions like run update action or remove action:

    <%
    from webreactor.lib.grid import Grid
    tickets_grid = Grid(c.groups, columns=['_numbered','group_name','options'])
    tickets_grid.format = {
    'options':lambda i,item: '\n\t<td><a href="%(update_url)s"><span class="icon roleUpdate"></span></a><a href="%(remove_url)s"><span class="icon roleRemove"></span></a></td>' % {'update_url':url('admin_groups',action='group_update',group_id=item.id),'remove_url':url('admin_groups',action='group_remove',group_id=item.id)}
    }
    %>
    

    <table class="stylized groups"> <caption>${_('Groups')}</caption> <tbody> ${tickets_grid.render()|n} </tbody> </table>

  8. Mike Orr reporter

    Those two examples of building up HTML strings with interpolation are precisely what we're trying to get away from, and why we ditched the rails helpers. If people need a column spanning multiple rows, they should make a subclass for that case, and not pass it in a .custom_row_format string. This also shows the limits of generating tables via a class: wouldn't a hardcoded table in a template be easier to read, or a Mako function? I tried earlier to make a grid class for laying out form fields in either table or div format, but gave up because of the number of arguments it required and disagreements among people about the best HTML structure for these. So maybe WebHelpers just wants a simple class that can handle two or three common scenarios, and leave it at that.

  9. Marcin Lulek

    Well, it's not what you want, but what would be most helpful to all WebHelpers users. :)

    hmm i took a peek into html webhelpers, i guess it would be possible to do it with them, but im affraid that would add quite more complexity to the class usage, but thats a guess, and i may be completly wrong about it.

  10. Marcin Lulek

    i think you are right about simple scenario, but there is one thing that i think its important to consider.

    If i really want to just print a simple table, helper would not be something very attractive to me at least, because doing a simple loop does the trick, and its trivial enough to do it in mako template, helper maybe could cut it to 1-2 lines, but then again i think 90% of use cases like admin panels of some kind etc. needs flexibility, and when it needs flexibility and we start messing up with webhelpers html, we will have so much complexity in code, that it may be possible that its easier to do it step by step with pure mako control structures. Ive hit that problem when i tried formalchemy, for most of my use cases i lost more time customizing what i needed, than i would do directly. so i think its something important to keep in mind.

  11. Mike Orr reporter

    OK, grid.py looks much better. The remaining issues are 'url_for' and 'pylons.request'. 'url_for' will go away eventually; its replacement is 'pylons.url'. But if there are Pylons dependencies in the module, I'll have to put it under webhelpers.pylonslib and it won't be as easily available for non-Pylons uses.

    So we somehow need to get the results of the params and URL calls into the class rather than calling the pylons globals directly. They can go in as arguments, attributes, or method calls, anything that's in a generic (framework-neutral) format. Do you have any ideas? We should also not depend on the name of the query parameter; people may want to use 'order' or 'sort' instead, especially if they're typing the URL manually.

  12. Marcin Lulek

    hm... for the query parameter name , we could make a property that would hold the name of get param.

    but as for losing the request dependency, i dont have any idea how to go around this easy way.

  13. Mike Orr reporter

    That was grandfathered in because it was already there. But it's a dependency on Routes, which is OK for Pylons but not ideal for a framework-neutral library. The import was moved into the methods using it so that people without Routes could still run the unit tests.

    Since the grid helper operates within a request, the order value is already known, as is the theoretical URLs for all the sort columns. So I'm thinking about passing the order as a string (where None or "" mean don't change the order), and the URLs as a dict of {column id: url}. The helper can then take the URLs it needs and ignore the others. This all is more or less what I did in my application, and would make the helper usable everywhere.

  14. Marcin Lulek

    there was a very small bug in file i uploaded,

    this is how make columns should look like i accidently overwrote i in one place that would remove a helpful var access in column rendering

        def make_columns(self, i, record):
            columns = []
            if '_numbered' in self.columns:
                columns.append(self.numbered_column_format(i + self._start_number, record))
                for column in self.columns[1:]:
                    if column in self.format:
                        pass
                        columns.append(self.format[column](i, record))
                    else:
                        columns.append(self.default_column_format(i, record, column))     
            else:
                for y,column in enumerate(self.columns):
                    if column in self.format:
                        columns.append(self.format[column](i, record))
                    else:
                        columns.append(self.default_column_format(i, record, column)) 
            return HTML(*columns)
    
  15. Mike Orr reporter

    I've been planning to make an alternate implementation that hopefully will have the same essential features but I think might be a better fit for WebHelpers. To that end, could you make a list of what you think are the essential features, and a couple test cases to test it with?

  16. Marcin Lulek

    in my opinion:

    • support for ordering of columns
    • support for ability to change complete apperance of columns and rows ( for example adding clickable links)
    • including arbitrary columns with custom content
    • support for changing labels on headers
    • iteration counter for rows and applying odd/even class
    • show/hide specific columns

    at least i use those functionalities every day, basicly its everything my class does and we do use it in a lot of scenarios

  17. Mike Orr reporter

    I'd still like to put it in, but it's just too big a job to analyze the API and implementation and decide if it's right for WebHelpers long-term. I'm hoping to get WebHelpers 1.0 out this month and I still have a bunch of documentation to do, so it would probably be after that. You might want to release your package standalone in the meantime.

  18. Mike Orr reporter

    I decided to go ahead and put this in as webhelpers.html.grid, but we must eliminate the dependency on 'request' and 'url_for' first. Otherwise it'll have to go under webhelpers.pylonslib and non-Pylons users may not see it.

    I made some suggestions above about how to eliminate the dependencies by passing in the specific information it needs as arguments. I'm not sure whether these should be set in the constructor as instance attributes, or passed in to..., well, you can't pass anything into html or str , so I guess they'll have to go into the constructor.

    Is the 'default_' and non-default formatters necessary? Or can we just have people do formatters by subclassing? That would make the implementation simpler and I think prevent people from passing in lambdas, which are hard to debug.

    The other thing we'll need is a test suite that can also serve as usage examples. You can simply take the tests you have above, although please change the lambdas to named functions. If you're not familiar with nose, you just need a module with some 'def test_*' functions that raise AssertionError if a test fails.

  19. Marcin Lulek

    ill take a look at the code today and try to work on it to get rid of the depenencies and allow for passing of sorting mechanisms. Also i think we could add a subclass of grid that would simplyfy that for pylons.

    yes the default and non default formatters are needed in my opinion, because most of time you will use grid inside a template, and use default formatters for most of the columns, then if you need something custom you easly pass a simple lambda for specific column you want, subclassing would be counter productive in my opionion and hard to maintain, especially with big site you can end up with 50 or 100 classes just for simple purpose of changing some field i would go mad. Its also the same way formalchemy is using for formatting its grids - this approach works fine and i think its the easiest way to go

  20. Mike Orr reporter

    OK, I'll put it in as soon as you get rid of the dependencies, and people can start using it in the beta. If you want to make a subclass for Pylons, I can put it in webhelpers.pylonslib.

    Can you delete the obsolete grid attachments? If not, please put the date on new ones; e.g., grid_2009-12-22.py . That way I can make sure I get the current one. Or you can fork WebHelpers into your own repository and put your changes there; that way it'll be clear which version is current.

  21. Mike Orr reporter

    I made an outline for a demo program (webhelpers.html.grid_demo). It writes a series of grids to HTML files in a directory, and has space for a stylesheet. See what you think. If you like it you can put it in your repository and customize it.

    Each demo is a class with 'title' and 'description' attributes and a 'get_grid' method. I made a stub Tickets demo with methods instead of lambdas, based on your code above. You'll have to define the data in the class somehow. Maybe a class attribute 'items'? You may be able to factor out more of the grid structure/data into attributes but I didn't want to assume what kinds of grids you'd want to demonstrate. The 'description' attribute should be a paragraph describing the features of the demo.

    (I can't attach a file while writing a comment. I'll try committing the comment and then attaching the file.)

  22. Marcin Lulek

    i have updated everything and added support for developers to create their own mechanisms for header links to allow for data ordering.

    The grid by default now doesnt allow for ordering but there is also class (GridPylons) that inherits from it and overloads a method responsible for this with example implementation for pylons.

    everything sits here: http://bitbucket.org/ergo/webhelpers-ergo/

  23. Mike Orr reporter

    Yes, please, that's what the stylesheet string is for. Or if you think it would have other uses beyond the demo, we can put it under webhelpers/public/stylesheets/ .

  24. Mike Orr reporter

    Checked in webhelpers.html.grid, webhelpers.pylonslib.grid, and unfinished/grid_test.py.

    Made a few cosmetic changes to the source. Tightened up the init args. Deleted *args because it's unused. Changed **kwargs to discrete args. Made 'columns' required because it's assumed to exist. Renamed 'format' to 'column_formats' to document it. Moved the default formatting methods to the bottom of the class to put init at top.

    It still needs some nose-compatible tests and a stylesheet.

    Should "no." be "No."?

  25. Marcin Lulek

    i commited a stylesheet to my repo, feel free to pull the changes.

    Should "no." be "No."? - sorry, im not english speaker so im not sure whats the right form here ;-)

    Ive clened up old unnecessary files for grid in /unfinished, there are still some errors in the test file you have made, im not sure how should i proceed from this point.

  26. Mike Orr reporter

    webhelpers/html/grid.py and webhelpers/pylonslib/grid.py have already been created and I made some changes, so I need you to make your changes directly there rather than in unfinished/grid.py and unfinished/grid_pylons.py. Otherwise it's too difficult for me to tell which new changes you've made in unfinished that I need to apply. unfinished/grid.py and unfinished/grid_pylons.py should be deleted to avoid confusion.

    I can take grid.css directly because it's a new file. grid_demo.py can remain in unfinished until it's ready to run. (I also have to decide where to put it, whether in webhelpers/html or a new package webhelpers/demo.)

    Which errors of mine did you find? I have never run grid_demo.py so there could be some syntax errors or logic errors; go ahead and make any changes yourself.

    English speakers generally use "#" rather than "No." I know Russian uses "No" all the time in cases where we would use "#". But some business sites would find "#" too informal, and may prefer "No.", "ID", or blank. It would be best to give a default value but make it overridable, so that businesses can choose the label themselves.

  27. Marcin Lulek

    i deleted unfinished/grid.py and unfinished/grid_pylons.py, in last commit ;-),

    "It would be best to give a default value but make it overridable, so that businesses can choose the label themselves." - The ability to label the header is in there already, maybe we should just change default '.no' to '#' then.

    It would be good if we can talk on irc on #pylons or via im, cause communication here is slow :-)

  28. Mike Orr reporter

    It looks good except for two things.

    1) If '.request' is a required attribute, we should support it in the constructor. How about:

    def __init__(self, request, *args, **kw):
        self.request = request
        grid.Grid.__init__(self, *args, **kw)
    

    2) Break up lines longer than 79 characters. I don't follow everything in PEP 8 but I try to follow most of it.

  29. Mike Orr reporter

    OK, it's all merged. But WebHelpers 1.0b3 has the version as of yesterday.

    I'll write some basic documentation as part of the WebHelpers docs renovation, and then we can finish up the examples/demos.

    Do you like the basic concept of grid_demo.py?

  30. Marcin Lulek

    GridPylons will not function correctly then if I'm correct.

    The Grid should work without issues. The changes that i added today were important fixes and it would be very good to have them in (sorry about that).

    i changed that piece of code in demo generator because it gave me an exception

    demos = [x for x in globals().iteritems() if\ RuntimeError: dictionary changed size during iteration

    I also commited 2 demos in your file, that work fine and show basic functionality of grid.

  31. Mike Orr reporter

    There were multiple mistakes in the demos expression. I made a new helper for it.

    All your changes are pushed, and I made the demo live at webhelpers.html.grid_demo.

    I'll make a release in a few days, or sooner if somebody actually tries to use PylonsGrid and reports an error.

  32. Marcin Lulek

    Added some grid documentation, and ability to sort grid even for non subclassed grids if someone needs that + some more demos. Please pull all changes

  33. Mike Orr reporter

    OK, it's all pushed to dev.

    There are several stray 'c' characters in the class names in the docstring. Are they intended: <td class="cNO COLUMN_NAME">VALUE</td>

  34. Log in to comment