Allow complex filtering - NOT, OR

Issue #17 new
Ed McDonagh
created an issue

No description provided.

Comments (24)

  1. Ed McDonagh reporter

    Thanks David. It hadn't occurred to me that the actual database lookups were going to be difficult. Looking at the Q docs, it appears that I hadn't really looked into it very far!

    My initial issue is how to have an interface that allows complex sets of queries in a usable form, both for the end user and the person maintaining the code.

  2. David Platten

    You could have a series of drop-down lists containing field names. Each drop-down list could be separated from the next by a pair of fixed-choice buttons setting "OR" or "AND" between the lists. Where it gets tricky is that you may want the user to be able to use brackets around some of their selections.

  3. Luuk

    Does someone know if their are "plugins" for Django to get a type of filtering on the models like the "advanced" filter options on the Bitbucket issues page or even "more advanced" like pubmed? This last one can also do the "bracket-option". I would think this is a quite common feature needed, but I can't find anything. It might be I'm searching with the wrong terms.

  4. Luuk

    Refs #17 allow complex filtering

    First attempt implementing complex filtering by a query builder form using Django Q objects: - For now only fields in CT models are selectable (DX/RF/Mammo-fields are not added, but could easily) - AND / OR / AND NOT / OR NOT can be chosen to connect constraints on fields - Brackets can be used (first and last textbox, this might be unclear for the user) - For now the search is completely modality independent, although results use CT template and only CT models are loaded - For now page is only reachable by manually entering http://server:port/openrem/advanced_search

    Things that need to be done (at least): - Check for validity of user input * only decimals in decimal fields, dates in date fields, etc.. * Correct placing of brackets - Results for queries without brackets are almost certainly correct, but with brackets need to be tested thoroughly - If query is sent, the results page should also contain the query as built by the user (textboxes should be filled). So the user can adapt the query easily - Think about available fields...There are a lot. Do we need to sort or limit them? - Verbose name is used for displaying field names, but we may need to add these to the model for a number of fields for clarity - Should it be possible to select "NOT" for the first field search? - Pressing <Enter> while "add line" is selected will add extra line, but selected "add line" icon remains visible and next line added is empty - Write test script

    → <<cset 638fb3e0f115>>

  5. Luuk

    Refs #17 allow complex filtering

    First attempt implementing complex filtering by a query builder form using Django Q objects: - For now only fields in CT models are selectable (DX/RF/Mammo-fields are not added, but could easily) - AND / OR / AND NOT / OR NOT can be chosen to connect constraints on fields - Brackets can be used (first and last textbox, this might be unclear for the user) - For now the search is completely modality independent, although results use CT template and only CT models are loaded - For now page is only reachable by manually entering http://server:port/openrem/advanced_search

    Things that need to be done (at least): - Check for validity of user input * only decimals in decimal fields, dates in date fields, etc.. * Correct placing of brackets - Results for queries without brackets are almost certainly correct, but with brackets need to be tested thoroughly - If query is sent, the results page should also contain the query as built by the user (textboxes should be filled). So the user can adapt the query easily - Think about available fields...There are a lot. Do we need to sort or limit them? - Verbose name is used for displaying field names, but we may need to add these to the model for a number of fields for clarity - Should it be possible to select "NOT" for the first field search? - Pressing <Enter> while "add line" is selected will add extra line, but selected "add line" icon remains visible and next line added is empty - Write test script

    → <<cset faabb7ec3cdc>>

  6. Luuk

    Refs #17 Advanced Filtering

    • Added advanced filtering option to CT filtered page as example (can be quite easily implemented for other modalities)
    • Needed to rewrite ctfiltered template as "form" has to extend multiple blocks
    • Made it possible to swich from exam filter to advanced filter and vice versa
    • Improved advanced filtering with possibility having NOT in first line
    • Updated logic to never negate root node (but always add a NOT node)
    • Updated logic to respect logic for user specific "pid"
    • Updated logic to be able to insert filter on modality independent of user input (depending on modality viewed)
    • Resolved PEP8 issues in mod_filters

    → <<cset 406a6f51acae>>

  7. Luuk

    I just uploaded an advanced filtering option in the CT filtered page. I'm rather happy with it (of course there are some "issues'). It can be quite easily build in the DX and RF filtered pages also.

    Some "highlights":

    • The idea of the advanced filtering is based on the Pubmed advanced search
    • Default behaviour of the CT filtered is unchanged
    • It is possible to switch from the default "exam filter" to the advanced filtering within the CT filtered page and vice versa
    • Currently a lot of filter parameters can be selected (should they be sorted? reduced?).
    • You can use brackets and AND/OR/NOT operators in the search options (next to the comparison operators depending on the field type).

    Some Remarks:

    • Currently the plots don't respect the advanced filter
      • I don't understand how data flows to the plots, but it seems dependent on the specific "exam filter" fields
      • It would be great if you could pass a filter- or querset-object
    • I tested it with a limited number of datasets and it seems to do the right thing, but I'm not sure if it works in all cases
    • I think it is unclear that the textbox just before the filterfield and the last textbox in each line are meant to place brackets "(" or ")"
      • Any ideas how to improve this?
    • Checking that "number-fields" only contain a number in the filter value is not implemented (yet, same holds through for dates)
    • In the end it would be great if you would be able to save (and load) queries
      • Then it would also be nice if date-fields could be relative ("from on year ago till now")
      • For now you can just copy-paste URLs

    @Ed McDonagh, @David Platten what do you think? Any suggestions to improve this filtering?

  8. Log in to comment