1. OpenREM Avatar OpenREM
  2. OpenREM
  3. OpenREM
  4. Issues

Use AJAX for chart plotting to make pages more responsive

Issue #253 resolved
David Platten created an issue

When charts are active there can be a significant delay between the user clicking something and the data and charts appearing.

It would be better if the tabulated data appeared straight away, followed by the charts when the server has finished its calculations.

It should be possible to do this using AJAX.

https://code.djangoproject.com/wiki/AJAX ?

http://www.w3schools.com/ajax/

http://stackoverflow.com/questions/20306981/how-do-i-integrate-ajax-with-django-applications

Comments (87)

  3. Ed McDonagh

    I think when I was looking at this, the dajax project had been dumped - check out the github.

    I don't think there actually is anything you need to pull in for this - we already have jquery and I think that is all you will need. I'm hoping that a simple implementation won't be much more than a $(document).ready(function(){ ... } wrap around your code. Or maybe I'm being a little naive?

  4. David Platten reporter

    Hi @edmcdonagh. The cause of the delay in a chart-containing page appearing is the wait for the database to return data to views.py (the rendering of the chart once the data has arrived is pretty quick). Waiting for data from the database isn't helped by wrapping the chart JavaScript code in a ready function as you suggest. I think.

  5. Ed McDonagh

    Yeah, it will be something like rendering the page with just the tabular information and a chart holding area, then triggering a different view to go off and get the data for the charts. Down the line, as it is just for prettiness, but I'd say that if you have charts plotted, you aren't really as interested in the listing, so it might be smart to minimise that block? Or putting the charts at the top above the listing, with some sort of waiting indication in that region whilst it is computing?

  6. David Platten reporter

    I've been looking into Django / AJAX things and all I've achieved so far is an increased level of confusion.

    Most of the Django / AJAX stuff that I've found seems to use AJAX to deal with a form. I want to be able to use AJAX from the first initial view of the pages, without the user submitting a form.

    I think that there will need to be a view that returns the information to populate the main body of the page, and the associated tabular data etc. This doesn't need to be AJAX. I think there then needs to be a second view that is called: this will return the chart data via AJAX, and the charts can be plotted once the data is available.

    The problem with this is that I'm not sure that it's possible to have two views contributing to one Django html template.

    It would be good to speak to someone that's done this sort of thing before, as I feel that I don't really know where to start. Any suggestions?

  7. Ed McDonagh

    I became extremely confused for a long while, and was very frustrated that there were no examples that matched what I needed to do!

    In the 207 branch I'm working on, the AJAX works like this:

    1. Browse to admin/queryremote URL
    2. View dicom_qr_page (in netdicom/dicomviews.py) thows up dicomqr.py with a form and the admin user details
    3. dicomqr.py has a form for search details, and loads dicom.js
    4. dicom.js has the ready function, and when you click on the submit button the javascript takes over in the form function, and loads the q_process view - URL is in the html, javascript uses $(this).attr('action').
    5. q_process view starts the task with Celery
    6. if the q_process view returns success then the query_progress js is called from the form submit js
    7. query_progress calls the admin/dicomstore/ajax_test3 URL every so many milliseconds, which loads the ajax_test3 view which queries the database to see how the task is progressing.

    Each of these views pass data back via json to the javascript to populate the qr-status div in the template.

    So, this does use a form, but the form isn't really relevant to the process. A button to launch the chart would make it directly comparable, but I assume you could launch automatically on load.

    Unfortunately I've a lot of unused code and inappropriate function names, so it's not as easy to follow as I'd like!

    Not sure if this is at all helpful to you, I hope it is a little!

  9. David Platten reporter

    Chart data is only calculated if response.is_ajax() is true. The chart data is returned via a JsonResponse if response.is_ajax(), otherwise just the tabulated data is returned via a render_to_response pointing to the appropriate html page. I think that this will let me use the same view for the tabulated data and the later Ajax call to obtain chart data, with different types of return depending on the request type. I haven't written any JavaScript to make the Ajax call yet. References issue #253.

    → <<cset fc019b2c2f6f>>

  10. David Platten reporter

    I think I'm going to need a separate view for the chart data, as the DXSummaryListFilter and DXChartOptionsForm items that I am putting in the current view returnStructure are not JSON serializable.

    I'll create a new view that just sends back the plot data.

  11. David Platten reporter

    Removed chart data calculation from the dx_summary_list_filter view. Created new view called dx_summary_chart_data that will eventually calculate chart data and return it via a JsonResponse. Added a bit of JavaScript to dxfiltered.html that will need to call the new view. Mostly skeletons at the moment. References issue #253.

    → <<cset 51055be28582>>

  12. David Platten reporter

    Trying to get AJAX working - without success. I have created a new view that tries to return the required chart data via JSON. However, no data is ever recieved by the JavaScript in the DXfiltered.html page. Don't know why. References issue #253.

    → <<cset 8a52cf935233>>

  13. David Platten reporter

    AJAX response back to the html template now working. Need to work out how to trigger the AJAX to get the chart data without having to click on a button so that when the page initially comes up the chart data is polled. Also need to work out how to use the returned JSON data to populate the JavaScript arrays for the charts. References issue #253.

    → <<cset 4d30fe2eeef9>>

  16. David Platten reporter

    Moved AJAX JavaScript into its own file. Chart data is now returned via AJAX when the page is initially loaded. Chart data is also updated when the user submits the form. Something isn't quite right though, but it is an improvement. Need to work out how to get the charts to use the data that is send back via AJAX. References issue #253

    → <<cset c8cb8d6aafa8>>

  17. David Platten reporter

    Almost working... One of the DX charts now updates after a form submission. Doesn't populate to begin with though... References issue #253

    → <<cset e2d33b36ccfd>>

  18. David Platten reporter

    Tabulated data and one radiographic chart now updated using AJAX on form submission. Haven't got chart data to appear when page is first loaded yet. Also need to work out how to populate the chart's drilldown data. The tabulated data is updated by replacing a DIV's html with new html provided by the view. References issue #253

    → <<cset 4f03fa3b262b>>

  19. David Platten reporter

    Chart and tabulated data now updated on initial page load. To do: update top row showing number of studies; update chart options and chart display. Sanity check: is this a sensible direction to be going in? References issue #253

    → <<cset 8a74e0395e84>>

  21. David Platten reporter

    Chart and tabulated data now being updated correctly on initial page view and subsequent form submissions. Only works for plot of DAP per mean acquisition protocol at the moment. Drilldown data not currently updated. References issue #253

    → <<cset 68a09ffb7b39>>

  24. David Platten reporter

    Plot of acquisition protocol frequency now working. Removed redundant code from the html template for the two working plots. References issue #253

    → <<cset cc1dceb30039>>

  28. David Platten reporter

    Need to tweak the plot of DAP over time so that when there is no data in a given time period it doesn't plunge to zero.

  29. David Platten reporter

    Plot of DAP over time not working for median or both selection. Will fix after holidays.

  30. David Platten reporter

    Added check for zero values on plot of DAP over time. If found, replace with so that a gap is left in the line. Fixed plot of median DAP over time: the median values are returned as strings: they are now converted to a float. References issue #253.

    → <<cset 59c3f678005a>>

  31. David Platten reporter

    Tidied up variable declarations. Kept some variables as global so that chart sorting works. Combined the code for mean and median DAP over time into one routine. References issue #253.

    → <<cset b9bf5b2483a4>>

  35. David Platten reporter

    The JavaScript for chart plotting used to be constructed in the dxfiltered.html page using Django template code. It produced a large amount of badly-formatted JavaScript. The change to using AJAX has the side-effect that all this ugly JavaScript is gone. I suspect this may have speeded things up a little, but I'm not certain. Sort-of references issue #217

  43. David Platten reporter

    Made AJAX changes for CT chart of CTDI per acquisition protocol. That's really the last of the charts - all are now AJAX. Need to fix the sorting for the combined DLP and CTDI plot (4 series), then all will be done. References issue #253.

    → <<cset 44a93bffb689>>

  47. Ed McDonagh

    Chart data is only calculated if is true. The chart data is returned via a if , otherwise just the tabulated data is returned via a pointing to the appropriate html page. I think that this will let me use the same view for the tabulated data and the later Ajax call to obtain chart data, with different types of return depending on the request type. I haven't written any JavaScript to make the Ajax call yet. References issue #253.

    → <<cset fc019b2c2f6f>>

  48. Ed McDonagh

    Removed chart data calculation from the dx_summary_list_filter view. Created new view called dx_summary_chart_data that will eventually calculate chart data and return it via a JsonResponse. Added a bit of JavaScript to dxfiltered.html that will need to call the new view. Mostly skeletons at the moment. References issue #253.

    → <<cset 51055be28582>>

  49. Ed McDonagh

    Trying to get AJAX working - without success. I have created a new view that tries to return the required chart data via JSON. However, no data is ever recieved by the JavaScript in the DXfiltered.html page. Don't know why. References issue #253.

    → <<cset 8a52cf935233>>

  50. Ed McDonagh

    AJAX response back to the html template now working. Need to work out how to trigger the AJAX to get the chart data without having to click on a button so that when the page initially comes up the chart data is polled. Also need to work out how to use the returned JSON data to populate the JavaScript arrays for the charts. References issue #253.

    → <<cset 4d30fe2eeef9>>

  53. Ed McDonagh

    Moved AJAX JavaScript into its own file. Chart data is now returned via AJAX when the page is initially loaded. Chart data is also updated when the user submits the form. Something isn't quite right though, but it is an improvement. Need to work out how to get the charts to use the data that is send back via AJAX. References issue #253

    → <<cset c8cb8d6aafa8>>

  55. Ed McDonagh

    Tabulated data and one radiographic chart now updated using AJAX on form submission. Haven't got chart data to appear when page is first loaded yet. Also need to work out how to populate the chart's drilldown data. The tabulated data is updated by replacing a DIV's html with new html provided by the view. References issue #253

    → <<cset 4f03fa3b262b>>

  56. Ed McDonagh

    Chart and tabulated data now updated on initial page load. To do: update top row showing number of studies; update chart options and chart display. Sanity check: is this a sensible direction to be going in? References issue #253

    → <<cset 8a74e0395e84>>

  57. Ed McDonagh

    Chart and tabulated data now being updated correctly on initial page view and subsequent form submissions. Only works for plot of DAP per mean acquisition protocol at the moment. Drilldown data not currently updated. References issue #253

    → <<cset 68a09ffb7b39>>

  64. Ed McDonagh

    Added check for zero values on plot of DAP over time. If found, replace with so that a gap is left in the line. Fixed plot of median DAP over time: the median values are returned as strings: they are now converted to a float. References issue #253.

    → <<cset 59c3f678005a>>

  65. Ed McDonagh

    Tidied up variable declarations. Kept some variables as global so that chart sorting works. Combined the code for mean and median DAP over time into one routine. References issue #253.

    → <<cset b9bf5b2483a4>>

  67. Ed McDonagh

    Removed reliance on global JavaScript variables to hold chart data for all radiographic plots. These changes will have broken the sorting of the CT charts until I update them too. References issue #253.

    → <<cset c5df4b6b4246>>

  75. Ed McDonagh

    Made AJAX changes for CT chart of CTDI per acquisition protocol. That's really the last of the charts - all are now AJAX. Need to fix the sorting for the combined DLP and CTDI plot (4 series), then all will be done. References issue #253.

    → <<cset 44a93bffb689>>

  78. David Platten reporter

    Forgot to add the updated css file (Added animated gif that is displayed in the centre of the screen whilst waiting for chart data. References issue #253.)

    → <<cset 1d662d1e4a88>>

  81. Ed McDonagh

    I like, but I'd like it better if the charts, and the annimation, pushed the tabular data down the page.

  82. David Platten reporter

    Something like it is now? [or not - they're at the top and bottom... Odd - I moved the plotdata block into the toprow block. This did move the charts up to the top, but rather strangely there were also the chart titles below the tabulated data too. Don't understand why that was. Moved them back to where they were for the time being.]

  83. Ed McDonagh

    Ok. Just checked it after seeing the email, and the wait logo was at the top over the tabular data, which is fine, but the charts were still at the bottom. Now I've read your update and it makes sense.

  84. David Platten reporter

    I need to change the location of the plotdata block in filteredbase.html, rather than in dxfiltered.html and ctfiltered.html. The presence of a plotdata block in filteredbase.html explains why I saw duplicate chart titles at the top and bottom of the tabulated data yesterday. I'd forgotted about the base templates.

  85. David Platten reporter

    Added a line to remove the animated gif from the screen if something goes wrong with retrieving the chart data via AJAX. References issue #253.

    → <<cset c733213fc234>>

  88. Log in to comment
Assignee
Type
enhancement
Priority
major
Status
resolved
Component
Interface: Data visualisation
Milestone
0.7.0
Votes
0
Watchers
2
Jira: the preferred issue tracker for Bitbucket. Join the team!