Improve memory efficiency of chart DataFrame creation

Issue #870 resolved
David Platten created an issue

No description provided.

Comments (14)

  1. David Platten reporter

    Updated Plotly to latest version. This solves the multi-index error that Ed and I have seen. I've also upgraded the plotly JavaScript file. Upgrading Plotly has changed the data that is returned for the tests a little, so I've updated the test files too. They all run fine on my local computer. I have also updated pandas to the latest version. Refs issue #870 (sort of)

    → <<cset e03f1fa4eff2>>

  2. David Platten reporter

    Merged in issue870improveDataFrameMemoryEfficiency (pull request #426)

    Using values_list rather than values to read Django queryset into the DataFrame. This reduces the memory requirement by around 50 %. It is also faster (3 s vs 5 s for 600,000 radiographic studies when plotting the workload chart).

    Approved-by: Ed McDonagh

    Fixes issue #870

    → <<cset 2f90276772f2>>

  3. David Platten reporter

    Removed the custom ordering on the query set before creation of chart DataFrame. This custom ordering isn't needed by the chart routines (it is used to order the summary view tables in the order specified by the user), and slows down the DataFrame creation. In some of my tests this reduces the DataFrame creation time from 10 s to 7 s. Refs issue #870

    → <<cset de73ed7ac117>>

  4. Log in to comment