Issue488MostExportsDontIncludeStudyTime
Ed McDonagh
Branch: Issue488MostExportsDontIncludeStudyTime
Branch: develop
Merged
Merged pull request
Merged in Issue488MostExportsDontIncludeStudyTime (pull request #125)
Merged in Issue488MostExportsDontIncludeStudyTime (pull request #125)
Refs #488, #556, #484
Ready for review. Implements consistent common columns across all modalities, csv and xlsx.
Massively improved code, removal of duplication. Could definitely be improved further.
Improved efficiency too. Timings:
Export type
number of records
0.7.4
this branch
CT XLSX
183
16 minutes, 24 seconds
7 minutes, 12 seconds
CT CSV
183
6 minutes, 56 seconds
6 minutes, 40 seconds
RF XLSX
22
1 hour 13 minutes
1 hour 1 minute
RF CSV
22
22.4 seconds
15.8 seconds
RF CSV
47
47.2 seconds
31.8 seconds
DX XLSX
59
4 minutes 18 seconds
2 minutes 4 seconds
DX XLSX
205
15 minutes 35 seconds
6 minutes 39 seconds
DX CSV
205
5 minutes 36 seconds
5 minutes 50 seconds
MG CSV
137
14 minutes 37 seconds
8 minutes 16 seconds
MG XLSX
137
Â
8 minutes 47 seconds
Timings are from identical virtual machines running on a server with lots of capacity. Database is the same on both (dump from production database), data is last x days of a variety of different manufacturers and models.
One of the consequences of the better code is that the XLSX exports are now usually not much slower than CSV exports. And for the extra short while, you get more organisation and summarising, plus no issues with non-ASCII data. This is still a problem when opening CSV in Excel as it refuses to default to assuming the UFT-8 encoding has been used.