Missing data in Student_ and StudentEnrolment_

Brian Lewis repo owner

The PR above is a refactoring of validations ; especially on the Student page. The reason for failure to load has been validation failures that are reported reported back after the attempted process fails.

Now, the Student page is validated on Upload, rather than on Process.

Further, the validations are grouped by typed: number of occurrences, first row, last row. Row numbers here are actual row numbers in the source workbook (ie offset into the list + top row of list). this makes it easier to debug them.

Also, duplicated Student IDs are now reported in this preprocessing phase and while prevent proceeding to process. On the test data I had, it look like these may come from a school being merged in twice?

This same grouping has been applied to Staff validations as well - e.g. you will get just one row for any invalid Qualification code; rather than a validation record for each occurrence. This makes more sense because it will probably be fixed by adjusting the definition of the code, which will fix all instances at once.

Note that because the format of the returned data on ndoe upload is changed, you will need to deploy the database changes and code changes at the same time. for this reason I have deployed the database changes into pineapples_develop, but not into production fedemis or miemis.

As a smaller point, the uploaded file is now deleted from the FileStore if Upload is cancelled. Otherwise it just hangs around for no purpose.

2019-06-10T00:32:32+00:00

Comments (3)

Brian Lewis repo owner
- changed status to resolved
feat(census): refactor pof validations on student page - executed before upload

Resolves ~~#632~~

→ <<cset 6927567f635c>>
- 2019-06-10T00:19:54+00:00
Brian Lewis repo owner
The PR above is a refactoring of validations ; especially on the Student page. The reason for failure to load has been validation failures that are reported reported back after the attempted process fails.

Now, the Student page is validated on Upload, rather than on Process.

Further, the validations are grouped by typed: number of occurrences, first row, last row. Row numbers here are actual row numbers in the source workbook (ie offset into the list + top row of list). this makes it easier to debug them.

Also, duplicated Student IDs are now reported in this preprocessing phase and while prevent proceeding to process. On the test data I had, it look like these may come from a school being merged in twice?

This same grouping has been applied to Staff validations as well - e.g. you will get just one row for any invalid Qualification code; rather than a validation record for each occurrence. This makes more sense because it will probably be fixed by adjusting the definition of the code, which will fix all instances at once.

Note that because the format of the returned data on ndoe upload is changed, you will need to deploy the database changes and code changes at the same time. for this reason I have deployed the database changes into pineapples_develop, but not into production fedemis or miemis.

As a smaller point, the uploaded file is now deleted from the FileStore if Upload is cancelled. Otherwise it just hangs around for no purpose.
- 2019-06-10T00:32:32+00:00
Ghislain Hachey reporter
That’s good.

The row number to reflect the actual row was feedback during the workshop so happy to see that.

Duplicate due to merge was Yap and sorted during workshop.
- 2019-06-12T09:52:47+00:00
Log in to comment