Messaging system inside Fire

[Evgeny Kotelnikov]

Niklas Broberg suggested to add the ability to send a message to the students without having to either accept or reject. For example, a grader might request further information about the submission. It might be convenient in that case to do it inside Fire, and not through email.

[Víctor López Juan]

The way I would implement this is by adding a third button in addition to Accept and Reject, captioned f.ex. "Comment" or "Feedback". The color could be orange (halfway between green and red), and the icon could be an exclamation mark (as for "calling attention" to the student).

This button would send a comment to the student (just like with accept and reject), and allow them to resubmit either more comments, more files, or both. Submissions that have been commented on would not be considered as pending, and would show with an orange color and an exclamation mark icon in the interface.

I can think of two courses that would benefit from this feature.

• Dag Wedelin's Mathematical Modelling (DAT026): Students submit a solution to the assignment. Solutions are never "rejected". Instead, the student must read the feedback sent by the TA and submit a reflection explaining what they have learned. The course could use "reject" for this, but it sends the wrong message to the student.

• Peter Damaschke's Algorithms (TIN093): Assignments are not compulsory, and students are allowed to resubmit them as many times as they like. Very often the student submits a quite good solution, but they graders want to leave open the possibility of the student sending in a second version.

[Evgeny Kotelnikov]

I understood Niklas's request differently from these two use cases. I think his idea was to be able to ask for a clarification of the submission, but do not expect the students to change the files in the submission. Right now you would just ask the students by email, but it might be convenient to have the question and the answer in Fire.

If you want the students to change the files, you should reject the submission.

Dag Wedelin's Mathematical Modelling (DAT026)

To cater to this course I would rather create a lab option that disables binary grading. That is, if this option is on, instead of two buttons "Accept" and "Reject", there would be one button "Reply". It can co-exist with the "Comment" button, the difference would be that "Reply" enables the students to resubmit, whereas "Comment" doesn't. (They should probably be named differently though.)

Peter Damaschke's Algorithms (TIN093)

I think the solution here is simply to allow students to resubmit accepted solutions. We have discussed it.

[Víctor López Juan]

You're right, I hadn't considered that Niklas didn't want students to submit additional files with their comments.

I would still propose considering messages from students as submissions without files. Submissions can have an extra state (instead of accepted or rejected) which allows the student to submit more comments, but not more files.

Currently submissions can be in one of these three states:

• Pending: Grader needs to look at it
• Accepted: No one can/needs to do anything.
• Rejected: Student can resubmit if before final deadline.

We can extend the model with additional policies, for example:

• Pending: Grader needs to look at it. Student can withdraw.
• Accepted: No one can/needs to do anything. Submission shows green.
• Accepted but open to improvements: Student can resubmit. Submission shows green.
• Clarification required: Students can submit more information. Submission shows orange.
• Clarification required before final deadline: Students can submit more information if before final deadline. Submission shows orange. If the deadline has passed, the state shows as 'rejected' and red.
• Resubmission required before final deadline: Students can submit more information and/or files (if before final deadline). Submission shows orange. If the deadline has passed, the state shows as 'rejected' and red.

Each state (except for pending) corresponds to a button in the interface. Courses can choose, globally or on a per-lab basis, which of these states/buttons to enable. If we keep the button names consistent across fire instances then graders will know exactly what behaviour to expect.

States are implemented as classes with methods that return the colour, the name, whether the student can provide extra files and/or comments (taking into account deadlines and extensions), etc.

Notes:

• Every time a student submits, the submission switches to pending.
• If a student withdraws, the submission returns to the previous state.

---

Having a configurable range of submission states subsumes resubmitting accepted solutions, disabling binary grading, and messaging.

Something that would be nice is to give students a time limit to provide clarifications (e.g. 1 week). This could be handled as another type of deadline (final deadline for resubmissions, final deadline for clarifications), which can optionally be extended when replying to the student. I have not thought this through yet.

[Evgeny Kotelnikov]

This looks like a huge overkill to me. You assume that there are situations when the grader cannot accept a submission without some student comments about it, but also cannot reject it. It sounds like an extremely rare scenario, that isn't worth complicating the data model like that.

I don't see the problem in allowing students to resubmit accepted labs. To me it sounds like a simple and natural solution.

For the comments, I would simply suggest the following — detach comments from submissions and reviews into their own entity. So, a comment has an ID, an author(s), a text, a date, and a submission it is left on. When students make a submission with a comment, two things are added to the database. A grader can leave a comment, a decision (acceptance/rejection), or both. One or two things are added. Comments are displayed, sorted by time.

The only hard part about the comments is the interface and proper notifications, particularly on the graders side.

[Evgeny Kotelnikov]

Another thing. I think I know why Niklas suggested the messaging system — he used Fire to collect project proposals. For that, it makes a lot more sense to ask for clarification, rather than accept or reject. Whenever Fire is used for a course, a submission is a solution, that is almost always right or wrong, no discussion.

[Víctor López Juan]

The only hard part about the comments is the interface and proper notifications, particularly on the graders side.

If we handle comments as submissions/reviews, then we get this for free. But …

For the comments, I would simply suggest the following — detach > comments from submissions and reviews into their own entity.

If this will be easier to implement, sounds good to me. But in the GUI we can reuse the same text-field for both, just add an extra button for commenting.

We can the consider whether comments could have file attachments, but that's not strictly necessary.