Re-use CI system for automatic checks

[Daniel Schoepe]

Setting up a server for automatic tests is at the moment not trivial, as one needs to set up a webserver, worry about availability, persistent storage of incoming requests, isolating test runs from each other, and a host of other headaches.

An elegant solution would be to integrate with an existing CI system such as Travis[1]. The setup could be as follows:

• We run a Travis server somewhere, possibly alongside the Fire server
• Graders add a .travis.yml file + dependencies for each lab that describes how to run the tests
• When a student submits a solution, Fire consults the .travis.yml file and issues a corresponding request to the travis instance, and incorporates the result somehow.

This has several upsides over the current setup: - Travis handles proper storage of incoming requests, isolation of tests, etc. - Graders only have to know how to set up CI with travis instead of whatever homebrew systems people use at the moment. - We avoid reinventing the wheel.

[1] https://github.com/travis-ci/travis-ci

[Evgeny Kotelnikov]

I am fully onboard with this proposal. I would prefer to host a CI system separately from Fire. It should be possible to obtain another server from the Chalmers IT desk for that.

Do you feel like taking on this task and getting some teaching hours for it?

[Daniel Schoepe]

If I can get teaching hours for it, I'd be happy to take a shot at this.