Define collaboration principles

[Brian Lewis]

With multiple countries using the framework, each with specific requirements and education system parameters, we need to define a framework for collaboration between all developers working on these projects.

Goals of this collaboration :

• maximise reuse of software assets;

• enhance sustainability by increasing the number of people with experience of these systems;

• standardise on definitions and approaches to calculating indicators, so as to have comparable data between countries.

So we need to find the most efficient way to balance this reuse with the flexibility to customise on a country-by-country basis.

I think there is two approaches to this - not necessarily mutually exclusive :

1) use separate branches or forks for each country implementation.

2) account for local variations by "configuration"; in particular by the set-up of lookups and parameters.

2) is probably the more traditional method of maintaining a consistent source code base in "vertical-market" software. It's what has been done with pineapples desktop app, and, so far, there is the same source code for the Web site running in Kiribati and Sols.

So we could have:

• develop is the main thread of development. Changes in here are assumed to go everywhere, and are designed to be adaptable through configuration - primarily in data.

• develop is maintained through side branches related to specific development objectives, these are merged into develop when verified. We collapse all commits into a single commit when doing this for legibility.

• country specific branches can be maintained by branches directly from the Country-specific branch. Ideally , these should be compatible with the develop branch so they can be pulled back in to develop

• system releases for specific countries will pull develop into the country-specific branches (e.g. miemis) which can then be deployed to their target country. As far as possible, we keep the database schemas identical. Possibly, we could agree on a convention that schemas named for their system (e.g. miemis.xxx siemis.xxx) are not touched in any database schema synchronise.

As far as possible, we keep functionality in develop. If features aren't required in a given context, they can be switched off through the menu. (#52). I believe this will make it easier to support regional initiatives ( like the Perf Assess framework) , and leverage knowledge from one country to another.

I'd see the main use for customization in the country-branches as relating to style sheets , or html templates. For example, a page displaying school attributes may need to be different to account for particular data points - but even in that case, I think it would be good for any fields to be included universally.

[Brian Lewis]

For reference: from Ghislain:

OK. Brian recommends to "The Charts and Tables menu options we should remove for now" and I agree. I have not heard how you guys plan to manage the evolution of the software for multiple countries but James and I agreed a couple of weeks ago that having branches for each country with a generic stable master would be an ideal approach. Of course, this is still up for discussion but assuming this I would personally propose experimental unstable features such as "Schools/chart|table" and "Teachers/charts|table" should be completely removed from master and worked on their individual branches until tested thoroughly to work before being part of the Pacific EMIS offering. The branches could be branches forked from country specific branches (say a country is pioneering this framework with potential for Pacific adoption) or just generic branches from master. Until a consensus is reached I think this remains a major user bug.

[Ghislain Hachey]

All sounds like a good starting point. However, one small thing. If develop is to go everywhere then it must be consider a very stable and tested branch, the one we make releases from. If that is the case should it not be better named like 'stable' or 'master' (is often used in open source projects). I understand this is pushing towards the limit of personal taste so my take on this would be more about how well develop would be handled then the name itself.

Also, to add to the discussion, in develop (or stable or master,) whatever will be the most stable branch, there should contain a clean release/log history that could be generated from clean squashed commits from PRs. Those squashed commit should follow a certain convention, I recommend the same one the AngularJS team follows described here https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format

[Ghislain Hachey]

ok. yes, this makes sense. To make it simpler (and I think this may be what you suggested) country branches are merely placeholders for their releases. A note about configuration, as a system administrator I'm used to configuration of software through text files (plain, json, etc.), this has been the unix way for decades. You suggest adaptable through configuration in data, do you mean we need to access database directly just to configure country software like it was suggested to enable/disable menu items? This seems a bit daunting say for an administrator of systems. Could we work towards having a plain configuration file that could be easily edited and the system would have a script or something equivalent to make sure the configuration does the database changes required? That config file would essentially be the only committed code to country branches which would otherwise only pull new features in an ad-hoc manner from stable (which you call develop). Does that make sense?

[Ghislain Hachey]

Code formatting: There is currently none and lots of inconsistencies:

• C# is at 4 spaces, I think this is the accepted convention. Just minor cleanups across code neededed.
• JS is at 4 spaces but a bit messy formatting at places. I propose to follow Crawford, an authority on JS http://javascript.crockford.com/code.html. So leave at 4 spaces and slowly cleanup formatting as code is edited (though it is understood that major new code undertaking will be in TS)
• TS is at 2 spaces. Personally have usually always preferred 2 spaces for JS (and TS it seems) as I prefer shorter succinct lines of code.
• CSS is messy. Needs major cleanup. I propose 2 spaces and moving all app styles into the same location. No more inline CSS, all in CSS files. I propose it goes in src/apps/css as most of it seems there already.

Anyway, would be nice to hear what others think and prefer. Once it is agreed maybe should all set VS to the accepted convention and pass a format document on code before committing.

[Ghislain Hachey]

File/Directory Structure: Would be nice to have the compiled/concatenated TS/JS/CSS into a build/ (or dist/) directory and not at the root. My first impression when going through source code was that css/js was all over the place. Then, if I understand this correctly the ASP.NET takes it from there to minimize and compress everything going to browser.

[Brian Lewis]

C# - 4 spaces

JS - as you say, will be less of an issue over time as we migrate out. But I'm with you - I'd prefer to go with 2 spaces

TS - 2 spaces agreed.

CSS - agreed, 2 spaces, extracted into files.

Other possible linting conventions for TS/JS and C# -

-- no more than 1 consecutive empty line

-- no trailing spaces

These also seem to be common usages: TS/Js {} start on same line

e.g.

if ( var === 0) {

}

c# is generally like this:

if (myVar == 0)
{
...
}

palantir have a typescript linter ts-lint

I will raise issues to add lint support for typescript , javeascript, css and C3 into the project.

[Ghislain Hachey]

NOTE - Not currently using the process below but simply branching and merging everything so keep micro-commits as clean as possible.

Following from #60

Just putting this here as rough development process notes for myself.

create issueX branch from develop (fix, feature, new docs, etc.)

1. work on issueX locally with many work in progress commits until ready (eventually this should pass existing tests, linting, and mandatory additional tests for the fix/feature)
2. Rebase and squash commits into one clean message following AngularJS convention (or other established convention useful to extract a proper history/change log)
3. If any, pull latest from develop branch, merge it into the above and resolve any conflict
4. Push branch to origin with same name (push issueX to origin/issueX)
5. Create PR on Bitbucket
6. Evaluate PR
7. If Approved, merge into develop
8. If need change, go back to 3. when the rebase and squashing was done (and before the pull of any latest in develop if that took place). Do any changes through any number of commits. Rebase again into a single commit (previous rebase point combined with needed changes). Continue again from 4. and repeat until PR is Accepted or Denied.

Maybe once this process is smooth enough we should have something a bit more sophisticated. Master branch to hold production ready code with tagged releases. develop branch remains main development thread as it is now but forking release branches only accepting bug fixes, docs, etc. until enough feature for a release while others could continue with more development outside of stable release branches. Merge release branches to master when ready and tag it, merge also in develop where it all continues.

[Ghislain Hachey]

Reference: Some more discussions on development workflow at the end of issue #60

[Ghislain Hachey]

I see latest merges include all micro-commits. So those where not merged with --squash commits or similar.

Also, I notice that the issues were not automatically resolved when merging. I think the issues are automatically resolved if "Closes #X" is included in a commit message but not in a merge message.

[Ghislain Hachey]

I'm just learning that JIRA has to final state: closed and resolved. See Link Text and will need to discuss this as part of development workflow. For now, the Closes #X in the issue closes the issue with CLOSED meaning this is fixed and likely never have to come back to it. RESOLVED seems to be used for I think I've fix it, but maybe we just use CLOSED for now and make no distinction?!

[Ghislain Hachey]

A Note on merges: I think when we do the actual merge we should not use the commit message convention (adopted from AngularJS team). The merge message should just be something like "Merge issue65 branch".

[Ghislain Hachey]

Much of this already documented in developer guide. The collaboration could certainly still be improved. I propose further discussions in Slask in groups and once agreed documented in developer guide for posterity and reference.