Build: Scalariform plugin repeatedly formatting sources

[Simon Leischnig]

The sbt-scalariform plugin currently puts a big overhead on performing sbt tasks. It seems the plugin formats all available source in opal when targetting a single sub-project. It looks like this:

[info] Formatting 1449 Scala sources {file:/home/simon/git/opal_hiwi/}BytecodeRepresentation(compile) ...
[info] Reformatted 1 Scala source {file:/home/simon/git/opal_hiwi/}BytecodeRepresentation(compile).
[info] Formatting 1449 Scala sources {file:/home/simon/git/opal_hiwi/}BytecodeInfrastructure(compile) ...
[info] Formatting 1449 Scala sources {file:/home/simon/git/opal_hiwi/}Incubation(compile) ...
[...]

[Simon Leischnig]

Finding 1:

[IJ]> BytecodeInfrastructure/compile:scalariformFormat::sourceDirectories
[info] * /home/simon/git/opal_hiwi/OPAL/bi/src/main/scala-2.11
[info] * /home/simon/git/opal_hiwi/OPAL/bi/src/main/scala
[info] * /home/simon/git/opal_hiwi/OPAL/bi/src/main/java
[info] * /home/simon/git/opal_hiwi

Subprojects seem to have the OPAL root folder included in the sourceDirectories dependency. The sbt-scalariform source code [1] shows that all folders in the project/config:scalariformFormat::sourceDirectories setting are recursively filtered and processed. Most likely the reason for this behavior.

[1] https://github.com/sbt/sbt-scalariform/blob/3b83dbe29f739ec593769dda19810ba6a3b6e751/src/main/scala/com/typesafe/sbt/SbtScalariform.scala#L107

[Simon Leischnig]

Finding 2:

The root directory is included deliberately: https://github.com/sbt/sbt-scalariform/blob/3b83dbe29f739ec593769dda19810ba6a3b6e751/src/main/scala/com/typesafe/sbt/SbtScalariform.scala#L102

Following a longer chain of changes to source directory inclusion in 2016 started here: https://github.com/sbt/sbt-scalariform/issues/32#issuecomment-176560768

breaking change: https://github.com/sbt/sbt-scalariform/commit/42ca0f04b5e7280afcf8eec896580f141be85891#diff-2fc01855ee416e041f017955ce0d3a65

Issue discussing and initiating this change: https://github.com/sbt/sbt-scalariform/issues/40

Argument: https://github.com/sbt/sbt-scalariform/issues/40#issuecomment-323111656

Rundown:

andrebeat commented on Aug 17
I'm not sure this is intended behavior but this breaks the subproject scoping, i.e. 
compiling a subproject formats all the files in the whole project.

godenji commented on Aug 17
@andrebeat

``inspect scalariformFormat::sourceDirectories``
From top-level projects shows that the project root is added to eligible 
sources. Subprojects inherit the parent setting, thus the subproject formats the entire project.

Presumably one wants to format all sources and not just those in current subproject. 
Formatted sources are cached so I don't see the current behavior as particularly problematic.

Feel free to open a new issue to track this though, should be a way to exclude the project 
root from sourceDirectories for subprojects.

I don't get the closing argument: "Presumably one wants to format all sources and not just those in current subproject" - how come

The caching is apparently 1. not working as expected or 2. taking too long and we get to see cache-validated files showing up in the total count so we don't notice; I won't look into that

[Simon Leischnig]

workaround PR: https://bitbucket.org/delors/opal/pull-requests/430

[Simon Leischnig]

One reason to include the project source directory seems to be that Scala by default accepts .scala files in the project root. This still doesn't mean that a subproject should include the build root into its sources list.

Edit: each subproject has its own scalariform cache locally, as it is scoped to the subproject task. Thus, cached files from other subproject's tasks are not visible. Getting pretty sure this should be changed in sbt-scalariform...

[Michael Eichberg]

Ok - thanks for the insights.... maybe you should raise an issue over there and tell them that this increases the build times by a very, very significant margin...

[Simon Leischnig]

ok - https://github.com/sbt/sbt-scalariform/issues/40#issuecomment-332893123

[Michael Eichberg]

Removing version: 0.9.0 (automated comment)