Clone wiki

holy-gradle-plugins / HolyGradleCookbook

Read Me First

Gradle user home directory

The default location that will used for storing Gradle and the dependency cache will be under %USERPROFILE%/.gradle. This will probably be C:/Users/<user>/.gradle, so if you are short on space on your C drive then you may want to use a different location. The way to do this is to permanently set an environment variable GRADLE_USER_HOME to a location of your choice. You should arrange for this environment variable to be set automatically rather than setting it manually. If you are happy with the default location then don't set the environment variable.

Getting Started

My team is already using Gradle. What should I do?

  1. consider whether you want to change the Gradle user home directory
  2. since your project is already using Gradle you should have 'gw.bat' in your checkout - run gw tasks to see what tasks the build script supports
  3. read the documentation for the intrepid and devenv plugins

I want to retrieve components using a 'meta-package'

Some teams may publish meta-packages to facilitate retrieving releases. A meta-package is a zip file containing a Gradle Wrapper and a buildscript specifically for retrieving versioned components e.g. source-code repositories pinned at specific revisions and/or pre-built dependencies at specific versions.

  1. Unzip the meta-package to a location of your choice
  2. Open a command prompt at the unzipped location.
  3. Run gw fetchAllDependencies (or gw fAD for less typing)

My team hasn't adopted Gradle yet. What should I do?

If your team isn't already using Gradle, then you won't have a Gradle Wrapper in your project's source-control and you won't have any build scripts. You could of course download Gradle from their website but it is easier to use the custom gradle distribution.

Look for further instructions on your own company's wiki.

Mercurial

There's a problem when trying to retrieve a Mercurial repository

Make sure that you can successfully clone the repository using the command line, without it asking you to enter any credentials. In order for this to work you'll need to enable the 'mercurial keyring' extension. Do this by editing your 'mercurial.ini' and adding:

[extensions]
mercurial_keyring = 

Yes, it's correct that there's nothing on the right-hand-side of the equals.

I get the error 'abort: mercurial_keyring: http authorization required but program used in non-interactive mode' when fetching an Hg dependency

Make sure that your 'mercurial.ini' file contains something like:

[auth]
default.schemes = https
default.prefix = *
default.username = nm2501

Dependencies

See the symlinks DSL for defining symlinks from your workspace or consumer project to the dependent source-code module.

I want to use published pre-built artifacts for a module, instead of fetching the source-code dependency

The normal way to define a dependency on pre-built artifacts is to use the packedDependencies DSL.

However, if you're normally specifying that dependency within sourceDependencies then you want to quickly switch over to retrieving pre-built artifacts then see usePublishedVersion.

I need to use multiple versions of the same component

Assume that your module has dependencies as follows:

configurations {
    compileDebug
    compileRelease
}
packedDependencies {
    libA {
        dependency "company:libA:1.2"
        configuration "compileDebug->compileX64Debug", "compileRelease->compileX64Release" 
    }
    libB {
        dependency "company:libB:2.3"
        configuration "compileDebug->compileX64Debug", "compileRelease->compileX64Release" 
    }
}

and that the module descriptors for 'libA' and 'libB' both define dependencies on different versions of dllX. Let's say that your application is happy to load multiple different versions of the same DLL so you don't want to change any of the stated dependencies. If you attempt to fetchAllDependencies then Gradle will display an error message like this:

FAILURE: Build failed with an exception.
* What went wrong:
Could not resolve all dependencies for configuration ':example:compileDebug'.
> A conflict was found between the following modules:
   - company:dllX:1.0.0.2
   - company:dllX:1.0.0.1

This happens because the intrepid plugin has configured Gradle to throw an exception in the event of a conflict instead of using the default behaviour of choosing the latest of the conflicting versions. Gradle doesn't yet support the use case described above - to allow multiple different versions of the same module. However there are two ways to address this problem:

  1. Choose one version, by 'forcing' the use of a particular version. See here.
  2. Use both versions by introducing extra configurations. Continue reading...

By introducing extra configurations it is possible to circumvent the conflict detection in Gradle's dependency resolution. The work-around would look like this:

configurations {
    compileDebug
    compileRelease
    compileDebug_conflict
    compileRelease_conflict
}
packedDependencies {
    libA {
        dependency "company:libA:1.2"
        configuration "compileDebug->compileX64Debug", "compileRelease->compileX64Release" 
    }
    libB {
        dependency "company:libB:2.3"
        configuration "compileDebug_conflict->compileX64Debug", "compileRelease_conflict->compileX64Release" 
    }
}

As you can see, there are two new configurations ( compileDebug_conflict and compileRelease_conflict ) and they are used from one of the dependencies involved in the conflict. You might expect this work-around to have undesirable consequences in the published module descriptor - all those spurious _conflict configurations. However, the intrepid plugin has a convention to tidy up these unwanted configurations - any configurations matching <conf>_conflict<blah> will become simply <conf>.

The above example where a single module has dependencies on multiple versions of another module is somewhat artificial. This is because when you publish this module and the 'conflict' configuration get modified the published module will inevitably have conflicts. The published module simply won't be usable. Nevertheless the example might be representative of certain situations where you want to be able to build a module that has conflicting dependencies but you're not interested in publishing the results.

Here is another example of a situation where conflicts can exist, you want to resolve them /and/ you can still publish sensible artifacts. Let's say you have a multi-project workspace that you've configured using sourceDependencies. This workspace consists of 3 source-code modules 'a', 'b' and 'c'. However, 'b' and 'c' both have packedDependencies on different versions of 's'. Assume that 's' is a VC++ static library, 'b' and 'c' are DLLs and 'a' is an executable.

 a
 |- b
 |  |- s (version 1.1)
 |- c
 |  |- s (version 1.2) 

Even though different versions of 's' are used, there may be good reasons for *not* resolving the conflict but instead allowing 'b' and 'c' to use distinct versions of 's'. After all, once the static library has been linked into the DLLs, there may well be no run-time issues preventing the simultaneous use of multiple versions. And since the 'b' and 'c's dependencies on 's' are compile dependencies and not runtime dependencies, the module descriptors for 'b' and 'c' need not make any mention of 's'. Therefore there would be no conflict when retrieving the published artifacts for 'a', 'b' and 'c'.

I get a warning about using different versions of the same module. How do I find out what's including these versions?

You might see an error message such as:

Dependencies have been detected on different versions of the module 'Blah'. To prevent different versions of this module being unpacked to the same location, the version number will be appended to the path as 'Blah-<version>'. You can make this warning disappear by changing the name of the packed dependency yourself to include '<version>' somewhere within the name. For your information, here are the details of the affected dependencies:
  org:Blah:1.2.0 : (no version number) -> (includes version number)
  org:Blah:1.2.1 : (no version number) -> (includes version number)

or you might simply be interested to see how you depend on certain modules. To see the dependency graph you can use gw dependencies or gw <proj>:dependencies.

I want to try out a different version of one of my dependencies

You are using a specific version of a packed dependency e.g.

packedDependencies {
    "../Boost_vc10" {
        dependency "org:Boost_static:1.49.0"
        configuration "privateBuild->includes,libX64"
    }
}

This gives you a 'Boost_vc10' symlink in your workspace. However you want to try a different version of Boost and have the opportunity to easily change back if necessary.

Simply change the version number. If the new version defines a different set of configurations from the old version then you will need to determine which configurations to use. The easiest way to do this is to navigate to the dependency on your Artifactory server and have a look at the Ivy XML file.

Run fetchAllDependencies after saving the changes to the build script. It will fetch the new version if necessary and ensure that your symlinks are pointing to the right places.

If you choose to go back to the old version again then simply revert your changes to the build script and run fetchAllDependencies again. It will run very quickly this time because the old dependency is already cached on your machine and it only needs to set up the symlinks.

If you get a version conflict at this point it's worth trying gw fAD --refresh-dependencies.

I want to force a transitive dependency to a different version

You have a few options:

  1. Republish the parent module(s) so that it has the dependencies you want.
  2. If the module in question allows multiple different versions to be used within the same application then see I need to use multiple versions of the same component.
  3. Use Gradle's support for forcing dependency versions e.g.
configurations.all {
    resolutionStrategy {
        force 'org:my-module:1.2.3.4'
    }
}

This will force any usage of 'org:my-module' in the current project to be version '1.2.3.4'. If you want this behaviour to apply to all projects then you could wrap it in an allprojects block.

Bear in mind that 'forcing' a dependency adds configuration and makes things more complicated than they need to be. Where possible, the preferable option should be to modify dependencies so that no conflicts occur. But this can take time to achieve, so forcing the use of a specific version is a useful temporary fix.

Publishing

I need to have a source-code dependency converted to a published artifact dependency when I publish

That's easy - the sourceDependencies DSL specifies a publishing subsection which allows you to control how that source dependency will appear in the published Ivy XML module descriptor.

I have a third-party library that I would like to deploy to Artifactory

The easiest way to deploy artifacts to Artifactory is to write a build script (using the intrepid plugin) to package up and publish the dependency. Here is a basic example that you can use as a starting point:

buildscript {
    // TODO: have a look on your Artifactory server to see if more recent plugins are available.
    gplugins.use "intrepid:4.0.0.+"
    gplugins.use "my-credentials:2.0.0.+"
}
gplugins.apply()
 
configurations {
    // TODO: change the configurations as necessary. You may want to split things up into multiple configurations.
    // e.g. for different platforms (x86, x64), different compilers (VC9, VC10).
    bin
    // For future reference it's a good idea to publish the script that was used to publish the module.
    publishScript
}
 
packageArtifacts {
    // TODO: define what should be included and excluded in each package.
    bin {
        include "**/*/*.exe", "**/*/*.dll"
        include "info.txt"
        exclude "build"
    }
    publishScript.include "build.gradle"
}
 
publishPackages {
    // TODO: choose the group and version number. The module name isn't configured in this build script - it 
    // is actually controlled by the name of the directory in which this build script lives.
    group "org"
    nextVersionNumber "1.0.0"
    repositories.ivy {
        credentials {
            username my.username("Artifactory")
            password my.password("Artifactory")
        }
        // TODO: If you're publishing 3rd party artifacts then this is the correct repo to publish to.
        // But don't publish to this repo if you're publishing something developed within the organisation.
        url "http://artifactory-server/artifactory/externals-release-local/"
    }
}

To test the generation of the zip files run gw packageEverything. This will generate several zip files in a "packages" folder, which is useful for checking that things get generated correctly prior to publishing to the repository. When you're ready, run gw publish.

Plugin Development

What actually happens when I run a simple task like 'gw task'?

This is a rough explanation of what happens in Gradle and Holy Gradle plugins when you run a simple task such as 'gw task'.

  1. gw.bat is executed and it tries to determine which version of Java to use. If it can't find a suitable version of Java then it's a fatal error. If it can, then it uses Java to run the gradle wrapper jar file which lives in gradle/gradle-wrapper.jar.
  2. The Gradle Wrapper does some bootstrapping to ensure that Gradle is on your machine. gradle/gradle-wrapper.properties contains a URL to a Gradle distribution. If you haven't previously used this distribution then the Gradle Wrapper will download it to your Gradle user home directory and you'll see lots of dots while it does this.
  3. The Gradle Wrapper calls on to Gradle to run the requested task.
  4. Gradle will use the current working directory to determine which project(s) should be evaluated. If you're using a stand-alone project then Gradle will simply identify 'build.gradle' as your build script. If you're using a multi-project workspace then it's more complicated and depends on the presence of 'settings.gradle' files.
  5. Properties will be evaluated. These come from 'gradle.properties' files which can live in a number of locations. See here.
  6. Next, Gradle will look for any 'settings.gradle' scripts. These scripts do not contain configuration of individual projects but allow you to describe which projects to be included. This allows for multi-project workspaces.
  7. Now, for for each project in your multi-project workspace:
    1. Assuming that you're using a 'custom-gradle' distribution from Holy Gradle then the distribution will contain an init script. This is called 'holy-gradle-init.gradle' and can be found by running the task openInitScript. At this stage Gradle will invoke all init scripts. In the case of Holy Gradle, this results in the 'gplugins' extension being registered.
    2. Gradle will evaluate the buildscript block first. This is used for defining the dependencies of the buildscript itself e.g. repositories containing plugin, and version numbers for plugins etc. The 'gplugins' extension is really a helper to allow you to briefly define the version numbers of plugins that you want to use.
    3. When gplugins.apply() is called, all of the plugins you have previously specified will be applied. This invokes logic in each of the plugins, which typically adds further extensions and tasks. For example, the intrepid plugin would add a packageArtifacts extension.
    4. Now the build script itself is evaluated, and all of the extensions defined in plugins are available for use. This typically adds lots of configuration, but doesn't result in anything substantial happening. For example the packageArtifacts extension might now have a bunch of packages defined, but there wouldn't at this stage be any tasks which will perform the requested packaging.
    5. After the build script is evaluated Gradle will invoke any closures that have previously been registered using project.gradle.projectsEvaluated. Many of the Holy Gradle plugins use this mechanism in order to defer execution of logic until after the build script is evaluated. For example, at this stage the intrepid plugin can now iterate over all the packages that the build script configured in the packageArtifacts extension and configure tasks.
  8. At this stage all plugins have been applied, all project build scripts have been evaluated and all plugins have completed their project.gradle.projectsEvaluated work. Gradle now has a complete set of tasks and can now look at the original request 'gw task' and try to invoke a task. In this case, Gradle has defined a task named 'tasks' and since 'task' is an unambigous abbreviation, it's 'tasks' that is selected to be run.
  9. Gradle now invokes the task, first invoking any doFirst closures and then invoking doLast closures.

Other

I want to keep my password secret

It is recommended that you use the my-credentials plugin to store your credentials securely e.g.

repositories.ivy {
    credentials {
        username my.username("Artifactory")
        password my.password("Artifactory")
    }
    url "http://artifactory-server/artifactory/my-integration-repo/"
}

Although the my-credentials plugin stores the password securely on your machine, it is also a good idea to use Artifactory encrypted passwords to avoid plaintext passwords being included in HTTP requests. To get an encrypted password for Artifactory:

  1. Use a browser to navigate to your Artifactory server.
  2. Log in using your domain user-name and password.
  3. Click on your user-name at the top-right of the page.
  4. Enter your password and press 'Unlock'.
  5. Copy the contents of the Encrypted Password box.

I want to configure user-specific settings that aren't part of the build script in source control

There are two ways to specify user-specific properties that aren't part of the build script.

gradle.properties

You can keep machine-specific properties in your gradle.properties file, which is located at %USERPROFILE%/.gradle/gradle.properties or %GRADLE_USER_HOME%/gradle.properties if you have defined the GRADLE_USER_HOME environment variable. You can also create gradle.properties files in other locations in order to have different properties applied to different workspaces - see Gradle properties and system properties.

user.gradle

The Custom Gradle Distribution will automatically load user.gradle files that are in the same directory as an existing build script.

I want to configure the location of the unpacked dependency cache

The default location for the unpacked dependency cache is <Gradle user home dir>/unpackCache. Changing the Gradle user home directory will also affect the default location of the unpacked dependency cache, however if you want more specific control then that can be achieved with a project-level property i.e. a property that you define outside of any other DSL elements such as packedDependencies. For example:

unpackedDependenciesCache = "d:/dep_cache"
packedDependencies {
    foo {
        ...
    }
}

Since this specifies a machine-specific location you should probably avoid including the path in a build script that will be shared with other developers in the team. You may wish to use a user.gradle file or gradle.properties file to configure the path.

Updated