Wiki

Clone wiki

HG Dependency Group / Home

Hg Dependency Group

This is a Mercurial extension for managing groups of interdependent repositories.

Goals

  • Safely maintain (in source control) the relationships of several in-house and third-party libraries.
  • Avoid redundant copies of dependencies within projects.
  • Avoid package versioning issues by integrating module dependencies within source control operations.
  • Allow IDEs and makefiles to automatically determine the location of dependencies' source trees and perform co-builds.
  • Allow common users of Mercurial to benefit without needing to learn many (or any) new commands.

Background

A survey of existing solutions yielded many variations on two themes: mercurial extensions/wrappers and package managers. Although many of these solutions were rich with features and well implemented, I found them all to be wanting in one respect or another.

Mercurial Extensions and Wrappers

Notable Mercurial extensions are the built-in subrepos and the well-adopted Guest Repos extension. These solutions are attractive because the module relationships are safely tied to tracked files. Also, they enable a developer to act on the interdependent repositories as groups, carefully maintaining their interdependence. The clear benefit here is that a new team member (or a developer going back to an old project) can clone from a single source and have all that is necessary to recreate a release. An additional benefit is that a dependent repo can automatically find the source of its dependencies, which allows it more flexibility in how to build and link against the dependency.

These solutions use fixed mappings to clone subrepos to subpaths within the repo. For example, if repo A depends upon repos B and C, you could create the following dependency structure:

Simple Dependencies

The achilles heel with these solutions is that they carry along a lot of bagage when projects use many dependencies: they grow into a hierarchy of redundant dependencies. Take the above example, but assume that repo B also depends upon repo C. The resulting structure would something like:

Redundant Dependencies

Now repo C exists twice in the same project directory tree. Obviously, this can quickly get out of hand as more repos with interdepencies are added. The inefficiencies in file storage and source control operation time can be immense when these redundant repos are large.

There is another notable solution that falls into this family called Repoman. Repoman calls itself a "forest manager" and seems to attempt to solve some of the issues described above. However, the documentation and examples are scarce. Before digging too deep, I abandoned Repoman because it acts as a wrapper to Mercurial, which means I'd have to put aside all the the Mercurial commands I use on a regular basis and learn a new command set.

Package Managers

The other family of solutions includes package managers, such as Ant and Maven, which use strict versioning conventions and a central package store in which to find released versions. Build systems like CMake and Scons have their own conventions for package versioning and ways of finding packages on the system.

Package managers are very effective for dependencies on third-party libraries or modules that are well versioned. However, for in-house libraries that are being developed in tandem or code bases that have not been strictly versioned, package managers are not well suited. Though being disciplined in versioning is good practice, for some repos, maintaining versioning files is over-kill and redundant with the source control mechanisms already in place.

Using a package manager requires that the developer has separately pulled and built each dependency. In practice, this can be tedious with a large number of modules. It also imposes a level of decoupling that may not be desirable. For example, rather than linking against binary output from a release lib, I will frequently reference source or objects directly in the dependents makefile.

Perhaps package managers succeed in ideal cases, however in an enterprise setting with several developers maintaining several interdependent modules, the package paradigm lacks a certain measure of practical flexibility.

Updated