Overview

Win64 Utility Build Scripts

These scripts provide an automated process for building various GNU and other tools for Windows. In many cases, the build is a simple configure; make process, but occasionally patches are needed. The patches are included in this repository, in Mercurial patch queue format.

Requirements

To build the utilities, the 64-bit mingw toolchain and an msys environment are required. It is quite possible that the tools will build under 32-bit mingw as well, but I have not tested this. The specific toolchain I used is:

  • x86_64-w64-mingw32-gcc-4.6.3-2-release-win64_rubenvb.7z

This is downloadable from http://mingw-w64.sourceforge.net/.

Using other versions of mingw in place of the reubenvb one causes build errors. It looks like the gnulib code redefines some stat #defines in a way that is not compatible with other versions.

The msys environment can be obtained from http://www.mingw.org - using mingw-get. The following packages are needed: * msys * msys-wget * msys-m4 for gmp * msys-perl for openssl configure * msys-flex and msys-bison for bc` * msys-vim for general editing (not essential)

There is a full msys bundle available from http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/ (file name msys+7za+wget+svn+git+mercurial+cvs-rev12.7z) but the Mercurial executable included has some issues (see below).

You need to set /mingw to point to the mingw install directory in <msys>/etc/fstab and run the msys shell using <msys>/bin/sh.exe --login -i

In addition, a working version of Mercurial is required. The simplest solution is to have Windows Mercurial installed and on your $PATH before starting the msys shell. The hg command will then be available in msys.

If you use the msys from external-binary-packages mentioned above, this includes a copy of Mercurial, which appears to be broken. To be honest I haven't bothered trying to work out why. The simplest solution is just to remove hg.exe from the msys installation and use the Windows copy as above.

The Windows PATH should be very clean before starting msys. In particular, having GNU tools like gnuwin32 in PATH can completely confuse the configure process. For my most recent build, I had only the following in $PATH:

C:\Windows\system32 C:\Windows C:\Windows\System32\Wbem C:\Program Files\Mercurial\ C:\Utils\Vim

And of thses, Vim is not needed (there is a vim provided in msys).

Build Details

The build scripts create static builds of the tools - all required libraries are linked directly in, producing executables with no external dependencies. This is a deliberate choice, even though I suspect that many people will object to the use of static builds. My main reason for taking this approach was that I wanted single file tools - no DLLs or supporting files to ship. This allows me to ship the exes with other build scripts, without any fear that I'll forget an essential DLL or dependency.

The usual objection to static builds is dependency management. What if a new release of a key library comes out with security fixes or something like that. My answer, to be honest, is that I don't really care! Until I started creating these builds, I was using Gnuwin32 builds, which are frankly ancient, and are going to be far more of a security risk than anything I build. Also the build process is straightforward enough that a rebuild when a new release comes out shouldn't be hard (and if it is, it will be for portability reasons that will affect a dynamic build just as much...)

For the same reason (single-file executables) I do not include internationalised messages (as these need the message catalogs to be shipped with the executable). It's not an issue for me, and again if someone wants to add the gettext library into the build process, it shouldn't be hard to do so. If you prefer dynamic builds, it shouldn't be too hard to modify the scripts.

Utilities

The following utilities are includes (all in the bin directory).

  • download - downloads the archives from the internet
  • unpack - unpack the archive, initialise the source directory as a Mercurial repository and extract the build patches as a patch queue
  • configure - configure the sources
  • build - build the binaries
  • makepatch - refresh the master patch directory from the current Mercurial patch queue
  • clean - reset the source directory to the newly-unpacked state

Building Notes

The Gnu Autoconf process checks for dependencies at configure time, and configures the code to disable features where the necessary dependencies are not present. It is therefore essential to build and install all prerequisites before configuring a package. I will create a proper list of dependencies in due course, but for now the following build order is what I have used:

  • libiconv
  • zlib
  • bzip2
  • gzip
  • xz
  • lzo
  • libxml2
  • gmp
  • mpfr
  • libidn
  • openssl
  • pcre
  • libarchive
  • sed
  • bc
  • wget
  • grep
  • diffutils
  • autoconf (for coreutils)
  • automake (for coreutils)
  • mingw-libgnurx
  • file
  • gawk
  • recutils

The packages with the most dependencies here are libarchive and wget. Most of the libraries are needed for one or the other of these.

In Progress

The following packages are in progress:

  • ncurses - experimental patche based on Eli Zaretskii's ezwinports build not sure it's working
  • readline - no patches needed but needs ncurses and seems to fail

Future Work

The following may be worth looking at:

  • Get more of coreutils working
  • findutils
  • gcal
  • patch
  • Python ports of some coreutils (df, du, maybe others)
  • Python pipe utils (joins, groups, things like that)

Possible Issues

If you use a copy of gcc other than version 4.6.3, gzip seems to fail because of issues with MSVC_INVALID_PARAMETER_HANDLING. This looks like a compatibility issue between gcc and the gzip (gnulib) package.

Data Files

The following utilities use specific runtime data files:

  • openssl - certificates named cert.pem or curl-ca-bundle.crt
  • wget - certificates as for openssl
  • wget - config file wget.ini
  • file - magic definitions magic.mgc

All of these are expected to be in the directory containing the executable. Note that other locations are checked (depending on the tool) but I consider those to be secondary for my usage.

The certificate filename curl-ca-bundle.crt for openssl is for compatibility with curl. However, I may remove this as I typically don't install curl in the same directory as these utilities.