1. metrumrg Avatar metrumrg
  2. project
  3. nmqual

Wiki

Clone wiki

nmqual / UserGuide

View History

Instructions for using NMQual versions 6 and 7. See also NMQual_8.

Contents

NMQual is a tool for automating and documenting the installation of NONMEM®. This guide describes the normal use of NMQual, version 6, with notes about NMQual 7. NMQual is available under the GNU General Public License from Metrum Institute. NONMEM® is available under license from Globomax, LLC. Instruction is given here on how to obtain NMQual, but we assume you already have a Globomax CD with NONMEM® version V, level 1.1 (15 April 2004 or 18 March 2005) or version VI, level 1.0 (27 October 2006).

System Requirements

NMQual requires the presence of

  • Perl 5.8 or greater, such as !ActivePerl , freely available from !ActiveState.com for most platforms
  • the Perl package XML-XPath 1.13 or greater, freely available from www.cpan.org;
  • the Perl package Archive-Zip if using NMQual 7;
  • and a Fortran-77 compliant compiler, such as GNU Fortran 2.95 or greater, freely available from various sources based on code at gcc.gnu.org.

NMQual 6 has been tested with the following operating system(s) and Fortran compiler(s):

  • WindowsXP with GNU Fortran (g77) version gcc-2.95 for Windows;
  • WindowsXP with Compaq Visual FORTRAN version 6.6(C) for Windows (versions down to 5.x not tested but likely compatible);
  • Linux kernel 2.6 with GNU Fortran version gcc-3.4.6 for Linux.
  • Mac OS 10.4 with GNU Fortran version gcc-3.4.3 for Darwin.

NMQual 7 has been tested on Mac and Windows with both the g95 compiler and the Intel Fortran compiler. config.template.xml in NMQual 7 is pre-configured to install NONMEM 7 using Intel Fortran under Windows.

If g77 for Windows is used, make the two recommended permanent changes to envrironment variables (PATH, LIBRARY_PATH). See http://members.tripod.com/kkourakis/ for details. We provide a value-added redistribution at http://metruminstitute.org/downloads/g77. On Linux, the 'f77' executable is probably already in the 'PATH' environment variable. For Mac, there are several ports of Fortran77 compilers, like GNU Fortran from fink.sourceforge.net and hpc.sourceforge.net.

NMQual 6 was written to support the installation and qualification of NONMEM® version V, level 1.1, as it appears on the Globomax distribution CD-ROM dated 15 April 2004 or 18 March 2005; or version VI (6.2.0) as it appears on the CD-ROM dated 8 May 2008. Modifications to NMQual may be necessary if you are using NONMEM® distribution media generated on a different date. Customized installation is available as a fee-based service (contact nmqual-services@metrumrg.com).

NMQual 7 was designed to install NONMEM 7.1.0 (31 August 2009) or NONMEM 6.2.0. However, the file config.template.xml gives advice only for NONMEM 7 installations. If you need advice on making NONMEM 6 config files, see config.template.xml in NMQual 6.4.2.

Obtaining NMQual

NMQual is available at http://code.google.com/p/nmqual/.

Verifying the Distribution

You may want to verify the checksum for NMQual. Choose the downloads tab, click on an entry in the Summary column, and note the SHA1 checksum. Compare to the SHA1 checksum of your download (details are OS-dependent).

Installing NMQual

Make sure you have Perl 5.8 or greater installed. Type perl -V at a command prompt to diagnose.

Make sure you have XML-XPath installed. Type perl -e "use XML::XPath". Silence indicates success. Consult your system administrator if necessary. For !ActivePerl's PPM utility, type 'ppm' to get a prompt (full path may be necessary); then type install XML-XPath. Consult PPM's documentation for further help as necessary, or try Nick Holford's [PpmHelp instructions]. Newer versions of !ActivePerl have a GUI version of PPM that is much harder (!) to use: try View | All packages; then right-click on XML-XPath and check the box for install, then click on the little green arrow in the tool bar to install marked packages. Steve Chapel points indicates: you can by-pass the GUI by giving a complete command line thus: "ppm install XML-XPath".

For NMQual 7, make sure you have Archive-Zip installed. Type perl -e "use Archive::Zip". Silence indicates success. Consult your system administrator if necessary. For !ActivePerl's PPM utility, type 'ppm' to get a prompt (full path may be necessary); then type install Archive-Zip. Consult PPM's documentation for further help as necessary.

Make sure you have a Fortran77 compiler installed. Type, e.g., g77 -v at the command prompt. Windows users that need g77 can get it [WindowsFortran here].

To Make NONMEM 7 installations, you will need a Fortran 95 compliant compiler.

Make sure you have appropriate account permissions for your intended install locations (NMQual and NONMEM). Consult your system administrator if necessary.

  1. Unzip the nmqual zip file with appropriate decompression software. Your operating system may have a built-in decompression utility. Extract the directory nmqual to an appropriate directory, such as C:\nmqual\ (on Windows). Note: the full path can be nested arbitrarily deep, such as C:\metrum\downloads\utilities\. In our experience, g77 2.95 does not work with paths that contain spaces.
  2. Customize the configuration file (config.template.xml). At very least, you will probably need to edit the two directory specifications (source, target). Instructions and examples are provided. By default, config.template.xml is set up for WindowsXP using the g77 compiler, with NONMEM5in the d:\ drive, installing to c:\nmv.
  3. Did you break something when you edited the config file? Go to a validator and check your configuration file (XML Instance) against 'config.xsd' (XML Schema). If validation is successful, you have a better chance of getting good results when you run NMQual. If validation fails, it is likely that your configuration file is corrupted. The validator should give some hints.
  4. The vendor-supplied mechanism for installing NONMEM allows the user to adjust certain operating parameters at install time (i.e., edits to the SIZES file(s)). NMQual reproduces this facility through 'options' files. You may want to customize and rename changes.options.nm5.std.xml (or equivalent) if it's one of the change files mentioned in your configuration file. Pre-written customizations may be available, e.g. changes.options.nm5.big.xml. These files control the sizes of NONMEM® items. changes.options.nm5.std.xml gives Metrum Institute's 'standard' sizes, and .big. gives Metrum Institute's 'big' sizes. Both give NONMEM installations with sizes different from the defaults. hanges.options.nm6.std.xml contains non-default settings (bigger buffer sizes) for NONMEM6.
  5. You may want to edit the shebang line (#!/usr/bin/env perl) in each of nmqual/*.pl, if this matters for your system. Often it won't, especially if you are on Windows or use the recommended convention of invoking perl scripts as arguments to perl itself, e.g., perl nmqual.pl [...].
  6. For NMQual 7 installing NONMEM 7, you may need to change (in your config file) the path to the license file. The license file on the disk is likely out-of-date.
  7. For NMQual 7 installing NONMEM 7, you don't need the 'file' element that copies unzip.exe if you are not on a Windows system.

Upgrading NMQual

Periodically, new versions of NMQual are released. One upgrade strategy follows. 1. Install NMQual in a versioned directory, such as c:/nmqual-6.4.1/. 2. When a new version is released, install it also in a versioned directory, such as c:/nmqual-6.4.2. 3. Copy your config files from the prior NMQual directory to the current NMQual directory. 4. Edit your config file(s): change the target directory name(s) (optional, but recommended). 5. Run NMQual (see below) using the new version and the edited config file(s).

Running NMQual

  1. Make sure your NONMEM® files are available at the location specified in your configuration file.
  2. Open a command window (e.g. start->run->cmd on Windows).
  3. Navigate to the directory 'nmqual'.
  4. Execute nmqual.pl, specifying a configuration file, e.g., at the prompt, perl nmqual.pl config.template.xml. If you omit the config file, nmqual.pl will display usage advice.
  5. You can run NMQual repeatedly with different configuration files, to create different versions of NONMEM® on your system. (You can also tweak a config file and re-run it immediately.) Each completed NONMEM® installation is independent of other NONMEM installations and independent of the NMQual installation. The last subdirectory in each target directory must be unique, since NONMEM is invoked by a script bearing the same name (created for you by NMQual).
  6. After a successful run, you may want to add the relevant path to your system path variable, and alert other potential users to do the same. On Linux and Mac, you will probably need to add a line to your shell's profile script, such as export PATH=/opt/nmv:$PATH. Consult your system administrator if necessary. On Windows, right-click on My Computer, then choose Properties> Advanced>Environment Variables>User (or System) variables>PATH>Edit. Prepend e.g. 'C:\nmv;' to whatever is listed in the edit box. Close windows appropriately. On most systems, you will need to open a new command window, maybe even reboot (Windows), for path changes to take effect.

Prompts and Errors

Many prompts occur in supervised mode, fewer in unsupervised mode. Only the more noteworthy prompts are mentioned here. Errors should give some helpful details on command line when they occur.

Almost all errors are fatal: nmqual.pl should 'die' if it (or a secondary script) can't do what it was supposed to do.

  • License: before initiating, nmqual.pl displays license information. In pause mode, hit enter to continue. In no-pause mode, the message displays for about 5 seconds before execution continues.
  • configuration file: the configuration file will be copied to the installation and be renamed config.xml. The NONMEM® wrapper and checksum.p will consult it during normal operation.
  • "... is compromised": nmqual.pl by default verifies the contents of files and directories specified in the config file. This information is used at runtime to show that the installation has not been compromised.
  • creating installation log: this is the last thing to happen before nmqual.pl terminates. If any error occurs prior to this point, nmqual.pl will 'die' and no nmqual log will be created. Thus, the presence of a log is evidence supporting the conclusion that installation was successful.
  • "SUCCESS": if you don't see this message, the installation should be considered compromised.
  • "ERROR: system test command..." : your config file has a test element that requires nmqual.pl to try something on the command line, possibly with a predetermined result. The command failed, or the result was not as expected. Try the command manually. If the result is correct, change the test element in the config file so that it tests for the correct results. Or drop the test element altogether. It looks like this: <test class='system' command='some command' ... /> .
  • "ERROR: environment variable..." : your config file has a test element that requires nmqual.pl to search for an environment variable in your operating system. The variable was not found, or did not have the required value (if specified). Search for the variable manually. If present, and its value is correct, change the test element in the config file to correspond. Or drop the test element altogether. It looks like this: <test class='environment' variable='some variable' .../>.

Editing Change Logs

Editing the change logs is not terribly difficult, but requires precision, and some appreciation of the XML file format. You can add a change by cloning and editing an existing change, i.e., everything from a 'change' tag to its corresponding 'end change' tag, inclusive. Change other fields as appropriate, e.g. identify the file you want changed. In the 'from' field specify some text to find, and in the 'to' field, specify some replacement text. Preserve the line C ATTRIBUTION exactly as in other instances: NMQual will replace this with an installation-specific comment ('attribution' in the config file) and timestamp.

There is a bit of an art to selecting text for the 'from' field. First, you'll get better results if you copy text directly from a Fortran file. The match must be exact, and hand-typing can result in subtle errors. Second, it is useful to begin the selection with a return character (start highlighting at the end of the previous line) so that the Fortran code lines up nicely in the XML file. We generally don't include the final return character before the 'end from' tag. Third, be sure to include some static 'bounding' text before AND after a desired point of insertion. If you do, the 'from' text won't exist anymore as such, after the change is complete. If you don't, scripts may have a hard time later trying to tell if the change was made or not (scripts are not as smart as you).

After you've mangled one of the standard changes*.xml files, or written one of your own, you may want to verify that your file format is still what NMQual is expecting. First, the name of the change file must be listed in your config file, e.g.<file type='change'>my.changes.xml</file>. Second, the file must still be valid XML: matching tags and all that. Many web browsers will try to parse your XML file, and will show you any format errors. Third, the file must meet the content model that NMQual expects. For instance, the root element must be 'changes'. The file changes.nmqual-6.xsd (a 'schema'), completely specifies the content model for changes*.xml files. Submit changes.xsd and any homemade changes*.xml file to an online validator. The validator will tell you whether your XML file meets NMQual's content expectations, and will probably show you where the problems are, if any.

Some bug fixes will require no editing on your part. Metrum Institute may periodically release new changes files to reflect bug fixes recently identified by Globomax. Watch for updates to nmqual at http://code.google/p/nmqual/.

Running Control Tests

NMQual makes it easier to qualify a NONMEM® installation. It provides a tool called runtest.pl which compares the output of control tests to expected results. Several control tests and associated reference files are provided by NMQual. You can extend the usability of runtest.pl (i.e., add more control tests) by carefully imitating existing file structure. More on that later.

Currently, NMQual will implement 6 test cases. Tests 3 - 7 are the example control streams (CONTROL3 - CONTROL7) provided in the NONMEM® distribution media. Test 8 is a user-defined test case, based on simulated data from a published population pharmacokinetic model describing the disposition of parent and metabolite plasma and urine concentration data. It is provided as an example.Users who wish to contribute test cases to the NMQual project may send suggestions to nmqual-comment@metruminstitute.org.

After a successful install, you should run the control tests to see if NONMEM® is operating as expected. Navigate to the test directory of the NONMEM® installation , e.g. c:/nmv/test. Execute perl runtest.pl 3 4 5 6 7 8. Expect 7 and 8 to take a while. runtest.pl uses the argument list as file name stems for control tests, and generates/guesses the file names as necessary. For instance, in the example given, it will look for 3.ctl, 4.ctl, etc. (control streams) and pass them to the NONMEM® executable. After the run, it will look at the results and compare them to control3.ref.xml, control4.ref.xml, etc., and log the results.

The reference values for NONMEM 7 were generated with Intel Fortran on Windows.

Try this to make a new control test. Windows code is given, but the Linux equivalent should be obvious. 1. Navigate to: c:/nmv/test/. 2. Execute: copy 3.ctl 9.ctl. 3. Execute: copy PARAMS3.for PARAMS9.for. 4. Edit 9.ctl: change PARAMS3.for to PARAMS9.for. 5. Edit PARAMS9.for: change control3.test.xml to control9.test.xml. 6. Customize modeling code in 9.ctl and PARAMS9.for (or do nothing, for the sake of demonstration only). 7. Execute: perl runtest.pl 9. runtest.pl will complain that it cannot find control9.ref.xml. We haven't made control9.ref.xml yet, because we don't necessarily know the expected result of your code modifications above. However, if NONMEM® were functioning properly, newly created control9.test.xml should look pretty much like our ideal control9.ref.xml. So... 8. Execute: rename control9.test.xml control9.ref.xml. 9. Execute: perl runtest.pl 9. Now runtest.p finds what it wants. Of course, this example is circular. Ideally you would study 9.lst to decide if the content of control9.test.xml really does merit use as a reference file.

Running NONMEM

If you tried the control tests above after installation, NONMEM® was run for you by runtest.pl. To run NONMEM® yourself, simply invoke the script (same name as your install directory), supplying the name of a control stream and a name for an output file. Navigate to the test subdirectory and...

  1. Execute: perl -S nmvi.pl 3.ctl 3.lst. Your results are in 3.lst. nmvi.pl (or whatever yours is called) appends the installation log to the end of the output file; then it appends any runtest logs it finds. -S tells perl to search for nmvi.pl in directories listed in the path variable. If you didn't change the path variable on installation, you may have to specify the location of nmvi.pl on the command line.
  2. Optionally execute: perl -S nmv.pl 3.ctl 3.lst nochecksum. Same result, but nmvi.pl doesn't bother to run the checksums. This is handy if you've tinkered with the Fortran code, because a bad checksum would cause nmvi.pl to fail. You can run checksum.pl at anytime: perl checksum.pl c:/nmvi (c:/nmvi is your install directory). checksum.pl will calculate checksums (for items listed in the configuration file) and either create a checksum log or compare to a log if one exists. Caution: don't use the nochecksum option for qualified, production NONMEM® runs unless your qualification procedures specifically call for it.
  3. As of NMQual 6.4.0, compile-only and execute-only are suppported. The character c or e may precede the input file argument, e.g. perl -S nmvi.pl c 3.ctl 3.lst. For c, nonmem.exe is compiled, and then the script stops. For e, nonmem.exe is assumed to be present; it is executed, and normal output is generated. This feature allows, e.g., verification of successful batch compilation before committing to execution.

Directory and File Structure

The directory structure of NONMEM® installed by NMQual is similar to that created by a manual installation. The major difference is the addition of a test directory. The test directory contains the NMQual utilities, and is where the NONMEM® executable actually gets created. Some files are present on a configuration-dependent basis. Most are copied from the 'nmqual' directory.

  • 3.ctl (*.ctl): control streams for standard control tests.
  • changes.bugs.nm5.xml: bug fixes for NONMEM5 catalogued by Globomax, plus a few approved by NMQual developers. These latter are not necessarily considered 'bugs' by Globomax, but many find them useful, and they have been discussed with Globomax representatives. As for any fix, you may delete the non-official fixes from changes.bugs.nm5.xml (near the end; see comment there).
  • changes.bugs.nm6.xml: Like changes.bugs.nm5.xml, but for NONMEM6.
  • changes.g77.nm[x].xml: Fortran alterations needed when using the g77 compiler.
  • changes.nix.nm[x].xml: Fortran alterations needed when using Linux/Unix/Mac.
  • changes.ieee.nm[x].xml: Fortran alterations to switch to IEEE machine constants, e.g. for PowerPC Mac.
  • changes.nmqual-6.xsd: (see nmqual directory) a W3C-style XML Schema, specifying the content model for changes*.xml files. Submit this file and any homemade changes*.xml file to an online validator such as those mentioned above. This will tell you whether your XML file meets NMQual's content expectations.
  • checksum[timestamp].xml: initial directory checksums after installation.
  • config.xml: (a copy of) your configuration file; guides installation and operation.
  • config.nmqual-6.xsd: (see nmqual directory) a W3C-style XML Schema, specifying the content model for config*.xml files. Submit this file and any config*.xml file to an online validator such as those mentioned above. This will tell you whether your XML file meets NMQual's content expectations.
  • control3.ref.xml (control*.ref.xml): expected results from 3.ctl (*.ctl).
  • DATA3: used by 3.ctl.
  • data8.csv: used by 8.ctl.
  • nmqual.pm: nmqual perl package; routines used by nmqual perl scripts.
  • nmvi.pl (or similar): a script that invokes NONMEM®; always has the same name as the install directory.
  • nmqual_install.[timestamp].xml: an XML log giving many details about the original installation event.
  • a version identifier (a date)
  • copyright information
  • time of installation to nearest minute
  • the script name and arugments
  • compiler details
  • some environment variables
  • a copy of the data in checksum.xml
  • a summary of attempted and confirmed code changes.
  • PARAMS3 (PARAMS*): called by 3.ctl (*.ctl).
  • runtest.pl: a script that runs control tests and evaluates the results using reference files.
  • THEO: data file used by some control tests.
  • THEOPP: data file used by some control tests.
  • nm7control.zip: an archive of control test files for NMQual 7 with NONMEM 7. For completeness, analogs are provided for NONMEM 5 and NONMEM 6.

Similarity

The script runtest.pl compares NONMEM output to copies of accepted output for appropriately formatted control tests. The control tests consist of a control stream, a PARAMS file that creates test output, and a reference file that contains accepted output. The script runs the control stream, and then makes comparisons for each test/reference value pair. 'Status' and 'similarity' are calculated for each comparison. Results are stored as runtest.[timestamp].xml. Reference values for NONMEM 7 were created using Intel Fortran on Windows.

Behavior depends on whether the reference value is text, a nonzero number, or zero. Text is evaluated by string comparison: status is either "match" or "mismatch" and similarity is either 99 or -99. A zero invokes a numeric comparison: status is either "match" or "mismatch" and similarity is either 99 or -99. A non-zero numeric reference value invokes a numeric comparison: similarity is calculated as (-1)* log10(abs((ref-test)/ref)). Status will be set to "mismatch" for similarity <=-99; "match" for similarity >=99; and "partial match" for all other values. Very informally, similarity of "3" means that the reference value and the test value are the same to 3 significant digits.

At the end of runtest.pl, a summary table is given on the command line where rows correspond to analyses in a batch. Column "test" names the analysis; "mis" gives the number of mismatches for that analysis; "exact" gives the number of exact matches; "part" gives the number of partial matches; and "min" gives the minimum similarity across all partial matches.

NMQual and XML

XML is a presentation-neutral data storage format that promotes interoperability. XML is designed to be machine readable as well as somewhat human readable. All logs created by NMQual scripts are encoded as valid XML: nmqual*.xml, checksum*.xml, and runtest*.xml. Additionally, bug fixes and options are encoded as XML for use by NMQual during installation: changes*.xml. The configuration file is also XML. Even the NONMEM® output is repackaged as XML. An XML-aware browser like Mozilla Firefox is a convenient tool for viewing native XML files. Online web apps are convenient ways to test an XML file against a W3C Schema content model (usually *.xsd). Here's some we've tried. You can also do an internet search on something like 'XML schema validator'.

Administrative Acceptance

The utility accept.pl provides a mechanism to document administrative acceptance of an installation. It accepts two arguments: the path to an installation and (optionally) a string of text (presumably an administrative statement). The executor must have 'write' permission for the installation log. The user login (as defined by perl), a time stamp, and the text string (if present) are embedded in the installation log as items "acceptor", "time", and "statement" (respectively) under item "acceptance". For installations that have already been accepted, no changes are made and the conditions of acceptance are reported on the command line. Since the NONMEM invocation script attaches the installation log to all run output, acceptance information is propagated.

Additional Help

See FreqQuestions.

NONMEM® is a registered trademark of the University of California and is licensed exclusively to Icon Development Solutions.

Updated