Commits

Johannes Köster committed 563dfe5

Edited online

  • Participants
  • Parent commits 0860d63

Comments (0)

Files changed (1)

 [Snakemake Workflow Archive](Workflow Archive)
 
 # Latest News #
-* **26 Mar 2013:** Release 2.3 of Snakemake. The parser has been rewritten completely to increase the flexibility. The first yield is support for nesting rules into conditional statements and for-loops (see [here](https://bitbucket.org/johanneskoester/snakemake/src/master/tests/test_conditional/Snakefile) for an example). Second, the new locking mechanism has been improved to only report a lock when a running workflow uses conflicting files in the same directory. Further, incomplete files should be reported less often since cases where the file was correctly deleted by Snakemake after a failing job are now omitted. Finally some bugs were fixed and special thanks go to Marcel Martin and Hyeshik Chang for contributing patches.
+
+* **17 Jun 2013:** Release 2.4 of Snakemake. **The scheduling mechanism has been completely rewritten** based on a heuristic greedy algorithm that approximates a multicriterial knapsack problem. Besides speed improvements, this allows the user to **specify arbitrary resources in addition to threads, e.g. gpu or io**. The scheduler will ensure that they are not exceeded. A new flag **--ruledag allows you to print a reduced DAG** where similar jobs are collapsed to single nodes. This should help visualizing large workflows. *Minor improvements:* The output hase been extended to display counts of jobs to be executed. Snakemake now prints debugging output when invoked with --debug. Fixed a typo preventing the use of multiprocessing instead of threading in some cases. This should result in increased performance for workflows relying heavily on rules with inline python code. Params are now tracked for changes similar to input and code. If you change your params definition, --summary will notice you that params for the corresponding files have changed.
 
 # Installation #
 * On Ubuntu 12.04, you can install the Debian package python3-snakemake available in our [launchpad repository](https://launchpad.net/~genomeinformatics/+archive/bioinformatics-software).
 
 # Old News #
 
+* **26 Mar 2013:** Release 2.3 of Snakemake. The parser has been rewritten completely to increase the flexibility. The first yield is support for nesting rules into conditional statements and for-loops (see [here](https://bitbucket.org/johanneskoester/snakemake/src/master/tests/test_conditional/Snakefile) for an example). Second, the new locking mechanism has been improved to only report a lock when a running workflow uses conflicting files in the same directory. Further, incomplete files should be reported less often since cases where the file was correctly deleted by Snakemake after a failing job are now omitted. Finally some bugs were fixed and special thanks go to Marcel Martin and Hyeshik Chang for contributing patches.
 * **18 Feb 2013:** Release 2.2.2 of Snakemake. A persistence framework has been implemented. This for now supports three new features: it tracks incompletely written files (e.g. due to power loss), sourcecode changes, and version changes of used tools. These features are exposed by new command line options, e.g. `--summary`, `--list-version-changes` and `--list-code-changes`. Further, the `--cluster` option has been extended to allow additional parameters to be passed to your favorite qsub command, e.g. `snakemake --cluster "qsub -pe threaded {threads}"`. The check for missing output files after a job has been completed now waits  a configurable time in case of failure, in order to deal with filesystem latency. Finally it is now allowed to use multiple wildcards of the same name in output files. Note that yesterdays release 2.2 contained two small bugs that I have fixed in this updated release 2.2.2.
 * **2 Jan 2013:** Release 2.1.1 of Snakemake. Some bug fixes after the major rewrite, especially for dynamic rules. Importantly, `snakemake.utils` now provides a `report` function that can be used to easily create HTML reports for conducted analyes and an `R` function to specify R code inside rules. Various small improvements: a keep going feature (-k, similar to the one of GNU Make); multiple wildcards are now allowed for dynamic output files; input files can be grouped into named sublists.
 * **24 Nov 2012:** Release 2.0 of Snakemake. The application core was rewritten and cleaned up substantially. Support for dynamic output of rules (i.e. rules where the output files are unknown at worklow start, see here) is considered stable now. It is now possible to assign numeric prioities to rules that will guide the Snakemake scheduler to prefer these rules. Further it is possible to set files or rules to highest priority upon Snakemake invocation, which will lead to the scheduler trying to complete these with all dependencies first (see here for more details). Finally, a mechanism to specify log files for rules is provided (see here, --stats produces more information, and rule docstrings are printed upon -l. This release also paves the way for other features requested by some users that I hope to complete soon, i.e. version tracking for used tools and scripts, directory locking and atomic creation of output files.