Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning

1.2.1 - 2017-04-04

Added

  • A FastrInterface can now specify a negate flag on an automatic output that also has a prefix, which will negate the flag. This is useful for flag the suppress the creation of an output (e.g. no_mask). An example is given in the Tool fastr.util.AutoPrefixNegateTest.

Changed

  • The provenance and extra information of a Job now is not serialized in the Job, but exported to separate files next to the job file __fastr_prov__.json and __fastr_extra_job_info__.json which makes the information more accessible and reduces the memory footprint of the main process hugely as it will not read this information back anymore.
  • Most execution plugin will not overwrite the executionscript stdout and stderr but rather append it. This is only relevant when continuing a run in the an existing temporary directory, but avoids loss of information.

Fixed

  • Bug that stopped the Link.append function from returning the newly created link
  • Bugs that caused some cardinality computations of the output to fail during execution
  • Bug in the job.tmpurl that caused double slashes somewhere. Some tools chocked on this when it was used for parameters.

1.2.0 - 2017-03-15

Added

  • Failed sample annotation: when a job fails, the result is annotated and forwarded until a SinkNode, where we can determine the status and possibly point of failure of the Sample.
  • Commandline tool fastr trace that can inspect a workflow run and help trace errors and print debug information
  • Supported for Lmod modules environment next to the old environmentmodules
  • BaseDataType descendants are now (un)picklable (including EnumTypes)
  • Option to use {extension} field in sink_data, which differs from {ext} in that it doesn’t include a leading dot.
  • Support for Docker targets. A Docker target will execute a command inside of a specified docker container, allowing Tools to use Docker for distribution
  • Using the right and left shift operator (<< and >>) for creating links to Inputs using input << output or output >> input.
  • In the FastrInterfaces, automatic outputs can have a prefix for a flag that should be set for the output to be actually generated.
  • Fastr is now able to limit the amount of SourceJobs that are allowed to run concurrently.
  • Ability to report progress to PIM (use the pim_host field in the config)

Changed

  • Version can now also accept a format based on a date (e.g. 2017-02-17_bananas) which will be parsed the same way as 2017.02.17_bananas
  • Work on the ExecutionPlugin and the corresponding API. Has better fall-backs and a mechanism to advertise plugin capabilities.
  • The collector plugins have the input and input_parts fields merged, and the output and output_parts fields merged.

Fixed

  • In some cases the log directory was not created properly, causing an handled exception
  • A bug making the handling of Booleans incorrect for the FastrInterface, when a Boolean was given a flag would also appear when it was False
  • Serialization of the namespace of a Network was not correct
  • Check version of Fastr that creates and executes a Job against each other
  • load_gpickle helper can handle data with Enums that use to cause an AttributeError
  • Output validation of Jobs did not work correctly for automatic outputs

1.1.2 - 2016-12-22

Fixed

  • The example network in resources/networks/add_ints.json was using an old serialization format making it non-functions. Replaced by a new network file.

1.1.1 - 2016-12-22

Fixed

  • Network runs called from an interpreter (and not file) caused a crash because the network tried to report the file used. Better handling of these situations.

1.1.0 - 2016-12-08

Added

  • Namespaces for resources (tools and networks)
  • Network manager located at fastr.networklist
  • RQExecution plugin. This plugin uses python-rq to manage a job queue.
  • LinearExecution plugin. This plugin uses a background thread for execution.
  • BlockingExecution plugin. This plugin executes jobs in a blocking fashion.
  • Automatic generation of documentation for all plugins, the configuration fields and all commandline tools.

Changed

  • Provenance is updated with a network dump and used tool definitions.
  • New configuration system that uses python files
  • New plugin system that integrates with the new configuration system and enables automatic importing of plugins
  • The fastr command line tools now use an entrypoint which is located in fastr.utils.cmd. This code also dispatches the sub commands.

Removed

  • fastr.config file. This is replaced by the config.py file. Go to the docs!

Fixed

  • Adds explicit tool namespace and version to the provenance document.