Resource File Formats

This chapter describes the various files fastr uses. The function and format of the files is described allowing the user to configure fastr and add DataTypes and Tools.

Config file

Fastr reads the config files from $FASTRHOME/config.py by default. If the $FASTRHOME environment variable is not set it will default to ~/.fastr. As a result it read:

  • $FASTRHOME/config.py (if environment variable set)

  • ~/.fastr/config.py (otherwise)

Reading a new config file change or override settings, making the last config file read have the highest priority. All settings have a default value, making config files and all settings within optional.

Note

To verify which config files have been read you can see fastr.config.read_config_files which contains a list of the read config files (in read order).

Note

If $FASTRHOME is set, $FASTRHOME/tools is automatically added as a tool directory if it exists and $FASTRHOME/datatypes is automatically added as a type directory if it exists.

Splitting up config files

Sometimes it is nice to have config files split in multiple smaller files. Next to the config.py you can also created a directory config.d and all .py files in this directory will be sourced in alphabetical order.

Given the following layout of the $FASTRHOME directory:

./config.d/a.py
./config.d/b.txt
./config.d/c.py
./config.py

The following files will be read in order:

  1. ./config.py

  2. ./config.d/a.py

  3. ./config.d/c.py

Example config file

Here is a minimal config file:

# Enable debugging output
debug = False

# Define the path to the tool definitions
tools_path = ['/path/to/tools',
              '/path/to/other/tools'] + tools_path
types_path = ['/path/to/datatypes',
              '/path/to/other/datatypes'] + types_path


# Specify what your preferred output types are.
preferred_types += ["NiftiImageFileCompressed",
                    "NiftiImageFile"]

# Set the tmp mount
mounts['tmp'] = '/path/to/tmpdir'

Format

The config file is actually a python source file. The next syntax applies to setting configuration values:

# Simple values
float_value = 1.0
int_value = 1
str_value = "Some value"
other_str_value = 'name'.capitalize()

# List-like values
list_value = ['over', 'ride', 'values']
other_list_value.prepend('first')
other_list_value.append('list')

# Dict-like values
dict_value = {'this': 1, 'is': 2, 'fixed': 3}
other_dict_value['added'] = 'this key'

Note

Dictionaries and list always have a default, so you can always append or assign elements to them and do not have to create them in a config file. Best practice is to only edit them unless you really want to block out the earliers config files.

Most operations will be assigning values, but for list and dict values a special wrapper object is used that allows manipulations from the default. This limits the operations allowed.

List values in the config.py have the following supported operators/methods:

  • +, __add__ and __radd__

  • += or __iadd__

  • append

  • prepend

  • extend

Mapping (dict-like) values in the config.py have the following supported operators/methods:

  • update

  • [] or __getitem__, __setitem__ and __delitem__

Configuration fields

This is a table the known config fields on the system:

name

type

description

default

debug

bool

Flag to enable/disable debugging

False

examplesdir

str

Directory containing the fastr examples

$systemdir/examples

execution_plugin

str

The default execution plugin to use

‘ProcessPoolExecution’

executionscript

str

Execution script location

$systemdir/execution/executionscript.py

extra_config_dirs

list

Extra configuration directories to read

[‘’]

filesynchelper_url

str

Redis url e.g. redis://localhost:6379

‘’

job_cleanup_level

str

The level of cleanup required, options: all, no_cleanup, non_failed

no_cleanup

log_to_file

bool

Indicate if default logging settings should log to files or not

False

logdir

str

Directory where the fastr logs will be placed

$userdir/logs

logging_config

dict

Python logger config

{}

loglevel

int

The log level to use (as int), INFO is 20, WARNING is 30, etc

20

logtype

str

Type of logging to use

‘default’

mounts

dict

A dictionary containing all mount points in the VFS system

{‘tmp’: ‘$TMPDIR’, ‘examples’: ‘$systemdir/examples’, ‘example_data’: ‘$systemdir/examples/data’, ‘home’: ‘~/’, ‘fastr_home’: ‘$FASTRHOME or ~/.fastr’}

networks_path

list

Directories to scan for networks

[‘$userdir/networks’, ‘$resourcedir/networks’]

plugins_path

list

Directories to scan for plugins

[‘$userdir/plugins’, ‘$resourcedir/plugins’]

preferred_types

list

A list indicating the order of the preferred types to use. First item is most preferred.

[]

protected_modules

list

A list of modules in the environmnet modules that are protected against unloading

[]

queue_report_interval

int

Interval in which to report the number of queued jobs (default is 0, no reporting)

0

reporting_plugins

list

The reporting plugins to use, is a list of all plugins to be activated

[‘SimpleReport’]

resourcesdir

str

Directory containing the fastr system resources

$systemdir/resources

schemadir

str

Directory containing the fastr data schemas

$systemdir/schemas

source_job_limit

int

The number of source jobs allowed to run concurrently

0

systemdir

str

Fastr installation directory

Directory of the top-level fastr package

tools_path

list

Directories to scan for tools

[‘$userdir/tools’, ‘$resourcedir/tools’]

types_path

list

Directories to scan for datatypes

[‘$userdir/datatypes’, ‘$resourcedir/datatypes’]

userdir

str

Fastr user configuration directory

$FASTRHOME or ~/.fastr

warn_develop

bool

Warning users on import if this is not a production version of fastr

True

web_hostname

str

The hostname to expose the web app for

‘localhost’

Note

This tables only includes the fastr default config fields, but not the fields added by plugins. For information look at the appropriate plugin reference. For the built-in fastr plugins they can be found at the plugin reference

Tool description

Tools are the building blocks in the fastr network. To add new Tools to fastr, XML/json files containing a Tool definition can be added. These files have the following layout:

Attribute

Description

id

The id of this Tool (used internally in fastr)

name

The name of the Tool, for human readability

version

The version of the Tool wrapper (not the binary)

url

The url of the Tool wrapper

authors[]

List of authors of the Tools wrapper

name

Name of the author

email

Email address of the author

url

URL of the website of the author

tags

tag[]

List of tags describing the Tool

command

Description of the underlying command

version

Version of the tool that is wrapped

url

Website where the tools that is wrapped can be obtained

targets[]

Description of the target binaries/script of this Tool

os

OS targeted (windows, linux, macos or * (for any)

arch

Architecture targeted 32, 64 or * (for any)

Extra variables based on the target used, see Targets

description

Description of the Tool

license

License of the Tool, either full license or a clear name (e.g. LGPL, GPL v2)

authors[]

List of authors of the Tool (not the wrapper!)

name

Name of the authors

email

Email address of the author

url

URL of the website of the author

interface

The interface definition see Interfaces

help

Help text explaining the use of the Tool

cite

Bibtext of the Citation(s) to reference when using this Tool for a publication