Resource File Formats¶
This chapter describes the various files fastr uses. The function and format of the files is described allowing the user to configure fastr and add DataTypes and Tools.
Config file¶
Fastr reads the config files from $FASTRHOME/config.py
by default. If the
$FASTRHOME
environment variable is not set it will default to ~/.fastr
.
As a result it read:
$FASTRHOME/config.py
(if environment variable set)~/.fastr/config.py
(otherwise)
Reading a new config file change or override settings, making the last config file read have the highest priority. All settings have a default value, making config files and all settings within optional.
Note
To verify which config files have been read you can see
fastr.config.read_config_files
which contains a list
of the read config files (in read order).
Note
If $FASTRHOME
is set, $FASTRHOME/tools
is automatically added
as a tool directory if it exists and $FASTRHOME/datatypes
is automatically
added as a type directory if it exists.
Splitting up config files¶
Sometimes it is nice to have config files split in multiple smaller files. Next
to the config.py
you can also created a directory config.d
and all
.py
files in this directory will be sourced in alphabetical order.
Given the following layout of the $FASTRHOME
directory:
./config.d/a.py
./config.d/b.txt
./config.d/c.py
./config.py
The following files will be read in order:
./config.py
./config.d/a.py
./config.d/c.py
Example config file¶
Here is a minimal config file:
# Enable debugging output
debug = False
# Define the path to the tool definitions
tools_path = ['/path/to/tools',
'/path/to/other/tools'] + tools_path
types_path = ['/path/to/datatypes',
'/path/to/other/datatypes'] + types_path
# Specify what your preferred output types are.
preferred_types += ["NiftiImageFileCompressed",
"NiftiImageFile"]
# Set the tmp mount
mounts['tmp'] = '/path/to/tmpdir'
Format¶
The config file is actually a python source file. The next syntax applies to setting configuration values:
# Simple values
float_value = 1.0
int_value = 1
str_value = "Some value"
other_str_value = 'name'.capitalize()
# List-like values
list_value = ['over', 'ride', 'values']
other_list_value.prepend('first')
other_list_value.append('list')
# Dict-like values
dict_value = {'this': 1, 'is': 2, 'fixed': 3}
other_dict_value['added'] = 'this key'
Note
Dictionaries and list always have a default, so you can always append or assign elements to them and do not have to create them in a config file. Best practice is to only edit them unless you really want to block out the earliers config files.
Most operations will be assigning values, but for list and dict values a special wrapper object is used that allows manipulations from the default. This limits the operations allowed.
List values in the config.py
have the following supported operators/methods:
+
,__add__
and__radd__
+=
or__iadd__
append
prepend
extend
Mapping (dict-like) values in the config.py
have the following supported operators/methods:
update
[]
or__getitem__
,__setitem__
and__delitem__
Configuration fields¶
This is a table the known config fields on the system:
name |
type |
description |
default |
---|---|---|---|
debug |
bool |
Flag to enable/disable debugging |
False |
examplesdir |
str |
Directory containing the fastr examples |
$systemdir/examples |
execution_plugin |
str |
The default execution plugin to use |
‘ProcessPoolExecution’ |
executionscript |
str |
Execution script location |
$systemdir/execution/executionscript.py |
extra_config_dirs |
list |
Extra configuration directories to read |
[‘’] |
filesynchelper_url |
str |
Redis url e.g. redis://localhost:6379 |
‘’ |
job_cleanup_level |
str |
The level of cleanup required, options: all, no_cleanup, non_failed |
no_cleanup |
log_to_file |
bool |
Indicate if default logging settings should log to files or not |
False |
logdir |
str |
Directory where the fastr logs will be placed |
$userdir/logs |
logging_config |
dict |
Python logger config |
{} |
loglevel |
int |
The log level to use (as int), INFO is 20, WARNING is 30, etc |
20 |
logtype |
str |
Type of logging to use |
‘default’ |
mounts |
dict |
A dictionary containing all mount points in the VFS system |
{‘tmp’: ‘$TMPDIR’, ‘example_data’: ‘$systemdir/examples/data’, ‘home’: ‘~/’} |
networks_path |
list |
Directories to scan for networks |
[‘$userdir/networks’, ‘$resourcedir/networks’] |
plugins_path |
list |
Directories to scan for plugins |
[‘$userdir/plugins’, ‘$resourcedir/plugins’] |
preferred_types |
list |
A list indicating the order of the preferred types to use. First item is most preferred. |
[] |
protected_modules |
list |
A list of modules in the environmnet modules that are protected against unloading |
[] |
queue_report_interval |
int |
Interval in which to report the number of queued jobs (default is 0, no reporting) |
0 |
reporting_plugins |
list |
The reporting plugins to use, is a list of all plugins to be activated |
[‘SimpleReport’] |
resourcesdir |
str |
Directory containing the fastr system resources |
$systemdir/resources |
schemadir |
str |
Directory containing the fastr data schemas |
$systemdir/schemas |
source_job_limit |
int |
The number of source jobs allowed to run concurrently |
0 |
systemdir |
str |
Fastr installation directory |
Directory of the top-level fastr package |
tools_path |
list |
Directories to scan for tools |
[‘$userdir/tools’, ‘$resourcedir/tools’] |
types_path |
list |
Directories to scan for datatypes |
[‘$userdir/datatypes’, ‘$resourcedir/datatypes’] |
userdir |
str |
Fastr user configuration directory |
$FASTRHOME or ~/.fastr |
warn_develop |
bool |
Warning users on import if this is not a production version of fastr |
True |
web_hostname |
str |
The hostname to expose the web app for |
‘localhost’ |
Note
This tables only includes the fastr default config fields, but not the fields added by plugins. For information look at the appropriate plugin reference. For the built-in fastr plugins they can be found at the plugin reference
Tool
description¶
Tools
are the building blocks in the fastr network. To add new
Tools
to fastr, XML/json files containing a Tool
definition can be added. These files have the following layout:
Attribute |
Description |
||
---|---|---|---|
|
The id of this Tool (used internally in fastr) |
||
|
The name of the Tool, for human readability |
||
|
The version of the Tool wrapper (not the binary) |
||
|
The url of the Tool wrapper |
||
|
List of authors of the Tools wrapper |
||
|
Name of the author |
||
|
Email address of the author |
||
|
URL of the website of the author |
||
|
|
List of tags describing the Tool |
|
|
Description of the underlying command |
||
|
Version of the tool that is wrapped |
||
|
Website where the tools that is wrapped can be obtained |
||
|
Description of the target binaries/script of this Tool |
||
|
OS targeted (windows, linux, macos or * (for any) |
||
|
Architecture targeted 32, 64 or * (for any) |
||
… |
Extra variables based on the target used, see Targets |
||
|
Description of the Tool |
||
|
License of the Tool, either full license or a clear name (e.g. LGPL, GPL v2) |
||
|
List of authors of the Tool (not the wrapper!) |
||
|
Name of the authors |
||
|
Email address of the author |
||
|
URL of the website of the author |
||
|
The interface definition see Interfaces |
||
|
Help text explaining the use of the Tool |
||
|
Bibtext of the Citation(s) to reference when using this Tool for a publication |