fastr Package¶
fastr
Package¶
FASTR is a top level package which includes all parts required to create networks and edit networks.
-
class
fastr.__init__.
Network
[source]¶ The class representing a Network, this is in fact a reference to
fastr.core.network.Network
.
-
class
fastr.__init__.
Node
[source]¶ The class representing a Node, this is in fact a reference to
fastr.core.node.Node
.
-
class
fastr.__init__.
Link
[source]¶ The class representing a Link, this is in fact a reference to
fastr.core.link.Link
.
-
class
fastr.__init__.
SourceNode
[source]¶ The class representing a data source, this is in fact a reference to
fastr.core.node.SourceNode
.
-
class
fastr.__init__.
SinkNode
[source]¶ The class representing a data sink, this is in fact a reference to
fastr.core.node.SinkNode
.
-
class
fastr.__init__.
ConstantNode
[source]¶ The class representing a constant data source, this is in fact a reference to
fastr.core.node.ConstantNode
.
-
fastr.__init__.
toollist
¶ A
fastr.core.toolmanager.ToolManager
containing all Tools known to the FASTR environment. The toollist can be accessed in a similar way to a dict. Indexing with a tool id will return the newest version of the Tool. If a specific version of the tool is required a tuple can be used as the index:>>> import fastr >>> fastr.toollist['testtool'] <Tool: testtool version: 4.2> >>> fastr.toollist['testtool', '2.0'] <Tool: testtool version: 2.0>
-
fastr.__init__.
typelist
¶ A
fastr.core.datatypemanager.DataTypeManager
containing all Types known to the FASTR environment. This is usuable as a dict where the key is the datatype id and the value is the datatype itself.
-
class
fastr.__init__.
Network
(id_='unnamed_network', version=None)[source] Bases:
fastr.core.serializable.Serializable
The Network class represents a workflow. This includes all Nodes (including ConstantNodes, SourceNodes and Sinks) and Links.
-
NETWORK_DUMP_FILE_NAME
= '__fastr_network__.json'¶
-
SINK_DUMP_FILE_NAME
= '__sink_data__.json'¶
-
SOURCE_DUMP_FILE_NAME
= '__source_data__.pickle.gz'¶
-
__dataschemafile__
= 'Network.schema.json'¶
-
__eq__
(other)[source]¶ Compare two Networks and see if they are equal.
Parameters: other ( Network
) –Returns: flag indicating that the Networks are the same Return type: bool
-
__getitem__
(item)[source]¶ Get an item by its fullid. The fullid can point to a link, node, input, output or even subinput/suboutput.
Parameters: item (str,unicode) – fullid of the item to retrieve Returns: the requested item
-
__getstate__
()[source]¶ Retrieve the state of the Network
Returns: the state of the object Rtype dict:
-
__init__
(id_='unnamed_network', version=None)[source]¶ Create a new, empty Network
Parameters: name (str) – name of the Network Returns: newly created Network Raises: OSError – if the tmp mount in the config is not a writable directory
-
__module__
= 'fastr.core.network'¶
-
__setstate__
(state)[source]¶ Set the state of the Network by the given state. This completely overwrites the old state!
Parameters: state (dict) – The state to populate the object with Returns: None
-
add_link
(link)[source]¶ Add a Link to the Network. Make sure the link is in the link list and the link parent is set to this Network
Parameters: link (
Link
) – link to addRaises: - FastrTypeError – if link is incorrectly typed
- FastrNetworkMismatchError – if the link already belongs to another Network
-
add_node
(node)[source]¶ Add a Node to the Network. Make sure the node is in the node list and the node parent is set to this Network
Parameters: node ( Node
) – node to addRaises: FastrTypeError – if node is incorrectly typed
-
check_id
(id_)[source]¶ Check if an id for an object is valid and unused in the Network. The method will always returns True if it does not raise an exception.
Parameters: id (str) – the id to check
Returns: True
Raises: - FastrValueError – if the id is not correctly formatted
- FastrValueError – if the id is already in use
-
create_constant
(datatype, data, id_=None, stepid=None, nodegroup=None, sourcegroup=None)[source]¶ Create a ConstantNode in this Network. The Node will be automatically added to the Network.
Parameters: - datatype (
BaseDataType
) – The DataType of the constant node - data (datatype or list of datatype) – The data to hold in the constant node
- id (str) – The id of the constant node to be created
- stepid (str) – The stepid to add the created constant node to
- nodegroup (str) – The group the node belongs to, this can be important for FlowNodes and such, as they will have matching dimension names.
Returns: the newly created constant node
Return type: - datatype (
-
create_link
(source, target, id_=None, collapse=None, expand=None)[source]¶ Create a link between two Nodes and add it to the current Network.
Parameters: - source (
BaseOutput
) – the output that is the source of the link - target (
BaseInput
) – the input that is the target of the link - id (str) – the id of the link
Returns: the created link
Type: - source (
-
create_node
(tool, id_=None, stepid=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Create a Node in this Network. The Node will be automatically added to the Network.
Parameters: Returns: the newly created node
Return type:
-
create_sink
(datatype, id_=None, stepid=None)[source]¶ Create a SinkNode in this Network. The Node will be automatically added to the Network.
Parameters: Returns: the newly created sink node
Return type:
-
create_source
(datatype, id_=None, stepid=None, nodegroup=None, sourcegroup=None)[source]¶ Create a SourceNode in this Network. The Node will be automatically added to the Network.
Parameters: - datatype (
BaseDataType
) – The DataType of the source source_node - id (str) – The id of the source source_node to be created
- stepid (str) – The stepid to add the created source source_node to
- nodegroup (str) – The group the node belongs to, this can be important for FlowNodes and such, as they will have matching dimension names.
- sourcegroup (str) – DEPRECATED! The nodegroup this SourceNode will be added to
Returns: the newly created source source_node
Return type: SourceNode
- datatype (
-
draw_network
(name='network_layout', img_format='svg', draw_dimension=False)[source]¶ Output a dot file and try to convert it to an image file.
Parameters: img_format (str) – extension of the image format to convert to Returns: path of the image created or None if failed Return type: str or None
-
execute
(sourcedata, sinkdata, execution_plugin=None, tmpdir=None, cluster_queue=None)[source]¶ Execute the Network with the given data. This will analyze the Network, create jobs and send them to the execution backend of the system.
Parameters: Raises: - FastrKeyError – if a source has not corresponding key in sourcedata
- FastrKeyError – if a sink has not corresponding key in sinkdata
-
fullid
¶ The fullid of the Network
-
id
¶ The id of the Network. This is a read only property.
-
job_finished
(job, execution_interface)[source]¶ Call-back handler for when a job is finished. Will collect the results and handle blocking jobs. This function is automatically called when the execution plugin finished a job.
Parameters: job ( Job
) – the job that finished
-
remove
(value)[source]¶ Remove an item from the Network.
Parameters: value ( Node
orLink
) – the item to remove
-
run_id
¶ The run id of the network
-
-
class
fastr.__init__.
Link
(source, target, parent=None, id_=None, collapse=None, expand=None)[source] Bases:
fastr.core.updateable.Updateable
,fastr.core.serializable.Serializable
Class for linking outputs (
BaseOutput
) to inputs (BaseInput
)Examples:
>>> import fastr >>> network = fastr.Network() >>> link1 = network.create_link( n1.ouputs['out1'], n2.inputs['in2'] ) link2 = Link() link2.source = n1.ouputs['out1'] link2.target = n2.inputs['in2']
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'Link.schema.json'¶
-
__eq__
(other)[source]¶ Test for equality between two Links
Parameters: other (Link) – object to test against Returns: True for equality, False otherwise Return type: bool
-
__getitem__
(index)[source]¶ Get a an item for this Link. The item will be retrieved from the connected output, but a diverging or converging flow can change the number of samples/cardinality.
Parameters: index (SampleIndex) – index of the item to retrieve Returns: the requested item Return type: SampleItem
Raises: FastrIndexError – if the index length does not match the number dimensions in the source data (after collapsing/expanding)
-
__init__
(source, target, parent=None, id_=None, collapse=None, expand=None)[source]¶ Create a new Link in a Network.
Parameters: - source (
BaseOutput
) – the source output - target (
BaseInput
) – the target input - parent (
Network
or None) – the parent network, if None is given the fastr.current_network is assumed to be the parent - id (str or None) – the id of the link, if no id_ is given, the id will be in the form of “link_{:d}”
- collapse (int, str, or tuple of int/str) – the dimensions that the link has to collapse on
- expand (bool) – Does this link need to expand the cardinality into a new sample dimension
Returns: newly created Link
Raises: - FastrValueError – if parent is not given and fastr.current_network is not set
- FastrValueError – if the source output is not in the same network as the Link
- FastrValueError – if the target input is not in the same network as the Link
- source (
-
__module__
= 'fastr.core.link'¶
-
__repr__
()[source]¶ Get a string representation for the Link
Returns: the string representation Return type: str
-
__setstate__
(state)[source]¶ Set the state of the Link by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None Raises: FastrValueError – if the parent network and fastr.current_network are not set
-
cardinality
(index=None)[source]¶ Cardinality for a Link is given by source Output and the collapse/expand settings
Parameters: key (SampleIndex) – key for a specific sample (can be only a sample index!) Returns: the cardinality Return type: int, sympy.Symbol Raises: FastrIndexError – if the index length does not match the number of dimension in the data
-
collapse
¶ The converging dimensions of this link. Collapsing changes some dimensions of sample lists into cardinality, reshaping the data.
Collapse can be set to a tuple or an int/str, in which case it will be automatically wrapped in a tuple. The int will be seen as indices of the dimensions to collapse. The str will be seen as the name of the dimensions over which to collapse.
Raises: FastrTypeError – if assigning a collapse value of a wrong type
-
collapse_indexes
¶ The converging dimensions of this link as integers. Dimension names are replaces with the corresponding int.
Collapsing changes some dimensions of sample lists into cardinality, reshaping the data
-
classmethod
createobj
(state, network=None)[source]¶ Create object function for Link
Parameters: - cls – The class to create
- state – The state to use to create the Link
- network – the parent Network
Returns: newly created Link
-
destroy
()[source]¶ The destroy function of a link removes all default references to a link. This means the references in the network, input and output connected to this link. If there is no references in other places in the code, it will destroy the link (reference count dropping to zero).
This function is called when a source for an input is set to another value and the links becomes disconnected. This makes sure there is no dangling links.
-
dimnames
¶ The dimension names for this Link. The dimension names depend on the connected source output and the collapse/expand.
-
expand
¶ Flag indicating that the link will expand the cardininality into a new sample dimension to be created.
-
fullid
¶ The full defining ID for the Input
-
parent
¶ The Network to which this Link belongs.
-
size
¶ The size of the data delivered by the link. This can be different from the source size because the link can make data collapse or expand.
-
source
¶ The source
BaseOutput
of the Link. Setting the source will automatically register the Link with the source BaseOutput. Updating source will also make sure the Link is unregistered with the previous source.Raises: FastrTypeError – if assigning a non BaseOutput
-
status
¶
-
target
¶ The target
BaseInput
of the Link. Setting the target will automatically register the Link with the target BaseInput. Updating target will also make sure the Link is unregistered with the previous target.Raises: FastrTypeError – if assigning a non BaseInput
-
-
class
fastr.__init__.
Node
(tool, id_=None, parent=None, cores=None, memory=None, walltime=None)[source] Bases:
fastr.core.updateable.Updateable
,fastr.core.serializable.Serializable
The class encapsulating a node in the network. The node is responsible for setting and checking inputs and outputs based on the description provided by a tool instance.
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'Node.schema.json'¶
-
__eq__
(other)[source]¶ Compare two Node instances with each other. This function ignores the parent and update status, but tests rest of the dict for equality. equality
Parameters: other (Node) – the other instances to compare to Returns: True if equal, False otherwise
-
__init__
(tool, id_=None, parent=None, cores=None, memory=None, walltime=None)[source]¶ Instantiate a node.
Parameters: - tool (
Tool
) – The tool to base the node on - id (str) – the id of the node
- parent (
Network
) – the parent network of the node - cores (int) – number of cores required for executing this Node
- memory (str) – amount of memory required in the form d+[mMgG] where M is for megabyte and G for gigabyte
- walltime (str) – amount of time required in second or in the form HOURS:MINUTES:SECOND
Returns: the newly created Node
- tool (
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.node'¶
-
__repr__
()[source]¶ Get a string representation for the Node
Returns: the string representation Return type: str
-
__setstate__
(state)[source]¶ Set the state of the Node by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
blocking
¶ Indicate that the results of this Node cannot be determined without first executing the Node, causing a blockage in the creation of jobs. A blocking Nodes causes the Chunk borders.
-
create_job
(sample_id, sample_index, job_data, job_dependencies, **kwargs)[source]¶ Create a job based on the sample id, job data and job dependencies.
Parameters: - sample_id (
SampleId
) – the id of the corresponding sample - sample_index (
SampleIndex
) – the index of the corresponding sample - job_data (dict) – dictionary containing all input data for the job
- job_dependencies – other jobs that need to finish before this job can run
Returns: the created job
Return type: - sample_id (
-
dimnames
¶ Names of the dimensions in the Node output. These will be reflected in the SampleIdList of this Node.
-
execute
()[source]¶ Execute the node and create the jobs that need to run
Returns: list of jobs to run Return type: list of Jobs
-
fullid
¶ The full defining ID for the Node
-
get_sourced_nodes
()[source]¶ A list of all Nodes connected as sources to this Node
Returns: list of all nodes that are connected to an input of this node
-
id
¶ The id of the Node
-
input_groups
¶ - A list of input groups for this Node. An input group is InputGroup
- object filled according to the Node
-
listeners
¶ All the listeners requesting output of this node, this means the listeners of all Outputs and SubOutputs
-
merge_dimensions
¶
-
name
¶ Name of the Tool the Node was based on. In case a Toolless Node was used the class name is given.
-
nodegroup
¶
-
outputsize
¶ Size of the outputs in this Node
-
parent
¶ The parent network of this node.
-
prepare
()[source]¶ Prepare the node for execution. It will create a SampleIdList of the correct size and prepare the outputs.
-
required_cores
¶ Number of cores required for the execution of this Node
-
required_memory
¶ Amount of memory required for the execution of this Node. Follows the format d+[mMgG] so 500M or 4g would be valid ways to specify 500 megabytes or 4 gigabyte of memory.
-
required_time
¶ Amount of time required for the execution of this Node. Follows the format of a number of second or H:M:S, with H the number of hours, M the number of minutes and S the number of seconds.
-
set_result
(job, failed_annotation)[source]¶ Incorporate result of a job into the Node.
Parameters: - job (Type) – job of which the result to store
- failed_annotation – A set of annotations, None if no errors else containing a tuple describing the errors
-
status
¶
-
tool
¶
-
-
class
fastr.__init__.
ConstantNode
(datatype, data, id_=None)[source] Bases:
fastr.core.node.SourceNode
Class encapsulating one output for which a value can be set. For example used to set a scalar value to the input of a node.
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'ConstantNode.schema.json'¶
-
__getstate__
()[source]¶ Retrieve the state of the ConstantNode
Returns: the state of the object Rtype dict:
-
__init__
(datatype, data, id_=None)[source]¶ Instantiation of the ConstantNode.
Parameters: - datatype – The datatype of the output.
- data – the prefilled data to use.
- id – The url pattern.
This class should never be instantiated directly (unless you know what you are doing). Instead create a constant using the network class like shown in the usage example below.
usage example:
>>> import fastr >>> network = fastr.Network() >>> source = network.create_source(datatype=fastr.typelist['ITKImageFile'], id_='sourceN')
or alternatively create a constant node by assigning data to an item in an InputDict:
>>> node_a.inputs['in'] = ['some', 'data']
which automatically creates and links a ConstantNode to the specified Input
-
__module__
= 'fastr.core.node'¶
-
__setstate__
(state)[source]¶ Set the state of the ConstantNode by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
data
¶ The data stored in this constant node
-
-
class
fastr.__init__.
SourceNode
(datatype, id_=None)[source] Bases:
fastr.core.node.FlowNode
Class providing a connection to data resources. This can be any kind of file, stream, database, etc from which data can be received.
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'SourceNode.schema.json'¶
-
__eq__
(other)[source]¶ Compare two Node instances with each other. This function ignores the parent and update status, but tests rest of the dict for equality. equality
Parameters: other (Node) – the other instances to compare to Returns: True if equal, False otherwise
-
__getstate__
()[source]¶ Retrieve the state of the SourceNode
Returns: the state of the object Rtype dict:
-
__init__
(datatype, id_=None)[source]¶ Instantiation of the SourceNode.
Parameters: - datatype – The (id of) the datatype of the output.
- id – The url pattern.
This class should never be instantiated directly (unless you know what you are doing). Instead create a source using the network class like shown in the usage example below.
usage example:
>>> import fastr >>> network = fastr.Network() >>> source = network.create_source(datatype=fastr.typelist['ITKImageFile'], id_='sourceN')
-
__module__
= 'fastr.core.node'¶
-
__setstate__
(state)[source]¶ Set the state of the SourceNode by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
datatype
¶ The datatype of the data this source supplies.
-
dimnames
¶ Names of the dimensions in the SourceNode output. These will be reflected in the SampleIdLists.
-
execute
()[source]¶ Execute the source node and create the jobs that need to run
Returns: list of jobs to run Return type: list of Jobs
-
output
¶ Shorthand for
self.outputs['output']
-
outputsize
¶ The size of output of this SourceNode
-
set_data
(data, ids=None)[source]¶ Set the data of this source node.
Parameters: - data (dict, OrderedDict or list of urls) – the data to use
- ids – if data is a list, a list of accompanying ids
-
sourcegroup
¶
-
valid
¶ This does nothing. It only overloads the valid method of Node(). The original is intended to check if the inputs are connected to some output. Since this class does not implement inputs, it is skipped.
-
configmanager
Module¶
This module defines the Fastr Config class for managing the configuration of Fastr. The config object is stored directly in the fastr top-level module.
-
class
fastr.configmanager.
Config
(*configfiles)[source]¶ Bases:
object
Class contain the fastr configuration
-
DEFAULT_FIELDS
= {'resourcesdir': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources', 'Directory containing the fastr system resources', '$systemdir/resources'), 'examplesdir': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/examples', 'Directory containing the fastr examples', '$systemdir/examples'), 'plugins_path': (<type 'list'>, ['/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins'], 'Directories to scan for plugins', ['$userdir/plugins', '$resourcedir/plugins']), 'executionscript': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/execution/executionscript.py', 'Execution script location', '$systemdir/execution/executionscript.py'), 'protected_modules': (<type 'list'>, [], 'A list of modules in the environmnet modules that are protected against unloading'), 'warn_develop': (<type 'bool'>, True, 'Warning users on import if this is not a production version of fastr'), 'preferred_types': (<type 'list'>, [], 'A list indicating the order of the preferred types to use. First item is most preferred.'), 'pim_host': (<type 'str'>, '', 'Host of the PIM server to report to'), 'logtype': (<type 'str'>, 'default', 'Type of logging to use'), 'systemdir': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr', 'Fastr installation directory'), 'execution_plugin': (<type 'str'>, 'ProcessPoolExecution', 'The default execution plugin to use'), 'logdir': (<type 'str'>, '/home/docs/.fastr/logs', 'Directory where the fastr logs will be placed', '$userdir/logs'), 'web_secret_key': (<type 'str'>, 'VERYSECRETKEY!', 'The secret key to use for the flask web app'), 'types_path': (<type 'list'>, ['/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes'], 'Directories to scan for datatypes', ['$userdir/datatypes', '$resourcedir/datatypes']), 'mounts': (<type 'dict'>, {'tmp': '/tmp', 'home': '/home/docs/', 'example_data': '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/examples/data'}, 'A dictionary containing all mount points in the VFS system', {'tmp': '$TMPDIR', 'home': '~/', 'example_data': '$systemdir/examples/data'}), 'tools_path': (<type 'list'>, ['/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/tools'], 'Directories to scan for tools', ['$userdir/tools', '$resourcedir/tools']), 'source_job_limit': (<type 'int'>, 4, 'The number of source jobs allowed to run concurrently'), 'web_port': (<type 'str'>, '5000', 'The port to expose the web app on'), 'web_hostname': (<type 'str'>, 'localhost', 'The hostname to expose the web app for'), 'networks_path': (<type 'list'>, ['/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/networks'], 'Directories to scan for networks', ['$userdir/networks', '$resourcedir/networks']), 'userdir': (<type 'str'>, '/home/docs/.fastr', 'Fastr user configuration directory', '~/.fastr'), 'schemadir': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/schemas', 'Directory containing the fastr data schemas', '$systemdir/schemas'), 'debug': (<type 'bool'>, False, 'Flag to enable/disable debugging')}¶
-
__dict__
= dict_proxy({'resourcesdir': <property object>, '__module__': 'fastr.configmanager', 'plugins_path': <property object>, 'executionscript': <property object>, '_field_property': <staticmethod object>, 'warn_develop': <property object>, 'protected_modules': <property object>, 'web_url': <function web_url>, '__dict__': <attribute '__dict__' of 'Config' objects>, '__weakref__': <attribute '__weakref__' of 'Config' objects>, '__init__': <function __init__>, 'DEFAULT_FIELDS': {'resourcesdir': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources', 'Directory containing the fastr system resources', '$systemdir/resources'), 'examplesdir': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/examples', 'Directory containing the fastr examples', '$systemdir/examples'), 'plugins_path': (<type 'list'>, ['/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins'], 'Directories to scan for plugins', ['$userdir/plugins', '$resourcedir/plugins']), 'executionscript': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/execution/executionscript.py', 'Execution script location', '$systemdir/execution/executionscript.py'), 'protected_modules': (<type 'list'>, [], 'A list of modules in the environmnet modules that are protected against unloading'), 'warn_develop': (<type 'bool'>, True, 'Warning users on import if this is not a production version of fastr'), 'preferred_types': (<type 'list'>, [], 'A list indicating the order of the preferred types to use. First item is most preferred.'), 'pim_host': (<type 'str'>, '', 'Host of the PIM server to report to'), 'logtype': (<type 'str'>, 'default', 'Type of logging to use'), 'systemdir': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr', 'Fastr installation directory'), 'execution_plugin': (<type 'str'>, 'ProcessPoolExecution', 'The default execution plugin to use'), 'logdir': (<type 'str'>, '/home/docs/.fastr/logs', 'Directory where the fastr logs will be placed', '$userdir/logs'), 'web_secret_key': (<type 'str'>, 'VERYSECRETKEY!', 'The secret key to use for the flask web app'), 'types_path': (<type 'list'>, ['/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes'], 'Directories to scan for datatypes', ['$userdir/datatypes', '$resourcedir/datatypes']), 'mounts': (<type 'dict'>, {'tmp': '/tmp', 'home': '/home/docs/', 'example_data': '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/examples/data'}, 'A dictionary containing all mount points in the VFS system', {'tmp': '$TMPDIR', 'home': '~/', 'example_data': '$systemdir/examples/data'}), 'tools_path': (<type 'list'>, ['/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/tools'], 'Directories to scan for tools', ['$userdir/tools', '$resourcedir/tools']), 'source_job_limit': (<type 'int'>, 4, 'The number of source jobs allowed to run concurrently'), 'web_port': (<type 'str'>, '5000', 'The port to expose the web app on'), 'web_hostname': (<type 'str'>, 'localhost', 'The hostname to expose the web app for'), 'networks_path': (<type 'list'>, ['/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/networks'], 'Directories to scan for networks', ['$userdir/networks', '$resourcedir/networks']), 'userdir': (<type 'str'>, '/home/docs/.fastr', 'Fastr user configuration directory', '~/.fastr'), 'schemadir': (<type 'str'>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/schemas', 'Directory containing the fastr data schemas', '$systemdir/schemas'), 'debug': (<type 'bool'>, False, 'Flag to enable/disable debugging')}, 'preferred_types': <property object>, 'pim_host': <property object>, 'userdir': <property object>, 'examplesdir': <property object>, 'register_fields': <function register_fields>, 'set_field': <function set_field>, 'execution_plugin': <property object>, 'logdir': <property object>, 'get_field': <function get_field>, 'read_config': <function read_config>, '__doc__': '\n Class contain the fastr configuration\n ', 'web_secret_key': <property object>, 'types_path': <property object>, 'mounts': <property object>, 'tools_path': <property object>, '_create_field_properties': <classmethod object>, '_update_logging': <function _update_logging>, 'source_job_limit': <property object>, 'web_port': <property object>, 'systemdir': <property object>, 'web_hostname': <property object>, 'process_pool_worker_number': <property object>, 'networks_path': <property object>, 'logtype': <property object>, '__repr__': <function __repr__>, 'schemadir': <property object>, 'debug': <property object>, 'x': '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins'})¶
-
__module__
= 'fastr.configmanager'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
debug
¶
-
examplesdir
¶
-
execution_plugin
¶
-
executionscript
¶
-
logdir
¶
-
logtype
¶
-
mounts
¶
-
networks_path
¶
-
pim_host
¶
-
plugins_path
¶
-
preferred_types
¶
-
process_pool_worker_number
¶
-
protected_modules
¶
-
read_config
(filename)[source]¶ Read a configuration and update the configuration object accordingly
Parameters: filename – the configuration file to read
-
read_config_files
= None¶ Trace of the config files read by this object
-
resourcesdir
¶
-
schemadir
¶
-
source_job_limit
¶
-
systemdir
¶
-
tools_path
¶
-
types_path
¶
-
userdir
¶
-
warn_develop
¶
-
web_hostname
¶
-
web_port
¶
-
web_secret_key
¶
-
web_url
()[source]¶ Construct a fqdn from the web[‘hostname’] and web[‘port’] settings. :return: FQDN :rtype: str
-
x
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins'¶
-
-
class
fastr.configmanager.
EmptyDefault
(data=None)[source]¶ Bases:
object
-
__dict__
= dict_proxy({'extend': <function extend>, '__delitem__': <function __delitem__>, '__module__': 'fastr.configmanager', '__getitem__': <function __getitem__>, 'asdict': <function asdict>, 'update': <function update>, '__iadd__': <function __iadd__>, '__radd__': <function __radd__>, '__setitem__': <function __setitem__>, 'append': <function append>, '__add__': <function __add__>, '__dict__': <attribute '__dict__' of 'EmptyDefault' objects>, 'prepend': <function prepend>, '__weakref__': <attribute '__weakref__' of 'EmptyDefault' objects>, '__doc__': None, '__init__': <function __init__>, 'aslist': <function aslist>})¶
-
__module__
= 'fastr.configmanager'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
datatypes
Module¶
The datatypes module holds all DataTypes generated by fastr and all the base classes for these datatypes.
-
class
fastr.datatypes.
AnalyzeImageFile
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'Analyze Image file formate'¶
-
extension
= 'hdr'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/AnalyzeImageFile.py'¶
-
module
= <module 'AnalyzeImageFile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/AnalyzeImageFile.pyc'>¶
-
-
class
fastr.datatypes.
AnyFile
(value=None)[source]¶ Bases:
fastr.datatypes.TypeGroup
Special Datatype in fastr that is a TypeGroup with all known DataTypes as its members.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'TypeGroup AnyFile\nAnyFile (AnyFile) is a group of consisting of all URLTypes known by fastr, currently:\n - <URLType: TxtFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: MetaImageFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: Directory class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: TifImageFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: NiftiImageFileUncompressed class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: FilePrefix class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: ProvNFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: JsonFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: AnalyzeImageFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: NiftiImageFileCompressed class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>'¶
-
-
class
fastr.datatypes.
AnyType
(value=None)[source]¶ Bases:
fastr.datatypes.TypeGroup
Special Datatype in fastr that is a TypeGroup with all known DataTypes as its members.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'TypeGroup AnyType\nAnyType (AnyType) is a group of consisting of all DataTypes known by fastr, currently:\n - <URLType: TxtFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: MetaImageFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: Directory class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: TifImageFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <ValueType: UnsignedInt class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <DataType: Deferred class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <EnumType: __CrossValidation_0.1_interface__method__Enum__ class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <ValueType: String class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <ValueType: Boolean class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: ProvNFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: FilePrefix class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: NiftiImageFileUncompressed class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <EnumType: __source-interface__behaviour__Enum__ class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: JsonFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: AnalyzeImageFile class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <ValueType: Int class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <ValueType: Float class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <URLType: NiftiImageFileCompressed class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>\n - <EnumType: __ioplugin__behaviour__Enum__ class [\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m]>'¶
-
-
class
fastr.datatypes.
BaseDataType
(value=None, format_=None)[source]¶ Bases:
fastr.core.baseplugin.BasePlugin
The base class for all datatypes in the fastr type system.
-
__abstractmethods__
= frozenset(['__init__'])¶
-
__eq__
(other)[source]¶ Test the equality of two DataType objects
Parameters: other (DataType) – the object to compare against Returns: flag indicating equality Return type: bool
-
__init__
(value=None, format_=None)[source]¶ The BaseDataType constructor.
Parameters: - value – value to assign to the new BaseDataType object
- format – the format used for the ValueType
Returns: new BaseDataType object
Raises: FastrNotImplementedError – if id, name, version or description is None
-
__module__
= 'fastr.datatypes'¶
-
__ne__
(other)[source]¶ Test if two objects are not equal. This is by default done by negating the __eq__ operator
Parameters: other (DataType) – the object to compare against Returns: flag indicating equality Return type: bool
-
__repr__
()[source]¶ Returns string representation of the BaseDataType
Returns: string represenation Return type: str
-
__str__
()[source]¶ Returns the string version of the BaseDataType
Returns: string version Return type: str
-
checksum
()[source]¶ Generate a checksum for the value of this DataType
Returns: the checksum of the value Return type: str
-
description
= ''¶
-
extension
= None¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/datatypes.pyc'¶
-
fullid
= 'fastr://typelist/BaseDataType'¶
-
id
= 'BaseDataType'¶
-
classmethod
isinstance
(value)[source]¶ Indicate whether value is an instance for this DataType.
Returns: the flag indicating the value is of this DataType Return type: bool
-
name
= 'BaseDataType'¶
-
parent
= DataTypeManager AnalyzeImageFile : <URLType: AnalyzeImageFile> AnyFile : <TypeGroup: AnyFile> AnyType : <TypeGroup: AnyType> Boolean : <ValueType: Boolean> Deferred : <DataType: Deferred> Directory : <URLType: Directory> FilePrefix : <URLType: FilePrefix> Float : <ValueType: Float> ITKImageFile : <TypeGroup: ITKImageFile> Int : <ValueType: Int> JsonFile : <URLType: JsonFile> MetaImageFile : <URLType: MetaImageFile> NiftiImageFile : <TypeGroup: NiftiImageFile> NiftiImageFileCompressed : <URLType: NiftiImageFileCompressed> NiftiImageFileUncompressed : <URLType: NiftiImageFileUncompressed> Number : <TypeGroup: Number> ProvNFile : <URLType: ProvNFile> String : <ValueType: String> TifImageFile : <URLType: TifImageFile> TxtFile : <URLType: TxtFile> UnsignedInt : <ValueType: UnsignedInt> VirtualFileSystem : <IOPlugin: VirtualFileSystem> ¶
-
parsed_value
¶ The parsed value of object instantiation of this DataType.
-
raw_value
¶ The raw value of object instantiation of this DataType. For datatypes that override value (like Deferred) this is the way to access the _value field.
-
classmethod
test
()[source]¶ Define the test for the BasePluginManager. Make sure we are not one of the base classes
-
valid
¶ A boolean flag that indicates weather or not the value assigned to this DataType is valid. This property is generally overwritten by implementation of specific DataTypes.
-
value
¶ The value of object instantiation of this DataType.
-
version
= <Version: 1.0>¶
-
-
class
fastr.datatypes.
Boolean
(value=None, format_=None)¶ Bases:
fastr.datatypes.ValueType
Datatype representing a boolean
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'A boolean value (True of False)'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Boolean.py'¶
-
module
= <module 'Boolean' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Boolean.pyc'>¶
-
value
¶ The value of object instantiation of this DataType.
-
-
class
fastr.datatypes.
DataType
(value=None, format_=None)[source]¶ Bases:
fastr.datatypes.BaseDataType
This class is the base class for all DataTypes that can hold a value.
-
__abstractmethods__
= frozenset(['__init__'])¶
-
__init__
(value=None, format_=None)[source]¶ The DataType constructor.
Parameters: - value – value to assign to the new DataType object
- format – the format used for the ValueType
Returns: new DataType object
-
__module__
= 'fastr.datatypes'¶
-
action
(name)[source]¶ This function can be overwritten by subclasses to implement certain action that should be performed. For example, the Directory DataType has an action ensure. This method makes sure the Directory exists. A Tool can indicate an action that should be called for an Output which will be called before execution.
Parameters: name (str) – name of the action to execute Returns: None
-
-
class
fastr.datatypes.
Deferred
(value=None, format_=None)[source]¶ Bases:
fastr.datatypes.DataType
-
__abstractmethods__
= frozenset([])¶
-
__init__
(value=None, format_=None)[source]¶ The Deferred constructor.
Parameters: - value – value to assign to the new DataType object
- format – This is ignore but here for compatibility
Returns: new Deferred object
-
__module__
= 'fastr.datatypes'¶
-
__repr__
()[source]¶ Returns string representation of the BaseDataType
Returns: string represenation Return type: str
-
checksum
()[source]¶ Generate a checksum for the value of this DataType
Returns: the checksum of the value Return type: str
-
job
¶
-
classmethod
lookup
(value)[source]¶ Look up the deferred target and return that object
Param: value
Returns: The value the deferred points to
Return type: Raises: - FastrKeyError – if the deferred is not available (yet)
- FastrValueError – if the value is not a valid deferrred url
-
parsed_value
¶ The value of object instantiation of this DataType.
-
provenance
¶
-
target
¶ Target object for this deferred.
Raises: - FastrKeyError – if the deferred is not available (yet)
- FastrValueError – if the value is not a valid deferrred url
-
value
¶ The value of object instantiation of this DataType.
-
-
class
fastr.datatypes.
Directory
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
DataType representing a directory.
-
__abstractmethods__
= frozenset([])¶
-
__eq__
(other)[source]¶ Directories are equal by default as long as the validatity matches.
Parameters: other (Directory) – other to compare against Returns: equality flag
-
__module__
= 'fastr.datatypes'¶
-
action
(name)[source]¶ This method makes sure the Directory exists. A Tool can indicate an action that should be called for an Output which will be called before execution.
Parameters: name (str) – name of the action to execute Returns: None
-
description
= 'A directory on the disk'¶
-
extension
= None¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Directory.py'¶
-
module
= <module 'Directory' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Directory.pyc'>¶
-
-
class
fastr.datatypes.
EnumType
(value=None, format_=None)[source]¶ Bases:
fastr.datatypes.DataType
The EnumType is the base for DataTypes that can have a value which is an option from a predefined set of possibilities (similar to an enum type in many programming languages).
-
__abstractmethods__
= frozenset([])¶
-
__init__
(value=None, format_=None)[source]¶ The EnumType constructor.
Parameters: - value – value to assign to the new EnumType object
- format – the format used for the ValueType
Returns: new EnumType object
Raises: FastrDataTypeNotInstantiableError – if not subclassed
-
__module__
= 'fastr.datatypes'¶
-
description
= 'EnumType (EnumType) is a enumerate type with options:\n\n\nEnumType can take the value of any of the option, but any other value is considered invalid.'¶
-
options
= frozenset([])¶
-
version
= <Version: 1.0>¶
-
-
class
fastr.datatypes.
FilePrefix
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'Prefix for another file'¶
-
extension
= 'prefix'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/FilePrefix.py'¶
-
module
= <module 'FilePrefix' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/FilePrefix.pyc'>¶
-
-
class
fastr.datatypes.
Float
(value=None, format_=None)¶ Bases:
fastr.datatypes.ValueType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'A floating point value'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Float.py'¶
-
module
= <module 'Float' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Float.pyc'>¶
-
value
¶ The value of object instantiation of this DataType.
-
-
class
fastr.datatypes.
ITKImageFile
(value=None)¶ Bases:
fastr.datatypes.TypeGroup
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'Text file to store point coordinates'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/ITKImageFile.py'¶
-
module
= <module 'ITKImageFile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/ITKImageFile.pyc'>¶
-
-
class
fastr.datatypes.
Int
(value=None, format_=None)¶ Bases:
fastr.datatypes.ValueType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'an integer value'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Int.py'¶
-
module
= <module 'Int' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Int.pyc'>¶
-
value
¶ The value of object instantiation of this DataType.
-
-
class
fastr.datatypes.
JsonFile
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'json file'¶
-
extension
= 'json'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/JsonFile.py'¶
-
module
= <module 'JsonFile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/JsonFile.pyc'>¶
-
-
class
fastr.datatypes.
MetaImageFile
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
checksum
()[source]¶ Return the checksum of this MetaImageFile
Returns: checksum string Return type: str
-
description
= 'Meta Image file format'¶
-
extension
= 'mhd'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/MetaImageFile.py'¶
-
module
= <module 'MetaImageFile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/MetaImageFile.pyc'>¶
-
-
class
fastr.datatypes.
NiftiImageFile
(value=None)¶ Bases:
fastr.datatypes.TypeGroup
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'NiftiTypeGroup'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/NiftiImageFile.py'¶
-
module
= <module 'NiftiImageFile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/NiftiImageFile.pyc'>¶
-
-
class
fastr.datatypes.
NiftiImageFileCompressed
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'Compressed Nifti Image File format'¶
-
extension
= 'nii.gz'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/NiftiImageFileCompressed.py'¶
-
module
= <module 'NiftiImageFileCompressed' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/NiftiImageFileCompressed.pyc'>¶
-
-
class
fastr.datatypes.
NiftiImageFileUncompressed
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'Nifti Image File format'¶
-
extension
= 'nii'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/NiftiImageFileUncompressed.py'¶
-
module
= <module 'NiftiImageFileUncompressed' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/NiftiImageFileUncompressed.pyc'>¶
-
-
class
fastr.datatypes.
Number
(value=None)¶ Bases:
fastr.datatypes.TypeGroup
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'an numeric value'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Number.py'¶
-
module
= <module 'Number' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/Number.pyc'>¶
-
-
class
fastr.datatypes.
ProvNFile
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'Provenance file in the prov N format'¶
-
extension
= 'provn'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/ProvNFile.py'¶
-
module
= <module 'ProvNFile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/ProvNFile.pyc'>¶
-
-
class
fastr.datatypes.
String
(value=None, format_=None)¶ Bases:
fastr.datatypes.ValueType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'A simple string value'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/String.py'¶
-
module
= <module 'String' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/String.pyc'>¶
-
-
class
fastr.datatypes.
TifImageFile
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'Tif Image File format'¶
-
extension
= 'tif'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/TifImageFile.py'¶
-
module
= <module 'TifImageFile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/TifImageFile.pyc'>¶
-
-
class
fastr.datatypes.
TxtFile
(value=None, format_=None)¶ Bases:
fastr.datatypes.URLType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'General text file'¶
-
extension
= 'txt'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/TxtFile.py'¶
-
module
= <module 'TxtFile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/TxtFile.pyc'>¶
-
-
class
fastr.datatypes.
TypeGroup
(value=None)[source]¶ Bases:
fastr.datatypes.BaseDataType
The TypeGroup is a special DataType that does not hold a value of its own but is used to group a number of DataTypes. For example ITK has a list of supported file formats that all tools build on ITK support. A group can be used to conveniently specify this in multiple Tools that use the same set DataTypes.
-
__abstractmethods__
= frozenset(['_members'])¶
-
__init__
(value=None)[source]¶ Dummy constructor. TypeGroups are not instantiable and cannot hold a value of its own.
Raises: FastrDataTypeNotInstantiableError – if called
-
__module__
= 'fastr.datatypes'¶
-
static
__new__
(value=None, format_=None)[source]¶ Instantiate a TypeGroup. This will for match the value to the best matching type and instantiate that. Not that the returned object will not be of type TypeGroup but one of the TypeGroup members.
-
members
¶ A descriptor that can act like a property for a class.
-
-
class
fastr.datatypes.
URLType
(value=None, format_=None)[source]¶ Bases:
fastr.datatypes.DataType
The URLType is the base for DataTypes that point to a resource somewhere else (typically a filesystem). The true value is actually the resource referenced by the value in this object.
-
__abstractmethods__
= frozenset([])¶
-
__eq__
(other)[source]¶ Test the equality of two DataType objects
Parameters: other (URLType) – the object to compare against Returns: flag indicating equality Return type: bool
-
__init__
(value=None, format_=None)[source]¶ The URLType constructor
Parameters: - value – value to assign to the new URLType
- format – the format used for the ValueType
Returns: new URLType object
-
__module__
= 'fastr.datatypes'¶
-
classmethod
content
(inval, outval=None)[source]¶ Give the contents of a URLType, this is generally useful for filetypes that consists of multiple files (e.g. AnalyzeImageFile, DICOM). The value will indicate the main file, and the contents function can determine all files that form a single data value.
Parameters: - inval – a value to figure out contents for this type
- outval – the place where the copy should point to
Returns: a list of all files part of the value (e.g. header and data file)
Return type: list
-
parsed_value
¶ The parsed value of object instantiation of this DataType.
-
valid
¶ A boolean flag that indicates weather or not the value assigned to this DataType is valid. This property is generally overwritten by implementation of specific DataTypes.
-
-
class
fastr.datatypes.
UnsignedInt
(value=None, format_=None)¶ Bases:
fastr.datatypes.ValueType
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.datatypes'¶
-
description
= 'an unsigned integer value'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/UnsignedInt.py'¶
-
module
= <module 'UnsignedInt' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/datatypes/UnsignedInt.pyc'>¶
-
value
¶ The value of object instantiation of this DataType.
-
-
class
fastr.datatypes.
ValueType
(value=None, format_=None)[source]¶ Bases:
fastr.datatypes.DataType
The ValueType is the base for DataTypes that hold simple values (not an EnumType and not a file/URL). The values is generally represented by a string.
-
__abstractmethods__
= frozenset([])¶
-
__init__
(value=None, format_=None)[source]¶ The ValueType constructor
Parameters: - value – value to assign to the new ValueType
- format – the format used for the ValueType
Returns: new ValueType object
-
__module__
= 'fastr.datatypes'¶
-
-
fastr.datatypes.
fastr_isinstance
(obj, datatype)[source]¶ Check if an object is of a specific datatype.
Parameters: - obj – Object to inspect
- datatype (tuple, BaseDataType) – The datatype(s) to check
Returns: flag indicating object is of datatype
Return type:
exceptions
Module¶
This module contains all Fastr-related Exceptions
-
exception
fastr.exceptions.
FastrAttributeError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.AttributeError
AttributeError in the fastr system
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrCannotChangeAttributeError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
Attempting to change an attribute of an object that can be set only once.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrCardinalityError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
The description of the cardinality is not valid.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrCollectorError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
Cannot collect the results from a Job because of an error
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrDataTypeFileNotReadable
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
Could not read the datatype file.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrDataTypeMismatchError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
When using a DataType as the key for the DataTypeManager, the DataTypeManager found another DataType with the same name already in the DataTypeManager. The means fastr has two version of the same DataType in the system, which should never happen!
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrDataTypeNotAvailableError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
The DataType requested is not found by the fastr system. Typically this means that no matching DataType is found in the DataTypeManager.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrDataTypeNotInstantiableError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
The base classes for DataTypes cannot be instantiated and should always be sub-classed.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrDataTypeValueError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
This value in fastr did not pass the validation specificied for its DataType, typically means that the data is missing or corrupt.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrError
(*args, **kwargs)[source]¶ Bases:
exceptions.Exception
This is the base class for all fastr related exceptions. Catching this class of exceptions should ensure a proper execution of fastr.
-
__init__
(*args, **kwargs)[source]¶ Constructor for all exceptions. Saves the caller object fullid (if found) and the file, function and line number where the object was created.
-
__module__
= 'fastr.exceptions'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
exception
fastr.exceptions.
FastrErrorInSubprocess
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrExecutionError
Encountered an error in the subprocess started by the execution script
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrExecutableNotFoundError
(executable=None, *args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrExecutionError
The executable could not be found!
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrExecutionError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
Base class for all fastr execution related errors
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrIOError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.IOError
IOError in the fastr system
-
__module__
= 'fastr.exceptions'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
exception
fastr.exceptions.
FastrImportError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.ImportError
ImportError in the fastr system
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrIndexError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.IndexError
IndexError in the fastr system
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrIndexNonexistent
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrIndexError
This is an IndexError for samples requested from a sparse data array. The sample is not there but is probably not there because of sparseness rather than being a missing sample (e.g. out of bounds).
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrKeyError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.KeyError
KeyError in the fastr system
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrLookupError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
Could not find specified object in the fastr environment.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrMountUnknownError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrKeyError
Trying to access an undefined mount
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrNetworkMismatchError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
Two interacting objects belong to different fastr network.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrNetworkUnknownError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrKeyError
Reference to a Tool that is not recognised by the fastr system. This typically means the specific id/version combination of the requested tool has not been loaded by the ToolManager.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrNoValidTargetError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrKeyError
Cannot find a valid target for the tool
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrNodeAreadyPreparedError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrStateError
A attempt is made at preparing a Node for the second time. This is not allowed as it would wipe the current execution data and cause data-loss.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrNodeNotPreparedError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrStateError
When trying to access executation data of a Node, the Node must be prepare. The Node has not been prepared by the execution, so the data is not available!
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrNodeNotValidError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrStateError
A Node is not in a valid state where it should be, typically an invalid Node is passed to the executor causing trouble.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrNotExecutableError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrExecutionError
The command invoked by subprocess is not executable on the system
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrNotImplementedError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.NotImplementedError
This function/method has not been implemented on purpose (e.g. should be overwritten in a sub-class)
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrOSError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.OSError
OSError in the fastr system
-
__module__
= 'fastr.exceptions'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
exception
fastr.exceptions.
FastrObjectUnknownError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrKeyError
Reference to a Tool that is not recognised by the fastr system. This typically means the specific id/version combination of the requested tool has not been loaded by the ToolManager.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrOptionalModuleNotAvailableError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrNotImplementedError
A optional modules for Fastr is needed for this function, but is not available on the current python installation.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrOutputValidationError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrExecutionError
An output of a Job does not pass validation
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrParentMismatchError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
Two interactive objects have different parent where they should be the same
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrPluginCapabilityNotImplemented
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrNotImplementedError
A plugin did not implement a capability that it advertised.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrPluginNotAvailable
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrKeyError
Indicates that a requested Plugin was not found on the system.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrPluginNotLoaded
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrStateError
The plugin was not successfully loaded. This means the plugin class cannot be instantiated.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrResultFileNotFound
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrExecutionError
Could not found the result file of job that finished. This means the executionscript process was killed during interruption. Generally this means a scheduler killed it because of resource shortage.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrSerializationError
(message, serializer, original_exception=None)[source]¶ Bases:
fastr.exceptions.FastrError
The serialization encountered a serious problem
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrSerializationIgnoreDefaultError
(message, serializer, original_exception=None)[source]¶ Bases:
fastr.exceptions.FastrSerializationError
The value and default are both None, so the value should not be serialized.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrSerializationInvalidDataError
(message, serializer, original_exception=None)[source]¶ Bases:
fastr.exceptions.FastrSerializationError
Encountered data to serialize that is invalid given the serialization schema.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrSerializationMethodError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrKeyError
The desired serialization method does not exist.
-
__module__
= 'fastr.exceptions'¶
-
Bases:
fastr.exceptions.FastrKeyError
Could not find the Sink data for the desire sink.
-
exception
fastr.exceptions.
FastrSizeInvalidError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
The given size cannot be valid.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrSizeMismatchError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
The size of two object in fastr is not matching where it should.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrSizeUnknownError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
The size of object is not (yet) known and only a theoretical estimate is available at the moment.
-
__module__
= 'fastr.exceptions'¶
-
Bases:
fastr.exceptions.FastrKeyError
Could not find the Source data for the desire source.
-
exception
fastr.exceptions.
FastrStateError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
An object is in an invalid/unexpected state.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrSubprocessNotFinished
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrExecutionError
Encountered an error before the subprocess call by the execution script was properly finished.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrToolNotAvailableError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
- The tool used is not available on the current platform (OS and architecture
- combination) and cannot be used.
-
__module__
= 'fastr.exceptions'¶
-
exception
fastr.exceptions.
FastrToolTargetNotFound
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
Could not determine the location of the tools target binary/script. The tool cannot be used.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrToolUnknownError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrKeyError
Reference to a Tool that is not recognised by the fastr system. This typically means the specific id/version combination of the requested tool has not been loaded by the ToolManager.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrTypeError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.TypeError
TypeError in the fastr system
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrUnknownURLSchemeError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrKeyError
Fastr encountered a data URL with a scheme that was not recognised by the IOPlugin manager.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrValueError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrError
,exceptions.ValueError
ValueError in the fastr system
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrVersionInvalidError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrValueError
The string representation of the version is malformatted.
-
__module__
= 'fastr.exceptions'¶
-
-
exception
fastr.exceptions.
FastrVersionMismatchError
(*args, **kwargs)[source]¶ Bases:
fastr.exceptions.FastrValueError
There is a mismatch between different parts of the Fastr environment and integrity is compromised.
-
__module__
= 'fastr.exceptions'¶
-
plugins
Module¶
The plugins module holds all plugins loaded by Fastr. It is empty on start and gets filled by the BasePluginManager
-
class
fastr.plugins.
BlockingExecution
(finished_callback=None, cancelled_callback=None, status_callback=None)¶ Bases:
fastr.execution.executionpluginmanager.ExecutionPlugin
The blocking execution plugin is a special plugin which is meant for debug purposes. It will not queue jobs but immediately execute them inline, effectively blocking fastr until the Job is finished. It is the simplest execution plugin and can be used as a template for new plugins or for testing purposes.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/blockingexecution.py'¶
-
module
= <module 'blockingexecution' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/blockingexecution.pyc'>¶
-
-
class
fastr.plugins.
CommaSeperatedValueFile
¶ Bases:
fastr.core.ioplugin.IOPlugin
The CommaSeperatedValueFile an expand-only type of IOPlugin. No URLs can actually be fetched, but it can expand a single URL into a larger amount of URLs.
The
csv://
URL is avfs://
URL with a number of query variables available. The URL mount and path should point to a valid CSV file. The query variable then specify what column(s) of the file should be used.The following variable can be set in the query:
variable usage value the column containing the value of interest, can be int for index or string for key id the column containing the sample id (optional) header indicates if the first row is considered the header, can be true
orfalse
(optional)delimiter the delimiter used in the csv file (optional) quote the quote character used in the csv file (optional) reformat a reformatting string so that value = reformat.format(value)
(used before relative_path)relative_path indicates the entries are relative paths (for files), can be true
orfalse
(optional)The header is by default
false
if the neither thevalue
andid
are set as a string. If either of these are a string, the header is required to define the column names and it automatically is assumedtrue
The delimiter and quota characters of the file should be detected automatically using the
Sniffer
, but can be forced by setting them in the URL.Example of valid
csv
URLs:# Use the first column in the file (no header row assumed) csv://mount/some/dir/file.csv?value=0 # Use the images column in the file (first row is assumed header row) csv://mount/some/dir/file.csv?value=images # Use the segmentations column in the file (first row is assumed header row) # and use the id column as the sample id csv://mount/some/dir/file.csv?value=segmentations&id=id # Use the first column as the id and the second column as the value # and skip the first row (considered the header) csv://mount/some/dir/file.csv?value=1&id=0&header=true # Use the first column and force the delimiter to be a comma csv://mount/some/dir/file.csv?value=0&delimiter=,
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/commaseperatedvaluefile.py'¶
-
module
= <module 'commaseperatedvaluefile' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/commaseperatedvaluefile.pyc'>¶
-
scheme
= 'csv'¶
-
-
class
fastr.plugins.
CrossValidation
¶ Bases:
flowinterface.FlowPlugin
Advanced flow plugin that generated a cross-validation data flow. The node need an input with data and an input number of folds. Based on that the outputs test and train will be supplied with a number of data sets.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/flowplugins/crossvalidation.py'¶
-
module
= <module 'crossvalidation' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/flowplugins/crossvalidation.pyc'>¶
-
-
class
fastr.plugins.
DRMAAExecution
(finished_callback=None, cancelled_callback=None, status_callback=None)¶ Bases:
fastr.execution.executionpluginmanager.ExecutionPlugin
A DRMAA execution plugin to execute Jobs on a Grid Engine cluster. It uses a configuration option for selecting the queue to submit to. It uses the python
drmaa
package.Note
To use this plugin, make sure the
drmaa
package is installed and that the execution is started on an SGE submit host with DRMAA libraries installed.Note
This plugin is at the moment tailored to SGE, but it should be fairly easy to make different subclasses for different DRMAA supporting systems.
-
CANCELS_DEPENDENCIES
= False¶
-
SUPPORTS_CANCEL
= True¶
-
SUPPORTS_DEPENDENCY
= True¶
-
SUPPORTS_HOLD_RELEASE
= True¶
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
configuration_fields
= {'drmaa_queue': (<type 'str'>, 'week', 'The default queue to use for jobs send to the scheduler')}¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/drmaaexecution.py'¶
-
module
= <module 'drmaaexecution' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/drmaaexecution.pyc'>¶
-
-
class
fastr.plugins.
FastrInterface
(id_, document)¶ Bases:
fastr.core.interface.Interface
The default Interface for fastr. For the command-line Tools as used by fastr.
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'FastrInterface.schema.json'¶
-
__getstate__
()[source]¶ Get the state of the FastrInterface object.
Returns: state of interface Return type: dict
-
__module__
= 'fastr.plugins'¶
-
check_input_id
(id_)[source]¶ Check if an id for an object is valid and unused in the Tool. The method will always returns True if it does not raise an exception.
Parameters: id (str) – the id to check
Returns: True
Raises: - FastrValueError – if the id is not correctly formatted
- FastrValueError – if the id is already in use
-
check_output_id
(id_)[source]¶ Check if an id for an object is valid and unused in the Tool. The method will always returns True if it does not raise an exception.
Parameters: id (str) – the id to check
Returns: True
Raises: - FastrValueError – if the id is not correctly formatted
- FastrValueError – if the id is already in use
-
collector_plugin_type
¶ alias of
CollectorPlugin
-
collectors
= CollectorPluginManager [37m[42m[1mLoaded[0m json : <CollectorPlugin: JsonCollector> [37m[42m[1mLoaded[0m path : <CollectorPlugin: PathCollector> [37m[42m[1mLoaded[0m stdout : <CollectorPlugin: StdoutCollector>¶
-
execute
(target, payload)[source]¶ Execute the interface using a specific target and payload (containing a set of values for the arguments)
Parameters: Returns: result of the execution
Return type:
-
expanding
¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/interfaceplugins/fastrinterface.py'¶
-
get_arguments
(values)[source]¶ Get the argument list for this interface
Returns: return list of arguments
-
get_specials
(payload, output, cardinality_nr)[source]¶ Get special attributes. Returns tuples for specials, inputs and outputs that are used for formatting substitutions.
Parameters: - output – Output for which to get the specials
- cardinality_nr (int) – the cardinality number
-
inputs
¶
-
module
= <module 'fastrinterface' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/interfaceplugins/fastrinterface.pyc'>¶
-
outputs
¶
-
-
class
fastr.plugins.
FileSystem
¶ Bases:
fastr.core.ioplugin.IOPlugin
The FileSystem plugin is create to handle
file://
type or URLs. This is generally not a good practice, as this is not portable over between machines. However, for test purposes it might be useful.The URL scheme is rather simple:
file://host/path
(see wikipedia for details)We do not make use of the
host
part and at the moment only support localhost (just leave the host empty) leading tofile:///
URLs.Warning
This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form
c:/
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
fetch_url
(inurl, outpath)[source]¶ Fetch the files from the file.
Parameters: - inurl – url to the item in the data store, starts with
file://
- outpath – path where to store the fetch data locally
- inurl – url to the item in the data store, starts with
-
fetch_value
(inurl)[source]¶ Fetch a value from an external file file.
Parameters: inurl – url of the value to read Returns: the fetched value
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/filesystem.py'¶
-
module
= <module 'filesystem' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/filesystem.pyc'>¶
-
path_to_url
(path, mountpoint=None)[source]¶ Construct an url from a given mount point and a relative path to the mount point.
-
put_url
(inpath, outurl)[source]¶ Put the files to the external data store.
Parameters: - inpath – path of the local data
- outurl – url to where to store the data, starts with
file://
-
put_value
(value, outurl)[source]¶ Put the value in the external data store.
Parameters: - value – value to store
- outurl – url to where to store the data, starts with
file://
-
scheme
= 'file'¶
-
-
class
fastr.plugins.
FlowInterface
(id_, document)¶ Bases:
fastr.core.interface.Interface
The Interface use for AdvancedFlowNodes to create the advanced data flows that are not implemented in the fastr. This allows nodes to implement new data flows using the plugin system.
The definition of
FlowInterfaces
are very similar to the defaultFastrInterfaces
.Note
A flow interface should be using a specific FlowPlugin
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'FastrInterface.schema.json'¶
-
__getstate__
()[source]¶ Get the state of the FastrInterface object.
Returns: state of interface Return type: dict
-
__module__
= 'fastr.plugins'¶
-
expanding
¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/interfaceplugins/flowinterface.py'¶
-
flow_plugin_type
¶ alias of
FlowPlugin
-
flow_plugins
= FlowPluginManager [37m[42m[1mLoaded[0m CrossValidation : <FlowPlugin: CrossValidation>¶
-
inputs
¶
-
module
= <module 'flowinterface' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/interfaceplugins/flowinterface.pyc'>¶
-
outputs
¶
-
-
class
fastr.plugins.
LinearExecution
(finished_callback=None, cancelled_callback=None, status_callback=None)¶ Bases:
fastr.execution.executionpluginmanager.ExecutionPlugin
An execution engine that has a background thread that executes the jobs in order. The queue is a simple FIFO queue and there is one worker thread that operates in the background. This plugin is meant as a fallback when other plugins do not function properly. It does not multi-processing so it is safe to use in environments that do no support that.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/linearexecution.py'¶
-
module
= <module 'linearexecution' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/linearexecution.pyc'>¶
-
-
class
fastr.plugins.
NipypeInterface
(id_, nipype_cls=None, document=None)¶ Bases:
fastr.core.interface.Interface
Experimental interfaces to using
nipype
interfaces directly in fastr tools, only using a simple reference.To create a tool using a nipype interface just create an interface with the correct type and set the
nipype
argument to the correct class. For example in an xml tool this would become:<interface class="NipypeInterface"> <nipype_class>nipype.interfaces.elastix.Registration</nipype_class> </interface>
Note
To use these interfaces
nipype
should be installed on the system.Warning
This interface plugin is basically functional, but highly experimental!
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
execute
(target, payload)[source]¶ Execute the interface using a specific target and payload (containing a set of values for the arguments)
Parameters: Returns: result of the execution
Return type:
-
expanding
¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/interfaceplugins/nipypeinterface.py'¶
-
inputs
¶
-
module
= <module 'nipypeinterface' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/interfaceplugins/nipypeinterface.pyc'>¶
-
outputs
¶
-
-
class
fastr.plugins.
Null
¶ Bases:
fastr.core.ioplugin.IOPlugin
The Null plugin is create to handle
null://
type or URLs. These URLs are indicating the sink should not do anything. The data is not written to anywhere. Besides the scheme, the rest of the URL is ignored.-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/null.py'¶
-
module
= <module 'null' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/null.pyc'>¶
-
put_url
(inpath, outurl)[source]¶ Put the files to the external data store.
Parameters: - inpath – path of the local data
- outurl – url to where to store the data, starts with
file://
-
put_value
(value, outurl)[source]¶ Put the value in the external data store.
Parameters: - value – value to store
- outurl – url to where to store the data, starts with
file://
-
scheme
= 'null'¶
-
-
class
fastr.plugins.
ProcessPoolExecution
(finished_callback=None, cancelled_callback=None, status_callback=None, nr_of_workers=None)¶ Bases:
fastr.execution.executionpluginmanager.ExecutionPlugin
A local execution plugin that uses multiprocessing to create a pool of worker processes. This allows fastr to execute jobs in parallel with true concurrency. The number of workers can be specified in the fastr configuration, but the default amount is the
number of cores - 1
with a minimum of1
.Warning
The ProcessPoolExecution does not check memory requirements of jobs and running many workers might lead to memory starvation and thus an unresponsive system.
-
__abstractmethods__
= frozenset([])¶
-
__init__
(finished_callback=None, cancelled_callback=None, status_callback=None, nr_of_workers=None)[source]¶
-
__module__
= 'fastr.plugins'¶
-
configuration_fields
= {'process_pool_worker_number': (<type 'int'>, 3, 'Number of workers to use in a process pool')}¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/processpoolexecution.py'¶
-
module
= <module 'processpoolexecution' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/processpoolexecution.pyc'>¶
-
-
class
fastr.plugins.
RQExecution
(finished_callback=None, cancelled_callback=None, status_callback=None)¶ Bases:
fastr.execution.executionpluginmanager.ExecutionPlugin
A execution plugin based on Redis Queue. Fastr will submit jobs to the redis queue and workers will peel the jobs from the queue and process them.
This system requires a running redis database and the database url has to be set in the fastr configuration.
Note
This execution plugin required the
redis
andrq
packages to be installed before it can be loaded properly.-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
configuration_fields
= {'rq_queue': (<type 'str'>, 'default', 'The redis queue to use'), 'rq_host': (<type 'str'>, 'redis://localhost:6379/0', 'The url of the redis serving the redis queue')}¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/rqexecution.py'¶
-
module
= <module 'rqexecution' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/executionplugins/rqexecution.pyc'>¶
-
-
class
fastr.plugins.
Reference
¶ Bases:
fastr.core.ioplugin.IOPlugin
The Reference plugin is create to handle
ref://
type or URLs. These URLs are to make the sink just write a simple reference file to the data. The reference file contains the DataType and the value so the result can be reconstructed. It for files just leaves the data on disk by reference. This plugin is not useful for production, but is used for testing purposes.-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/reference.py'¶
-
module
= <module 'reference' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/reference.pyc'>¶
-
push_sink_data
(value, outurl, datatype=None)[source]¶ Write out the sink data from the inpath to the outurl.
Parameters: Returns: None
-
scheme
= 'ref'¶
-
-
class
fastr.plugins.
VirtualFileSystem
¶ Bases:
fastr.core.ioplugin.IOPlugin
The virtual file system class. This is an IOPlugin, but also heavily used internally in fastr for working with directories. The VirtualFileSystem uses the
vfs://
url scheme.A typical virtual filesystem url is formatted as
vfs://mountpoint/relative/dir/from/mount.ext
Where the
mountpoint
is defined in the Config file. A list of the currently known mountpoints can be found in thefastr.config
object>>> fastr.config.mounts {'example_data': '/home/username/fastr-feature-documentation/fastr/fastr/examples/data', 'home': '/home/username/', 'tmp': '/home/username/FastrTemp'}
This shows that a url with the mount
home
such asvfs://home/tempdir/testfile.txt
would be translated into/home/username/tempdir/testfile.txt
.There are a few default mount points defined by Fastr (that can be changed via the config file).
mountpoint default location home the users home directory ( expanduser('~/')
)tmp the fastr temprorary dir, defaults to tempfile.gettempdir()
example_data the fastr example data directory, defaults $FASTRDIR/example/data
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
static
copy_file_dir
(inpath, outpath)[source]¶ Helper function, copies a file or directory not caring what the inpath actually is
Parameters: - inpath – path of the things to be copied
- outpath – path of the destination
Returns: the result of shutil.copy2 or shutil.copytree (depending on inpath pointing to a file or directory)
-
expand_url
(url)[source]¶ Try to expand the url. For vfs with will return the original url.
Parameters: url – url to expand Returns: the expanded url (same as url)
-
fetch_url
(inurl, outpath)[source]¶ Fetch the files from the vfs.
Parameters: - inurl – url to the item in the data store, starts with
vfs://
- outpath – path where to store the fetch data locally
- inurl – url to the item in the data store, starts with
-
fetch_value
(inurl)[source]¶ Fetch a value from an external vfs file.
Parameters: inurl – url of the value to read Returns: the fetched value
-
path_to_url
(path, mountpoint=None, scheme=None)[source]¶ Construct an url from a given mount point and a relative path to the mount point.
Parameters: path (str) – the path to find the url for Mountpoint str: mountpoint the url should be under Returns: url of the
-
put_url
(inpath, outurl)[source]¶ Put the files to the external data store.
Parameters: - inpath – path of the local data
- outurl – url to where to store the data, starts with
vfs://
-
put_value
(value, outurl)[source]¶ Put the value in the external data store.
Parameters: - value – value to store
- outurl – url to where to store the data, starts with
vfs://
-
scheme
¶
-
url_to_path
(url, scheme=None)[source]¶ Get the path to a file from a vfs url
Parameters: url (str) – url to get the path for
Returns: the matching path
Return type: Raises: - FastrMountUnknownError – if the mount in url is unknown
- FastrUnknownURLSchemeError – if the url scheme is not correct
Example (the mountpoint tmp points to /tmp):
>>> fastr.vfs.url_to_path('vfs://tmp/file.ext') '/tmp/file.ext'
-
-
class
fastr.plugins.
VirtualFileSystemRegularExpression
¶ Bases:
fastr.core.ioplugin.IOPlugin
The VirtualFileSystemValueList an expand-only type of IOPlugin. No URLs can actually be fetched, but it can expand a single URL into a larger amount of URLs.
A
vfsregex://
URL is a vfs URL that can contain regular expressions on every level of the path. The regular expressions follow there module
definitions.An example of a valid URLs would be:
vfsregex://tmp/network_dir/.*/.*/__fastr_result__.pickle.gz vfsregex://tmp/network_dir/nodeX/(?P<id>.*)/__fastr_result__.pickle.gz
The first URL would result in all the
__fastr_result__.pickle.gz
in the working directory of a Network. The second URL would only result in the file for a specific node (nodeX), but by adding the named groupid
using(?P<id>.*)
the sample id of the data is automatically set to that group (see Regular Expression Syntax under the special characters for more info on named groups in regular expression).Concretely if we would have a directory
vfs://mount/somedir
containing:image_1/Image.nii image_2/image.nii image_3/anotherimage.nii image_5/inconsistentnamingftw.nii
we could match these files using
vfsregex://mount/somedir/(?P<id>image_\d+)/.*\.nii
which would result in the following source data after expanding the URL:{'image_1': 'vfs://mount/somedir/image_1/Image.nii', 'image_2': 'vfs://mount/somedir/image_2/image.nii', 'image_3': 'vfs://mount/somedir/image_3/anotherimage.nii', 'image_5': 'vfs://mount/somedir/image_5/inconsistentnamingftw.nii'}
Showing the power of this regular expression filtering. Also it shows how the ID group from the URL can be used to have sensible sample ids.
Warning
due to the nature of regexp on multiple levels, this method can be slow when having many matches on the lower level of the path (because the tree of potential matches grows) or when directories that are parts of the path are very large.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/virtualfilesystemregularexpression.py'¶
-
module
= <module 'virtualfilesystemregularexpression' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/virtualfilesystemregularexpression.pyc'>¶
-
scheme
= 'vfsregex'¶
-
-
class
fastr.plugins.
VirtualFileSystemValueList
¶ Bases:
fastr.core.ioplugin.IOPlugin
The VirtualFileSystemValueList an expand-only type of IOPlugin. No URLs can actually be fetched, but it can expand a single URL into a larger amount of URLs. A
vfslist://
URL basically is a url that points to a file using vfs. This file then contains a number lines each containing another URL.If the contents of a file
vfs://mount/some/path/contents
would be:vfs://mount/some/path/file1.txt vfs://mount/some/path/file2.txt vfs://mount/some/path/file3.txt vfs://mount/some/path/file4.txt
Then using the URL
vfslist://mount/some/path/contents
as source data would result in the four files being pulled.Note
The URLs in a vfslist file do not have to use the
vfs
scheme, but can use any scheme known to the Fastr system.-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/virtualfilesystemvaluelist.py'¶
-
module
= <module 'virtualfilesystemvaluelist' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/virtualfilesystemvaluelist.pyc'>¶
-
scheme
= 'vfslist'¶
-
-
class
fastr.plugins.
XNATStorage
¶ Bases:
fastr.core.ioplugin.IOPlugin
Warning
As this IOPlugin is under development, it has not been thoroughly tested.
The XNATStorage plugin is an IOPlugin that can download data from and upload data to an XNAT server. It uses its own
xnat://
URL scheme. This is a scheme specific for this plugin and though it looks somewhat like the XNAT rest interface, a different type or URL.Data resources can be access directly by a data url:
xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/experiment001/scans/T1/resources/DICOM xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/*_BRAIN/scans/T1/resources/DICOM
In the second URL you can see a wildcard being used. This is possible at long as it resolves to exactly one item.
The
id
query element will change the field from the default experiment to subject and thelabel
query element sets the use of the label as the fastr id (instead of the XNAT id) toTrue
(the default isFalse
)To disable
https
transport and usehttp
instead the query string can be modified to addinsecure=true
. This will make the plugin send requests overhttp
:xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/*_BRAIN/scans/T1/resources/DICOM?insecure=true
For sinks it is import to know where to save the data. Sometimes you want to save data in a new assessor/resource and it needs to be created. To allow the Fastr sink to create an object in XNAT, you have to supply the type as a query parameter:
xnat://xnat.bmia.nl/data/archive/projects/sandbox/subjects/S01/experiments/_BRAIN/assessors/test_assessor/resources/IMAGE/files/image.nii.gz?resource_type=xnat:resourceCatalog&assessor_type=xnat:qcAssessmentData
Valid options are: subject_type, experiment_type, assessor_type, scan_type, and resource_type.
If you want to do a search where multiple resources are returned, it is possible to use a search url:
xnat://xnat.example.com/search?projects=sandbox&subjects=subject[0-9][0-9][0-9]&experiments=*_BRAIN&scans=T1&resources=DICOM
This will return all DICOMs for the T1 scans for experiments that end with _BRAIN that belong to a subjectXXX where XXX is a 3 digit number. By default the ID for the samples will be the experiment XNAT ID (e.g. XNAT_E00123). The wildcards that can be the used are the same UNIX shell-style wildcards as provided by the module
fnmatch
.It is possible to change the id to a different fields id or label. Valid fields are project, subject, experiment, scan, and resource:
xnat://xnat.example.com/search?projects=sandbox&subjects=subject[0-9][0-9][0-9]&experiments=*_BRAIN&scans=T1&resources=DICOM&id=subject&label=true
The following variables can be set in the search query:
variable default usage projects *
The project(s) to select, can contain wildcards (see fnmatch
)subjects *
The subject(s) to select, can contain wildcards (see fnmatch
)experiments *
The experiment(s) to select, can contain wildcards (see fnmatch
)scans *
The scan(s) to select, can contain wildcards (see fnmatch
)resources *
The resource(s) to select, can contain wildcards (see fnmatch
)id experiment
What field to use a the id, can be: project, subject, experiment, scan, or resource label false
Indicate the XNAT label should be used as fastr id, options true
orfalse
insecure false
Change the url scheme to be used to http instead of https regex false
Change search to use regex re.match()
instead of fnmatch for matchingFor storing credentials the
.netrc
file can be used. This is a common way to store credentials on UNIX systems. It is required that the file is only accessible by the owner only or aNetrcParseError
will be raised. A netrc file is really easy to create, as its entries look like:machine xnat.example.com login username password secret123
See the
netrc module
or the GNU inet utils website for more information about the.netrc
file.Note
On windows the location of the netrc file is assumed to be
os.path.expanduser('~/_netrc')
. The leading underscore is because windows does not like filename starting with a dot.Note
For scan the label will be the scan type (this is initially the same as the series description, but can be updated manually or the XNAT scan type cleanup).
Warning
labels in XNAT are not guaranteed to be unique, so be careful when using them as the sample ID.
For background on XNAT, see the XNAT API DIRECTORY for the REST API of XNAT.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.plugins'¶
-
fetch_url
(inurl, outpath)[source]¶ Get the file(s) or values from XNAT.
Parameters: - inurl – url to the item in the data store
- outpath – path where to store the fetch data locally
-
filename
= '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/xnatstorage.py'¶
-
module
= <module 'xnatstorage' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/1.2.2/lib/python2.7/site-packages/fastr-1.2.2-py2.7.egg/fastr/resources/plugins/ioplugins/xnatstorage.pyc'>¶
-
put_url
(inpath, outurl)[source]¶ Upload the files to the XNAT storage
Parameters: - inpath – path to the local data
- outurl – url to where to store the data in the external data store.
-
scheme
= 'xnat'¶
-
server
¶
-
xnat
¶
-
-
fastr.plugins.
json
¶ alias of
JsonCollector
-
fastr.plugins.
path
¶ alias of
PathCollector
-
fastr.plugins.
stdout
¶ alias of
StdoutCollector
version
Module¶
This module keeps track of the version of the currently used Fastr framework. It can check its version from mercurial or a saved file
-
fastr.version.
get_base_version
()[source]¶ Get the version from the top-level version file
Returns: the version Rtype str:
-
fastr.version.
get_hg_info
()[source]¶ Read information about the current mercurial branch and revision
Returns: tuple containing head revision and branch
Subpackages¶
- core Package
core
Packagebasemanager
Modulebaseplugin
Moduledatatypemanager
Moduledimension
Moduleinputoutput
Moduleinterface
Moduleioplugin
Modulelink
Modulenetwork
Modulenetworkmanager
Modulenode
Moduleobjectmanager
Modulepluginmanager
Moduleprovenance
Modulesamples
Moduleserializable
Moduletarget
Moduletool
Moduletoolmanager
Moduleupdateable
Moduleversion
Modulevfs
Module- Subpackages
- data Package
- examples Package
- execution Package
- resources Package
resources
Package- Subpackages
- datatypes Package
datatypes
PackageAnalyzeImageFile
ModuleBoolean
ModuleDirectory
ModuleFilePrefix
ModuleFloat
ModuleITKImageFile
ModuleInt
ModuleJsonFile
ModuleMetaImageFile
ModuleNiftiImageFile
ModuleNiftiImageFileCompressed
ModuleNiftiImageFileUncompressed
ModuleNumber
ModuleProvNFile
ModuleString
ModuleTifImageFile
ModuleTxtFile
ModuleUnsignedInt
Module
- tools Package
- datatypes Package
- test Package
- utils Package
utils
Packagechecksum
Moduleclassproperty
Moduleclear_pycs
Modulecompare
Moduledicteq
Modulegettools
Moduleiohelpers
Modulejsonschemaparser
Modulemultiprocesswrapper
Modulepim_publisher
Moduleprocutils
Modulerest_generation
Moduleschematotable
Modulesysinfo
Moduleverify
Modulexmltodict
Module- Subpackages
- web Package