core Package¶
core
Package¶
This module contains all of the core components of fastr. It has the classes to create networks and work with them.
basemanager
Module¶
This module contains the core class for all managers
-
class
fastr.core.basemanager.
BaseManager
(path=None, recursive=False)[source]¶ Bases:
_abcoll.MutableMapping
- Baseclass for a Manager, subclasses needs to override the following methods:
- BaseManager._item_extension, BaseManager._load_item()
-
_item_extension
()¶ Abstract property that sets the extension of the files to be loaded by the BaseManager. When scanning for items, only files with this extension will be loaded.
Returns: desired extension Return type: str Raises: FastrNotImplementedError – if property not reimplemented in subclass
-
_load_item
(filepath, namespace)[source]¶ Abstract method to load an item of the BaseManager. This function is not implemented and needs to be reimplemented by a subclass.
Parameters: Returns: the loaded item
Raises: FastrNotImplementedError – if called without being reimplemented by a subclass
-
__abstractmethods__
= frozenset(['_load_item', '_item_extension'])¶
-
__delitem__
(key)[source]¶ Remove item from the BaseManager
Parameters: key – key of the item to remove Returns: None Raises: FastrKeyError – if the key is not found in the BaseManager
-
__getitem__
(key)[source]¶ Retrieve item from BaseManager
Parameters: key – the key of the item to retrieve Returns: the value indicated by the key Raises: FastrKeyError – if the key is not found in the BaseManager
-
__init__
(path=None, recursive=False)[source]¶ The BaseManager constructor
Parameters: Returns: the newly created BaseManager
Return type:
-
__iter__
()[source]¶ Get an iterator from the BaseManager. The iterator will iterate over the keys of the BaseManager.
Returns: the iterator Return type: dictionary-keyiterator
-
__keytransform__
(key)[source]¶ Identity transform for the keys. This function can be reimplemented by a subclass to implement a different key transform.
Parameters: key – key to transform Returns: the transformed key (in this case the same key as inputted)
-
__len__
()[source]¶ Return the number of items in the BaseManager
Returns: number of items in the BaseManager Return type: int
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.basemanager'¶
-
__repr__
()[source]¶ Convert the BaseManager to a representation string.
Returns: Representation string Return type: str
-
__setitem__
(key, value)[source]¶ Set item in the BaseManager
Parameters: - key – the key of the item to store
- value – the value of the item to store
Returns: None
-
data
¶ The actual data dict underlying this Manager
-
match_filename
(filename)[source]¶ Check if the filename matches the pattern the manager expects.
Parameters: filename – filename to match Returns: flag indicating that the filename matches
baseplugin
Module¶
The base class for all Plugins in the fastr system
-
class
fastr.core.baseplugin.
BasePlugin
[source]¶ Bases:
object
Base class for Plugins in the fastr system.
-
__abstractmethods__
= frozenset([])¶
-
__dict__
= dict_proxy({'status': <fastr.utils.classproperty.ClassPropertyDescriptor object>, '__module__': 'fastr.core.baseplugin', '__abstractmethods__': frozenset([]), '__metaclass__': <class 'fastr.core.baseplugin.PluginMeta'>, '_abc_negative_cache': <_weakrefset.WeakSet object>, '__str__': <function __str__>, '_instantiate': False, 'module': None, '__dict__': <attribute '__dict__' of 'BasePlugin' objects>, 'fullid': <fastr.utils.classproperty.ClassPropertyDescriptor object>, '_source_code': None, 'source_code': <fastr.utils.classproperty.ClassPropertyDescriptor object>, '__weakref__': <attribute '__weakref__' of 'BasePlugin' objects>, 'id': <fastr.utils.classproperty.ClassPropertyDescriptor object>, '__init__': <function __init__>, '_abc_cache': <_weakrefset.WeakSet object>, 'register_configuration': <classmethod object>, 'status_message': <fastr.utils.classproperty.ClassPropertyDescriptor object>, 'instantiate': <fastr.utils.classproperty.ClassPropertyDescriptor object>, 'configuration_fields': <fastr.utils.classproperty.ClassPropertyDescriptor object>, '_abc_registry': <_weakrefset.WeakSet object>, 'cleanup': <function cleanup>, '_abc_negative_cache_version': 34, '__repr__': <function __repr__>, 'set_status': <classmethod object>, 'test': <classmethod object>, '_status': (<PluginState.uninitialized: '\x1b[46mUnInitialized\x1b[0m'>, 'Plugin object created', None), 'set_code': <classmethod object>, '__doc__': '\n Base class for Plugins in the fastr system.\n '})¶
-
__init__
()[source]¶ The BasePlugin constructor.
Returns: the created plugin Return type: BasePlugin Raises: FastrPluginNotLoaded – if the plugin did not load correctly
-
__metaclass__
¶ alias of
PluginMeta
-
__module__
= 'fastr.core.baseplugin'¶
-
__str__
()[source]¶ Creare string representation of the plugin.
Returns: string represenation Return type: str
-
__weakref__
¶ list of weak references to the object (if defined)
-
cleanup
()[source]¶ Perform any cleanup action needed when the plugin use ended. This can be closing files/streams etc.
-
configuration_fields
= {}¶
-
fullid
= 'fastr://plugins/BasePlugin'¶
-
id
= 'BasePlugin'¶
-
instantiate
= False¶
-
module
= None¶
-
classmethod
register_configuration
()[source]¶ Register and test the configuation fields of the plugin
-
classmethod
set_code
(source_code)[source]¶ Set the filename and source code of the plugin
Parameters: source_code (str) – the source code of the plugin
-
classmethod
set_status
(status, message, exception=None)[source]¶ Update the status of the plugin
Parameters:
-
source_code
¶ A descriptor that can act like a property for a class.
-
status_message
= 'Plugin object created'¶
-
-
class
fastr.core.baseplugin.
Plugin
[source]¶ Bases:
fastr.core.baseplugin.BasePlugin
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.baseplugin'¶
-
-
class
fastr.core.baseplugin.
PluginMeta
[source]¶ Bases:
abc.ABCMeta
Meta class for the BasePlugin.
-
__module__
= 'fastr.core.baseplugin'¶
-
-
class
fastr.core.baseplugin.
PluginState
[source]¶ Bases:
enum.Enum
Plugin status Enum.
-
__format__
(format_spec)¶
-
__module__
= 'fastr.core.baseplugin'¶
-
static
__new__
(value)¶
-
__reduce_ex__
(proto)¶
-
__repr__
()¶
-
__str__
()¶
-
failed
= '\x1b[37m\x1b[41m\x1b[1mFailed\x1b[0m'¶
-
loaded
= '\x1b[37m\x1b[42m\x1b[1mLoaded\x1b[0m'¶
-
preload
= '\x1b[102mPreLoad\x1b[0m'¶
-
uninitialized
= '\x1b[46mUnInitialized\x1b[0m'¶
-
unloaded
= '\x1b[46mUnLoaded\x1b[0m'¶
-
datatypemanager
Module¶
This module manages datatypes. These datatypes are python classes generated from the XML/JSON datatype files.
-
class
fastr.core.datatypemanager.
DataTypeManager
[source]¶ Bases:
fastr.core.pluginmanager.BasePluginManager
The DataTypeManager hold a mapping of all DataTypes in the fast system and can create new DataTypes from files/data structures.
-
__abstractmethods__
= frozenset([])¶
-
__init__
()[source]¶ The DataTypeManager constructor will create a new DataTypeManager and populate it with all DataTypes it can find in the paths set in
fastr.config.types_path
.Returns: the created DataTypeManager
-
__keytransform__
(key)[source]¶ Key transformation for this mapping. The key transformation allows indexing by both the DataType name as well as the DataType it self.
Parameters: key (fastr.datatypes.BaseDataType or str) – The name of the requested datatype or the datatype itself Returns: The requested datatype
-
__module__
= 'fastr.core.datatypemanager'¶
-
create_enumtype
(type_id, options, name=None)[source]¶ Create a python class based on an XML file. This function return a completely functional python class based on the contents of a DataType XML file.
Such a class will be of type EnumType.
Parameters: - type_id (str) – the id of the new class
- options (iterable) – an iterable of options, each option should be str
Returns: the newly created subclass of EnumType
Raises: FastrTypeError – if the options is not an iterable of str
-
fullid
¶ The fullid of the datatype manager
-
get_type
(name)[source]¶ Read a type given a typename. This will scan all directories in types_path and attempt to load the newest version of the DataType.
Parameters: name (str) – Name of the datatype that should be imported in the system Returns: the datatype with the requested name, or None if datatype is not found Note
If type is already in TypeManager it will not load anything and return the already loaded version.
-
guess_type
(value, exists=True, options=None, preferred=None)[source]¶ Guess the DataType based on a value str.
Parameters: - value (str) – the value to guess the type for
- options (TypeGroup, DataType or tuple of DataTypes) – The options that are allowed to be guessed from
- extists (bool) – Indicate the value exists (if file) and can be checked for validity, if false skip validity check
- preferred (iterable) – An iterable of preferred types in case multiple types match.
Returns: The resulting DataType or None if no match was found
Raises: FastrTypeError – if the options argument is of the wrong type
The function will first create a list of all candidate DataTypes. Subsequently, it will check for each candidate if the value would valid. If there are multiple matches, the config value for preferred types is consulted to break the ties. If non of the DataTypes are in the preferred types list, a somewhat random DataType will be picked as the most optimal result.
-
has_type
(name)[source]¶ Check if the datatype with requested name exists
Parameters: name (str) – the name of the requested datatype Returns: flag indicating if the datatype exists Return type: bool
-
static
isdatatype
(item)[source]¶ Check if item is a valid datatype for the fastr system.
Parameters: item – item to check Returns: flag indicating if the item is a fastr datatype Return type: bool
-
match_types
(*args, **kwargs)[source]¶ Find the match between a list of DataTypes/TypeGroups, see resolve-datatype for details
Parameters: - args – A list of DataType/TypeGroup objects to match
- kwargs – A ‘preferred’ keyword argument can be used to indicate a list of DataTypes to prefer in case of ties (first has precedence over later in list)
Returns: The best DataType match, or None if no match is possible.
Raises: FastrTypeError – if not all args are subclasses of BaseDataType
-
match_types_any
(*args)[source]¶ Find the match between a list of DataTypes/TypeGroups, see resolve-datatype for details
Parameters: args – A list of DataType/TypeGroup objects to match Returns: A set with all DataTypes that match. Return type: set Raises: FastrTypeError – if not all args are subclasses of BaseDataType
-
plugin_class
¶ The PluginClass of the items of the BasePluginManager
-
poll_datatype
(filename)[source]¶ Poll an xml file to see if there is a definition of a datatype in it.
Parameters: filename (str) – path of the file to poll Returns: tuple with (id, version, basetype) if a datatype is found or (None, None, None) if no datatype is found
-
populate
()[source]¶ Populate Manager. After scanning for DataTypes, create the AnyType and set the preferred types
-
preferred_types
¶
-
dimension
Module¶
-
class
fastr.core.dimension.
Dimension
(name, size)[source]¶ Bases:
object
A class representing a dimension. It contains the name and size of the dimension.
-
__dict__
= dict_proxy({'__ne__': <function __ne__>, '__module__': 'fastr.core.dimension', 'name': <property object>, '__weakref__': <attribute '__weakref__' of 'Dimension' objects>, 'update_size': <function update_size>, '__repr__': <function __repr__>, '__dict__': <attribute '__dict__' of 'Dimension' objects>, '__eq__': <function __eq__>, '__doc__': '\n A class representing a dimension. It contains the name and size of the\n dimension.\n ', '__init__': <function __init__>, 'size': <property object>})¶
-
__module__
= 'fastr.core.dimension'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
name
¶
-
size
¶
-
-
class
fastr.core.dimension.
ForwardsDimensions
[source]¶ Bases:
fastr.core.dimension.HasDimensions
Class of objects that have dimensions not because they contain data with dimensions but forward them (optionally with changes via combine_dimensions)
-
__abstractmethods__
= frozenset(['source', 'combine_dimensions'])¶
-
__module__
= 'fastr.core.dimension'¶
-
combine_dimensions
(dimensions)[source]¶ Method to combine/manipulate the dimensions
Parameters: dimensions – the input dimensions from the source Returns: dimensions manipulated for this object Return type: tuple of dimensions
-
dimensions
¶ The dimensions of the object based on the forwarding
-
source
¶ The source object from which the dimensions are forwarded
Returns: the object from which the dimensions are forwarded Return type: HasDimensions
-
-
class
fastr.core.dimension.
HasDimensions
[source]¶ Bases:
object
A Mixin class for any object that has a notion of dimensions and size. It uses the dimension property to expose the dimension name and size.
-
__abstractmethods__
= frozenset(['dimensions'])¶
-
__dict__
= dict_proxy({'__module__': 'fastr.core.dimension', '__metaclass__': <class 'abc.ABCMeta'>, '_abc_negative_cache': <_weakrefset.WeakSet object>, '__dict__': <attribute '__dict__' of 'HasDimensions' objects>, '__weakref__': <attribute '__weakref__' of 'HasDimensions' objects>, 'dimnames': <property object>, 'size': <property object>, '_abc_cache': <_weakrefset.WeakSet object>, 'dimensions': <abc.abstractproperty object>, '__abstractmethods__': frozenset(['dimensions']), '_abc_negative_cache_version': 34, 'ndims': <property object>, '_abc_registry': <_weakrefset.WeakSet object>, '__doc__': '\n A Mixin class for any object that has a notion of dimensions and size. It\n uses the dimension property to expose the dimension name and size.\n '})¶
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.dimension'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
dimensions
¶ The dimensions has to be implemented by any subclass. It has to provide a tuple of Dimensions.
Returns: dimensions Return type: tuple
-
dimnames
¶ A tuple containing the dimension names of this object. All items of the tuple are of type str.
-
ndims
¶ The number of dimensions in this object
-
size
¶ A tuple containing the size of this object. All items of the tuple are of type int or sympy.Symbol.
-
inputgroup
Module¶
-
class
fastr.core.inputgroup.
InputGroup
(*args, **kwargs)[source]¶ Bases:
collections.OrderedDict
,fastr.core.dimension.HasDimensions
A class representing a group of inputs. Input groups allow the
-
__abstractmethods__
= frozenset([])¶
-
__delitem__
(*args, **kwargs)¶ od.__delitem__(y) <==> del od[y]
Note
This is a wrapped version of
collections.__delitem__
which triggers an update of the object after being called
-
__init__
(*args, **kwargs)¶ Create a new InputGroup representation
Parameters: - parent (
NodeRun
) – the parent node - id (str) – the id of the input group
Raises: FastrTypeError – if parent is not a NodeRun
Note
This is a wrapped version of
fastr.core.inputgroup.__init__
which triggers an update of the object after being called- parent (
-
__metaclass__
¶ alias of
UpdateableMeta
-
__module__
= 'fastr.core.inputgroup'¶
-
__setitem__
(*args, **kwargs)¶ Assign an input to this input group.
Parameters: Raises: FastrTypeError – if value of valid type
Note
This is a wrapped version of
fastr.core.inputgroup.__setitem__
which triggers an update of the object after being called
-
__updatefunc__
()[source]¶ Update the InputGroup. Triggers when a change is made to the content of the InputGroup. Automatically recalculates the size, primary Input etc.
-
__updatetriggers__
= ['__init__', '__setitem__', '__delitem__', 'clear', 'pop', 'popitem', 'setdefault', 'update']¶
-
clear
() → None. Remove all items from od.¶ Note
This is a wrapped version of
collections.clear
which triggers an update of the object after being called
-
dimensions
¶ The dimensions of this InputGroup
-
empty
¶ Bool indicating that this InputGroup is empty (has no data connected)
-
find_source_index
(target_size, target_dimnames, source_size, source_dimnames, target_index)[source]¶
-
fullid
¶
-
iterinputvalues
¶ Iterate over the item in this InputGroup
Returns: iterator yielding SampleItems
-
parent
¶ The parent node of this InputGroup
-
pop
(k[, d]) → v, remove specified key and return the corresponding¶ - value. If key is not found, d is returned if given, otherwise KeyError
- is raised.
Note
This is a wrapped version of
collections.pop
which triggers an update of the object after being called
-
popitem
() → (k, v), return and remove a (key, value) pair.¶ Pairs are returned in LIFO order if last is true or FIFO order if false.
Note
This is a wrapped version of
collections.popitem
which triggers an update of the object after being called
-
primary
¶ The primary Input in this InputGroup. The primary Input is the Input that defines the size of this InputGroup. In case of ties it will be the first in the tool definition.
-
setdefault
(k[, d]) → od.get(k,d), also set od[k]=d if k not in od¶ Note
This is a wrapped version of
collections.setdefault
which triggers an update of the object after being called
-
classmethod
solve_broadcast
(target_size, target_dimnames, source_size, source_dimnames, target_index, nodegroups=None)[source]¶
-
update
([E, ]**F) → None. Update D from mapping/iterable E and F.¶ - If E present and has a .keys() method, does: for k in E: D[k] = E[k]
- If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
Note
This is a wrapped version of
_abcoll.update
which triggers an update of the object after being called
-
inputgroupcombiner
Module¶
-
class
fastr.core.inputgroupcombiner.
BaseInputGroupCombiner
(parent)[source]¶ Bases:
fastr.core.dimension.HasDimensions
An object that takes the different input groups and combines them in the correct way.
-
__abstractmethods__
= frozenset(['merge', 'unmerge', 'iter_input_groups'])¶
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.inputgroupcombiner'¶
-
dimensions
¶
-
fullid
¶ The full id of the InputGroupCombiner
-
input_groups
¶
-
merge
(list_of_items)[source]¶ Given a list of items for each input group, it returns the combined list of items.
Parameters: list_of_items (list) – items to combine Returns: combined list
-
unmerge
(item)[source]¶ Given a item it will recreate the seperate items, basically this is the inverse operation of merge. However, this create an OrderedDict so that specific input groups can be easily retrieved. To get a round trip, the values of the OrderedDict should be taken:
>>> odict_of_items = combiner.unmerge(item) >>> item = combiner.merge(odict_of_items.values())
Parameters: item (list) – the item to unmerge Returns: items Return type: OrderedDict
-
-
class
fastr.core.inputgroupcombiner.
DefaultInputGroupCombiner
(parent)[source]¶ Bases:
fastr.core.inputgroupcombiner.BaseInputGroupCombiner
The default input group combiner combines the input group in a cross product version, taking each combinations of samples between the input groups. So if there are two input groups with one with size N and the other with size M x P the result would be N x M x P samples, with all possible combinations of the samples in each input group.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.inputgroupcombiner'¶
-
merge
(list_of_items)[source]¶ Given a list of items for each input group, it returns the combined list of items.
Parameters: list_of_items (list) – items to combine Returns: combined list
-
unmerge
(item)[source]¶ Given a item it will recreate the seperate items, basically this is the inverse operation of merge. However, this create an OrderedDict so that specific input groups can be easily retrieved. To get a round trip, the values of the OrderedDict should be taken:
>>> odict_of_items = combiner.unmerge(item) >>> item = combiner.merge(odict_of_items.values())
Parameters: item (list) – the item to unmerge Returns: items Return type: OrderedDict
-
-
class
fastr.core.inputgroupcombiner.
MergingInputGroupCombiner
(input_groups, merge_dimension)[source]¶ Bases:
fastr.core.inputgroupcombiner.BaseInputGroupCombiner
The merging input group combiner takes a similar approach as the default combiner but merges dimensions that are the same. If input group A has N(3) x M(2) samples and B has M(2) x P(4) it wil not result in N(3) x M(2) x M(2) x P(4), but merge the dimensions M leading to N(3) x M(2) x P(4) in resulting size.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.inputgroupcombiner'¶
-
inputoutput
Module¶
Classes for arranging the input and output for nodes.
Exported classes:
Input – An input for a node (holding datatype). Output – The output of a node (holding datatype and value). ConstantOutput – The output of a node (holding datatype and value).
Warning
Don’t mess with the Link, Input and Output internals from other places. There will be a huge chances of breaking the network functionality!
-
class
fastr.core.inputoutput.
AdvancedFlowOutput
(node, description)[source]¶ Bases:
fastr.core.inputoutput.Output
Output for nodes that have an advanced flow. This means that the output sample id and index is not the same as the input sample id and index. The AdvancedFlowOutput has one extra dimensions that is created by the Node.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.inputoutput'¶
-
dimensions
¶
-
-
class
fastr.core.inputoutput.
BaseInput
(node, description)[source]¶ Bases:
fastr.core.inputoutput.BaseInputOutput
Base class for all inputs.
-
__abstractmethods__
= frozenset(['itersubinputs', '_update', 'dimensions', 'fullid'])¶
-
__init__
(node, description)[source]¶ Instantiate a BaseInput
Parameters: - node – the parent node the input/output belongs to.
- description – the
ParameterDescription
describing the input/output.
Returns: the created BaseInput
Raises: - FastrTypeError – if description is not of class
ParameterDescription
- FastrDataTypeNotAvailableError – if the DataType requested cannot be found in the
fastr.typelist
-
__module__
= 'fastr.core.inputoutput'¶
-
-
class
fastr.core.inputoutput.
BaseInputOutput
(node, description)[source]¶ Bases:
fastr.core.dimension.HasDimensions
,fastr.core.updateable.Updateable
,fastr.core.serializable.Serializable
Base class for Input and Output classes. It mainly implements the properties to access the data from the underlying ParameterDescription.
-
__abstractmethods__
= frozenset(['_update', 'dimensions', 'fullid'])¶
-
__getstate__
()[source]¶ Retrieve the state of the BaseInputOutput
Returns: the state of the object Rtype dict:
-
__init__
(node, description)[source]¶ Instantiate a BaseInputOutput
Parameters: - node – the parent node the input/output belongs to.
- description – the
ParameterDescription
describing the input/output.
Returns: created BaseInputOutput
Raises: - FastrTypeError – if description is not of class
ParameterDescription
- FastrDataTypeNotAvailableError – if the DataType requested cannot be found in the
fastr.typelist
-
__iter__
()[source]¶ This function is blocked to avoid support for iteration using a lecacy __getitem__ method.
Returns: None Raises: FastrNotImplementedError – always
-
__module__
= 'fastr.core.inputoutput'¶
-
__repr__
()[source]¶ Get a string representation for the Input/Output
Returns: the string representation Return type: str
-
__setstate__
(state)[source]¶ Set the state of the BaseInputOutput by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
cardinality
(key=None, job_data=None)[source]¶ Determine the cardinality of this Input/Output. Optionally a key can be given to determine for a sample.
Parameters: key – key for a specific sample Returns: the cardinality Return type: int, sympy.Symbol, or None
-
check_cardinality
(key=None)[source]¶ Check if the actual cardinality matches the cardinality specified in the ParameterDescription. Optionally you can use a key to test for a specific sample.
Parameters: key – sample_index (tuple of int) or SampleId
for desired sampleReturns: flag indicating that the cardinality is correct Return type: bool Raises: FastrCardinalityError – if the Input/Output has an incorrect cardinality description.
-
datatype
¶ The datatype of this Input/Output
-
description
¶ The description object of this input/output
-
fullid
¶ The fullid of the Input/Output, the fullid should be unnique and makes the object retrievable by the network.
-
id
¶ Id of the Input/Output
-
node
¶ The NodeRun to which this Input/Output belongs
-
required
¶ Flag indicating that the Input/Output is required
-
-
class
fastr.core.inputoutput.
BaseOutput
(node, description)[source]¶ Bases:
fastr.core.inputoutput.BaseInputOutput
Base class for all outputs.
-
__abstractmethods__
= frozenset(['_update', 'dimensions', 'fullid'])¶
-
__init__
(node, description)[source]¶ Instantiate a BaseOutput
Parameters: - node – the parent node the output belongs to.
- description – the
ParameterDescription
describing the output.
Returns: created BaseOutput
Raises: - FastrTypeError – if description is not of class
ParameterDescription
- FastrDataTypeNotAvailableError – if the DataType requested cannot be found in the
fastr.typelist
-
__module__
= 'fastr.core.inputoutput'¶
-
automatic
¶ Flag indicating that the Output is generated automatically without being specified on the command line
-
-
class
fastr.core.inputoutput.
Input
(node, description)[source]¶ Bases:
fastr.core.inputoutput.BaseInput
Class representing an input of a node. Such an input will be connected to the output of another node or the output of an constant node to provide the input value.
-
__abstractmethods__
= frozenset([])¶
-
__eq__
(other)[source]¶ Compare two Input instances with each other. This function ignores the parent node and update status, but tests rest of the dict for equality.
Parameters: other ( Input
) – the other instances to compare toReturns: True if equal, False otherwise Return type: bool
-
__getitem__
(key)[source]¶ Retrieve an item from this Input.
Parameters: key (str,
SampleId
or tuple) – the key of the requested item, can be a key str, sample index tuple or aSampleId
Returns: the return value depends on the requested key. If the key was an int the corresponding
SubInput
will be returned. If the key was aSampleId
or sample index tuple, the correspondingSampleItem
will be returned.Return type: SampleItem
orSubInput
Raises: - FastrTypeError – if key is not of a valid type
- FastrKeyError – if the key is not found
-
__init__
(node, description)[source]¶ Instantiate an input.
Parameters: - node (
NodeRun
) – the parent node of this input. - description (ParameterDescription) – the ParameterDescription of the input.
Returns: the created Input
- node (
-
__module__
= 'fastr.core.inputoutput'¶
-
__setitem__
(key, value)[source]¶ Create a link between a SubInput of this Inputs and an Output/Constant
Parameters: - key (int, str) – the key of the SubInput
- value (BaseOutput, list, tuple, dict, OrderedDict) – the target to link, can be an output or a value to create a constant for
Raises: FastrTypeError – if key is not of a valid type
-
__setstate__
(state)[source]¶ Set the state of the Input by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
append
(value)[source]¶ When you want to append a link to an Input, you can use the append property. This will automatically create a new SubInput to link to.
example:
>>> link = node2['input'].append(node1['output'])
will create a new SubInput in node2[‘input’] and link to that.
-
cardinality
(key=None, job_data=None)[source]¶ Cardinality for an Input is the sum the cardinalities of the SubInputs, unless defined otherwise.
Parameters: key (tuple of int or SampleId
) – key for a specific sample, can be sample index or idReturns: the cardinality Return type: int, sympy.Symbol, or None
-
datatype
¶ The datatype of this Input
-
dimensions
¶ The list names of the dimensions in this Input. This will be a list of str.
-
fullid
¶ The full defining ID for the Input
-
get_sourced_nodes
()[source]¶ Get a list of all
Nodes
connected as sources to this InputReturns: list of all connected Nodes
Return type: list
-
get_sourced_outputs
()[source]¶ Get a list of all
Outputs
connected as sources to this InputReturns: tuple of all connected Outputs
Return type: tuple
-
index
(value)[source]¶ Find index of a SubInput
Parameters: value ( SubInput
) – theSubInput
to find the index ofReturns: key Return type: int, str
-
input_group
¶ The id of the
InputGroup
this Input belongs to.
-
insert
(index)[source]¶ Insert a new SubInput at index in the sources list
Parameters: key (int) – positive integer for position in _source list to insert to Returns: newly inserted SubInput
Return type: SubInput
-
itersubinputs
()[source]¶ Iterate over the
SubInputs
in this Input.Returns: iterator yielding SubInput
example:
>>> for subinput in input_a.itersubinputs(): print subinput
-
-
class
fastr.core.inputoutput.
MacroOutput
(node, description)[source]¶ Bases:
fastr.core.inputoutput.Output
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.inputoutput'¶
-
dimensions
¶
-
-
class
fastr.core.inputoutput.
Output
(node, description)[source]¶ Bases:
fastr.core.inputoutput.BaseOutput
Class representing an output of a node. It holds the output values of the tool ran. Output fields can be connected to inputs of other nodes.
-
__abstractmethods__
= frozenset([])¶
-
__eq__
(other)[source]¶ Compare two Output instances with each other. This function ignores the parent node, listeners and update status, but tests rest of the dict for equality.
Parameters: other (Output) – the other instances to compare to Returns: True if equal, False otherwise Return type: bool
-
__getitem__
(key)[source]¶ Retrieve an item from this Output. The returned value depends on what type of key used:
- Retrieving data using index tuple: [index_tuple]
- Retrieving data sample_id str: [SampleId]
- Retrieving a list of data using SampleId list: [sample_id1, ..., sample_idN]
- Retrieving a
SubOutput
using an int or slice: [n] or [n:m]
Parameters: key (int, slice,
SampleId
or tuple) – the key of the requested item, can be a number, slice, sample index tuple or aSampleId
Returns: the return value depends on the requested key. If the key was an int or slice the corresponding
SubOutput
will be returned (and created if needed). If the key was aSampleId
or sample index tuple, the correspondingSampleItem
will be returned. If the key was a list ofSampleId
a tuple ofSampleItem
will be returned.Return type: SubInput
orSampleItem
or list ofSampleItem
Raises: - FastrTypeError – if key is not of a valid type
- FastrKeyError – if the parent Node has not been executed
-
__getstate__
()[source]¶ Retrieve the state of the Output
Returns: the state of the object Rtype dict:
-
__init__
(node, description)[source]¶ Instantiate an Output
Parameters: - node – the parent node the output belongs to.
- description – the
ParameterDescription
describing the output.
Returns: created Output
Raises: - FastrTypeError – if description is not of class
ParameterDescription
- FastrDataTypeNotAvailableError – if the DataType requested cannot be found in the
fastr.typelist
-
__module__
= 'fastr.core.inputoutput'¶
-
__setstate__
(state)[source]¶ Set the state of the Output by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
blocking
¶ Flag indicating that this Output will cause blocking in the execution
-
cardinality
()[source]¶ Cardinality of this Output, may depend on the inputs of the parent Node.
Returns: the cardinality
Return type: Raises: - FastrCardinalityError – if cardinality references an invalid
Input
- FastrTypeError – if the referenced cardinality values type cannot be case to int
- FastrValueError – if the referenced cardinality value cannot be case to int
- FastrCardinalityError – if cardinality references an invalid
-
static
create_output_cardinality
(desc)[source]¶ Create a lambda function that returns an integer value of the cardinality.
param str desc: The cardinality description string return: output cardinality description rtype tuple: The description string can be one of the following forms:
- N: N number of values needed.
- as:input_name: the same number of values as attached to input_name are needed.
- val:input_name: the number of values needed is the value of input_name.
- unknown: the output cardinality cannot be estimated a priori
-
datatype
¶ The datatype of this Output
-
dimensions
¶ The list of the dimensions in this Output. This will be a tuple of Dimension.
-
fullid
¶ The full defining ID for the Output
-
preferred_types
¶ The list of preferred
DataTypes
for this Output.
-
resulting_datatype
¶ The
DataType
that will the results of this Output will have.
-
valid
¶ Check if the output is valid, i.e. has a valid cardinality
-
-
class
fastr.core.inputoutput.
SourceOutput
(node, description)[source]¶ Bases:
fastr.core.inputoutput.Output
Output for a SourceNodeRun, this type of Output determines the cardinality in a different way than a normal NodeRun.
-
__abstractmethods__
= frozenset([])¶
-
__getitem__
(item)[source]¶ Retrieve an item from this Output. The returned value depends on what type of key used:
- Retrieving data using index tuple: [index_tuple]
- Retrieving data sample_id str: [SampleId]
- Retrieving a list of data using SampleId list: [sample_id1, ..., sample_idN]
- Retrieving a
SubOutput
using an int or slice: [n] or [n:m]
Parameters: key (int, slice,
SampleId
or tuple) – the key of the requested item, can be a number, slice, sample index tuple or aSampleId
Returns: the return value depends on the requested key. If the key was an int or slice the corresponding
SubOutput
will be returned (and created if needed). If the key was aSampleId
or sample index tuple, the correspondingSampleItem
will be returned. If the key was a list ofSampleId
a tuple ofSampleItem
will be returned.Return type: SubInput
orSampleItem
or list ofSampleItem
Raises: - FastrTypeError – if key is not of a valid type
- FastrKeyError – if the parent NodeRun has not been executed
-
__init__
(node, description)[source]¶ Instantiate a FlowOutput
Parameters: - node – the parent node the output belongs to.
- description – the
ParameterDescription
describing the output.
Returns: created FlowOutput
Raises: - FastrTypeError – if description is not of class
ParameterDescription
- FastrDataTypeNotAvailableError – if the DataType requested cannot be found in the
fastr.typelist
-
__module__
= 'fastr.core.inputoutput'¶
-
cardinality
()[source]¶ Cardinality of this SourceOutput, may depend on the inputs of the parent NodeRun.
Parameters: key (tuple of int or SampleId
) – key for a specific sample, can be sample index or idReturns: the cardinality Return type: int, sympy.Symbol, or None
-
linearized
¶ A linearized version of the sample data, this is lazily cached linearized version of the underlying SampleCollection.
-
-
class
fastr.core.inputoutput.
SubInput
(input_)[source]¶ Bases:
fastr.core.inputoutput.BaseInput
This class is used by
Input
to allow for multiple links to anInput
. The SubInput class can hold only a single Link to a (Sub)Output, but behaves very similar to anInput
otherwise.-
__abstractmethods__
= frozenset([])¶
-
__eq__
(other)[source]¶ Compare two SubInput instances with each other. This function ignores the parent, node, source and update status, but tests rest of the dict for equality.
Parameters: other (SubInput) – the other instances to compare to Returns: True if equal, False otherwise
-
__getitem__
(key)[source]¶ Retrieve an item from this SubInput.
Parameters: key (int) – the index of the requested item Returns: the corresponding SubInput
Return type: SubInput
Raises: FastrTypeError – if key is not of a valid type Note
As a SubInput has only one SubInput, only requesting int key 0 or -1 is allowed, and it will return self
-
__getstate__
()[source]¶ Retrieve the state of the SubInput
Returns: the state of the object Rtype dict:
-
__init__
(input_)[source]¶ Instantiate an SubInput.
Parameters: input ( Input
) – the parent of this SubInput.Returns: the created SubInput
-
__module__
= 'fastr.core.inputoutput'¶
-
__setstate__
(state)[source]¶ Set the state of the SubInput by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
cardinality
(key=None, job_data=None)[source]¶ Get the cardinality for this SubInput. The cardinality for a SubInputs is defined by the incoming link.
Parameters: key ( SampleIndex
orSampleId
) – key for a specific sample, can be sample index or idReturns: the cardinality Return type: int, sympy.Symbol, or None
-
description
¶
-
dimensions
¶ List of dimension for this SubInput
-
fullid
¶ The full defining ID for the SubInput
-
get_sourced_nodes
()[source]¶ Get a list of all
Nodes
connected as sources to this SubInputReturns: list of all connected Nodes
Return type: list
-
get_sourced_outputs
()[source]¶ Get a list of all
Outputs
connected as sources to this SubInputReturns: list of all connected Outputs
Return type: list
-
input_group
¶ The id of the
InputGroup
this SubInputs parent belongs to.
-
iteritems
()[source]¶ Iterate over the
SampleItems
that are in the SubInput.Returns: iterator yielding SampleItem
objects
-
itersubinputs
()[source]¶ Iterate over SubInputs (for a SubInput it will yield self and stop iterating after that)
Returns: iterator yielding SubInput
example:
>>> for subinput in input_a.itersubinputs(): print subinput
-
node
¶ The Node to which this SubInputs parent belongs
-
-
class
fastr.core.inputoutput.
SubOutput
(output, index)[source]¶ Bases:
fastr.core.inputoutput.Output
The SubOutput is an Output that represents a slice of another Output.
-
__abstractmethods__
= frozenset([])¶
-
__eq__
(other)[source]¶ Compare two SubOutput instances with each other. This function ignores the parent, node and update status, but tests rest of the dict for equality. equality
Parameters: other (SubOutput) – the other instances to compare to Returns: True if equal, False otherwise Return type: bool
-
__getitem__
(key)[source]¶ Retrieve an item from this SubOutput. The returned value depends on what type of key used:
- Retrieving data using index tuple: [index_tuple]
- Retrieving data sample_id str: [SampleId]
- Retrieving a list of data using SampleId list: [sample_id1, ..., sample_idN]
- Retrieving a
SubOutput
using an int or slice: [n] or [n:m]
Parameters: key (int, slice, SampleId
or tuple) – the key of the requested item, can be a number, slice, sample index tuple or aSampleId
Returns: the return value depends on the requested key. If the key was an int or slice the corresponding SubOutput
will be returned (and created if needed). If the key was aSampleId
or sample index tuple, the correspondingSampleItem
will be returned. If the key was a list ofSampleId
a tuple ofSampleItem
will be returned.Return type: SubInput
orSampleItem
or list ofSampleItem
Raises: FastrTypeError – if key is not of a valid type
-
__getstate__
()[source]¶ Retrieve the state of the SubOutput
Returns: the state of the object Rtype dict:
-
__init__
(output, index)[source]¶ Instantiate a SubOutput
Parameters: Returns: created SubOutput
Raises: - FastrTypeError – if the output argument is not an instance of
Output
- FastrTypeError – if the index argument is not an
int
orslice
- FastrTypeError – if the output argument is not an instance of
-
__module__
= 'fastr.core.inputoutput'¶
-
__setstate__
(state)[source]¶ Set the state of the SubOutput by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
__str__
()[source]¶ Get a string version for the SubOutput
Returns: the string version Return type: str
-
cardinality
()[source]¶ Cardinality of this SubOutput depends on the parent Output and
self.index
Parameters: key (tuple of int or
SampleId
) – key for a specific sample, can be sample index or idReturns: the cardinality
Return type: Raises: - FastrCardinalityError – if cardinality references an invalid
Input
- FastrTypeError – if the referenced cardinality values type cannot be case to int
- FastrValueError – if the referenced cardinality value cannot be case to int
- FastrCardinalityError – if cardinality references an invalid
-
datatype
¶ The datatype of this SubOutput
-
fullid
¶ The full defining ID for the SubOutput
-
indexrep
¶ Simple representation of the index.
-
node
¶ The NodeRun to which this SubOutput belongs
-
preferred_types
¶ The list of preferred
DataTypes
for this SubOutput.
-
resulting_datatype
¶ The
DataType
that will the results of this SubOutput will have.
-
samples
¶ The
SampleCollection
for this SubOutput
-
interface
Module¶
- A module that describes the interface of a Tool. It specifies how a set of
- input values will be translated to commands to be executed. This creates a generic interface to different ways of executing underlying software.
-
class
fastr.core.interface.
InputSpec
[source]¶ Bases:
fastr.core.interface.InputSpec
-
__dict__
= dict_proxy({'__dict__': <attribute '__dict__' of 'InputSpec' objects>, '__module__': 'fastr.core.interface', '__new__': <staticmethod object>, '__doc__': None})¶
-
__module__
= 'fastr.core.interface'¶
-
-
class
fastr.core.interface.
Interface
[source]¶ Bases:
fastr.core.baseplugin.Plugin
,fastr.core.serializable.Serializable
Abstract base class of all Interfaces. Defines the minimal requirements for all Interface implementations.
-
__abstractmethods__
= frozenset(['inputs', 'execute', '__setstate__', 'expanding', '__getstate__', 'outputs'])¶
-
__getstate__
()[source]¶ Retrieve the state of the Interface
Returns: the state of the object Rtype dict:
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.interface'¶
-
execute
(target, payload)[source]¶ Execute the interface given the a target and payload. The payload should have the form {‘input’: {‘input_id_a’: (value, value), ‘input_id_b’: (value, value)}, ‘output’: {‘output_id_a’: (value, value), ‘output_id_b’: (value, value)}}
Parameters: - target – the target to call
- payload – the payload to use
Returns: the result of the execution
Return type: (tuple of) InterfaceResult
-
expanding
¶ Indicates whether or not this Interface will result in multiple samples per run. If the flow is unaffected, this will be zero, if it is nonzero it means that number of dimension will be added to the sample array.
-
inputs
¶ OrderedDict of Inputs connected to the Interface. The format should be {input_id: InputSpec}.
-
outputs
¶ OrderedDict of Output connected to the Interface. The format should be {output_id: OutputSpec}.
-
-
class
fastr.core.interface.
InterfacePluginManager
[source]¶ Bases:
fastr.core.pluginmanager.PluginSubManager
Container holding all the CollectorPlugins
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.interface'¶
-
-
class
fastr.core.interface.
InterfaceResult
(result_data, log_data, payload, sample_index=None, sample_id=None, errors=None)[source]¶ Bases:
object
The class in which Interfaces should wrap their results to be picked up by fastr
-
__dict__
= dict_proxy({'__dict__': <attribute '__dict__' of 'InterfaceResult' objects>, '__module__': 'fastr.core.interface', '__weakref__': <attribute '__weakref__' of 'InterfaceResult' objects>, '__doc__': '\n The class in which Interfaces should wrap their results to be picked up by fastr\n ', '__init__': <function __init__>})¶
-
__module__
= 'fastr.core.interface'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
class
fastr.core.interface.
OutputSpec
[source]¶ Bases:
fastr.core.interface.OutputSpec
-
__dict__
= dict_proxy({'__dict__': <attribute '__dict__' of 'OutputSpec' objects>, '__module__': 'fastr.core.interface', '__new__': <staticmethod object>, '__doc__': None})¶
-
__module__
= 'fastr.core.interface'¶
-
-
fastr.core.interface.
OutputSpecBase
¶ alias of
OutputSpec
ioplugin
Module¶
This module contains the manager class for IOPlugins and the base class for all IOPlugins
-
class
fastr.core.ioplugin.
IOPlugin
[source]¶ Bases:
fastr.core.baseplugin.Plugin
IOPlugins
are used for data import and export for the sources and sinks. The main use of theIOPlugins
is during execution (see Execution). TheIOPlugins
can be accessed viafastr.ioplugins
, but generally there should be no need for direct interaction with these objects. The use of is mainly via the URL used to specify source and sink data.-
__abstractmethods__
= frozenset(['scheme'])¶
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.ioplugin'¶
-
cleanup
()[source]¶ (abstract)
Clean up the IOPlugin. This is to do things like closing files or connections. Will be called when the plugin is no longer required.
-
expand_url
(url)[source]¶ (abstract)
Expand an URL. This allows a source to collect multiple samples from a single url. The URL will have a wildcard or point to something with info and multiple urls will be returned.Parameters: url (str) – url to expand Returns: the resulting url(s), a tuple if multiple, otherwise a str Return type: str or tuple of str
-
fetch_url
(inurl, outfile)[source]¶ (abstract)
Fetch a file from an external data source.Parameters: - inurl – url to the item in the data store
- outpath – path where to store the fetch data locally
-
fetch_value
(inurl)[source]¶ (abstract)
Fetch a value from an external data source.Parameters: inurl – the url of the value to retrieve Returns: the fetched value
-
static
isurl
(string)[source]¶ Test if given string is an url.
Parameters: string (str) – string to test Returns: True
if the string is an url,False
otherwiseReturn type: bool
-
path_to_url
(path, mountpoint=None)[source]¶ (abstract)
Construct an url from a given mount point and a relative path to the mount point.Parameters: Returns: url matching the path
Return type:
-
static
print_result
(result)[source]¶ Print the result of the IOPlugin to stdout to be picked up by the tool
Parameters: result – value to print as a result Returns: None
-
pull_source_data
(inurl, outdir, sample_id, datatype=None)[source]¶ Transfer the source data from inurl to be available in outdir.
Parameters: Returns: None
-
push_sink_data
(inpath, outurl, datatype=None)[source]¶ Write out the sink data from the inpath to the outurl.
Parameters: Returns: None
-
put_url
(inpath, outurl)[source]¶ (abstract)
Put the files to the external data store.Parameters: - inpath – path to the local data
- outurl – url to where to store the data in the external data store.
-
put_value
(value, outurl)[source]¶ (abstract)
Put the files to the external data store.Parameters: - value – the value to store
- outurl – url to where to store the data in the external data store.
-
scheme
¶ (abstract)
This abstract property is to be overwritten by a subclass to indicate the url scheme associated with the IOPlugin.
-
-
class
fastr.core.ioplugin.
IOPluginManager
[source]¶ Bases:
fastr.core.pluginmanager.PluginSubManager
A mapping containing the IOPlugins known to this system
-
__abstractmethods__
= frozenset([])¶
-
__init__
()[source]¶ Create the IOPluginManager and populate it.
Returns: newly created IOPluginManager
-
__module__
= 'fastr.core.ioplugin'¶
-
cleanup
()[source]¶ Cleanup all plugins, this closes files, connections and other things that could be left dangling otherwise.
-
static
create_ioplugin_tool
()[source]¶ Create the tools which handles sinks and sources. The command of this tool is the main of core.ioplugin.
-
expand_url
(url)[source]¶ Expand the url by filling the wildcards. This function checks the url scheme and uses the expand function of the correct IOPlugin.
Parameters: url (str) – url to expand Returns: list of urls Return type: list of str
-
populate
()[source]¶ Populate the IOPlugins manager. After the default directory scan, add the vfs IOPlugin and create the Tools for the IOPlugins
-
pull_source_data
(url, outdir, sample_id, datatype=None)[source]¶ Retrieve data from an external source. This function checks the url scheme and selects the correct IOPlugin to retrieve the data.
Parameters: Returns: None
-
push_sink_data
(inpath, outurl, datatype=None)[source]¶ Send data to an external source. This function checks the url scheme and selects the correct IOPlugin to retrieve the data.
Parameters:
-
put_url
(inpath, outurl)[source]¶ Put the files to the external data store.
Parameters: - inpath – path to the local data
- outurl – url to where to store the data in the external data store.
-
link
Module¶
The link module contain the Link class. This class represents the links in a network. These links lead from an output (BaseOutput) to an input (BaseInput) and indicate the desired data flow. Links are smart objects, in the sense that when you set their start or end point, they register themselves with the Input and Output. They do all the book keeping, so as long as you only set the source and target of the Link, the link should be valid.
Warning
Don’t mess with the Link, Input and Output internals from other places. There will be a huge chances of breaking the network functionality!
-
class
fastr.core.link.
Link
(source, target, parent, id_=None, collapse=None, expand=None)[source]¶ Bases:
fastr.core.dimension.HasDimensions
,fastr.core.updateable.Updateable
,fastr.core.serializable.Serializable
Class for linking outputs (
BaseOutput
) to inputs (BaseInput
)Examples:
>>> import fastr >>> network = fastr.Network() >>> link1 = network.create_link( n1.ouputs['out1'], n2.inputs['in2'] ) link2 = Link() link2.source = n1.ouputs['out1'] link2.target = n2.inputs['in2']
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'Link.schema.json'¶
-
__eq__
(other)[source]¶ Test for equality between two Links
Parameters: other (Link) – object to test against Returns: True for equality, False otherwise Return type: bool
-
__init__
(source, target, parent, id_=None, collapse=None, expand=None)[source]¶ Create a new Link in a Network.
Parameters: - source (
BaseOutput
) – the source output - target (
BaseInput
) – the target input - parent (
Network
or None) – the parent network - id (str or None) – the id of the link, if no id_ is given, the id will be in the form of “link_{:d}”
- collapse (int, str, or tuple of int/str) – the dimensions that the link has to collapse on
- expand (bool) – Does this link need to expand the cardinality into a new sample dimension
Returns: newly created Link
Raises: - FastrValueError – if parent is not given
- FastrValueError – if the source output is not in the same network as the Link
- FastrValueError – if the target input is not in the same network as the Link
- source (
-
__module__
= 'fastr.core.link'¶
-
__repr__
()[source]¶ Get a string representation for the Link
Returns: the string representation Return type: str
-
__setstate__
(state)[source]¶ Set the state of the Link by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None Raises: FastrValueError – if the parent network is not set
-
cardinality
(index=None)[source]¶ Cardinality for a Link is given by source Output and the collapse/expand settings
Parameters: key (SampleIndex) – key for a specific sample (can be only a sample index!) Returns: the cardinality Return type: int, sympy.Symbol Raises: FastrIndexError – if the index length does not match the number of dimension in the data
-
collapse
¶ The converging dimensions of this link. Collapsing changes some dimensions of sample lists into cardinality, reshaping the data.
Collapse can be set to a tuple or an int/str, in which case it will be automatically wrapped in a tuple. The int will be seen as indices of the dimensions to collapse. The str will be seen as the name of the dimensions over which to collapse.
Raises: FastrTypeError – if assigning a collapse value of a wrong type
-
collapse_indexes
¶ The converging dimensions of this link as integers. Dimension names are replaces with the corresponding int.
Collapsing changes some dimensions of sample lists into cardinality, reshaping the data
-
classmethod
createobj
(state, network=None)[source]¶ Create object function for Link
Parameters: - cls – The class to create
- state – The state to use to create the Link
- network – the parent Network
Returns: newly created Link
-
destroy
()[source]¶ The destroy function of a link removes all default references to a link. This means the references in the network, input and output connected to this link. If there is no references in other places in the code, it will destroy the link (reference count dropping to zero).
This function is called when a source for an input is set to another value and the links becomes disconnected. This makes sure there is no dangling links.
-
dimensions
¶ The dimensions of the data delivered by the link. This can be different from the source dimensions because the link can make data collapse or expand.
-
expand
¶ Flag indicating that the link will expand the cardininality into a new sample dimension to be created.
-
fullid
¶ The full defining ID for the Input
-
parent
¶ The Network to which this Link belongs.
-
source
¶ The source
BaseOutput
of the Link. Setting the source will automatically register the Link with the source BaseOutput. Updating source will also make sure the Link is unregistered with the previous source.Raises: FastrTypeError – if assigning a non BaseOutput
-
status
¶
-
target
¶ The target
BaseInput
of the Link. Setting the target will automatically register the Link with the target BaseInput. Updating target will also make sure the Link is unregistered with the previous target.Raises: FastrTypeError – if assigning a non BaseInput
-
network
Module¶
Network module containing Network facilitators and analysers.
-
class
fastr.core.network.
Network
(id_='unnamed_network', version=None, filename=None)[source]¶ Bases:
fastr.core.serializable.Serializable
The NetworkRun contains the entire Run state for a Network execution. It has a working copy of the network, but also includes all temporary data required for the execution. These objects are meant to be single use.
-
NETWORK_DUMP_FILE_NAME
= '__fastr_network__.json'¶
-
SINK_DUMP_FILE_NAME
= '__sink_data__.json'¶
-
SOURCE_DUMP_FILE_NAME
= '__source_data__.pickle.gz'¶
-
__dataschemafile__
= 'Network.schema.json'¶
-
__eq__
(other)[source]¶ Compare two Networks and see if they are equal.
Parameters: other ( Network
) –Returns: flag indicating that the Networks are the same Return type: bool
-
__getitem__
(item)[source]¶ Get an item by its fullid. The fullid can point to a link, node, input, output or even subinput/suboutput.
Parameters: item (str,unicode) – fullid of the item to retrieve Returns: the requested item
-
__getstate__
()[source]¶ Retrieve the state of the Network
Returns: the state of the object Rtype dict:
-
__init__
(id_='unnamed_network', version=None, filename=None)[source]¶ Create a new, empty Network
Parameters: name (str) – name of the Network Returns: newly created Network Raises: OSError – if the tmp mount in the config is not a writable directory
-
__module__
= 'fastr.core.network'¶
-
__setstate__
(state)[source]¶ Set the state of the Network by the given state. This completely overwrites the old state!
Parameters: state (dict) – The state to populate the object with Returns: None
-
add_link
(link)[source]¶ Add a Link to the Network. Make sure the link is in the link list and the link parent is set to this Network
Parameters: link (
Link
) – link to addRaises: - FastrTypeError – if link is incorrectly typed
- FastrNetworkMismatchError – if the link already belongs to another Network
-
add_node
(node)[source]¶ Add a Node to the Network. Make sure the node is in the node list and the node parent is set to this Network
Parameters: node ( Node
) – node to addRaises: FastrTypeError – if node is incorrectly typed
-
check_id
(id_)[source]¶ Check if an id for an object is valid and unused in the Network. The method will always returns True if it does not raise an exception.
Parameters: id (str) – the id to check
Returns: True
Raises: - FastrValueError – if the id is not correctly formatted
- FastrValueError – if the id is already in use
-
create_constant
(datatype, data, id_=None, stepid=None, nodegroup=None, sourcegroup=None)[source]¶ Create a ConstantNode in this Network. The Node will be automatically added to the Network.
Parameters: - datatype (
BaseDataType
) – The DataType of the constant node - data (datatype or list of datatype) – The data to hold in the constant node
- id (str) – The id of the constant node to be created
- stepid (str) – The stepid to add the created constant node to
- nodegroup (str) – The group the node belongs to, this can be important for FlowNodes and such, as they will have matching dimension names.
Returns: the newly created constant node
Return type: - datatype (
-
create_link
(source, target, id_=None, collapse=None, expand=None)[source]¶ Create a link between two Nodes and add it to the current Network.
Parameters: - source (
BaseOutput
) – the output that is the source of the link - target (
BaseInput
) – the input that is the target of the link - id (str) – the id of the link
Returns: the created link
Type: - source (
-
create_node
(tool, id_=None, stepid=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Create a Node in this Network. The Node will be automatically added to the Network.
Parameters: Returns: the newly created node
Return type:
-
create_sink
(datatype, id_=None, stepid=None)[source]¶ Create a SinkNode in this Network. The Node will be automatically added to the Network.
Parameters: Returns: the newly created sink node
Return type:
-
create_source
(datatype, id_=None, stepid=None, nodegroup=None, sourcegroup=None)[source]¶ Create a SourceNode in this Network. The Node will be automatically added to the Network.
Parameters: - datatype (
BaseDataType
) – The DataType of the source source_node - id (str) – The id of the source source_node to be created
- stepid (str) – The stepid to add the created source source_node to
- nodegroup (str) – The group the node belongs to, this can be important for FlowNodes and such, as they will have matching dimension names.
- sourcegroup (str) – DEPRECATED! The nodegroup this SourceNode will be added to
Returns: the newly created source source_node
Return type: SourceNode
- datatype (
-
draw
(name=None, image_format=None, draw_dimensions=False, context=None, graph=None, expand_macro=False)[source]¶
-
draw_network
(name='network_layout', img_format='svg', draw_dimension=False, expand_macro=False)[source]¶ Output a dot file and try to convert it to an image file.
Parameters: img_format (str) – extension of the image format to convert to Returns: path of the image created or None if failed Return type: str or None
-
fullid
¶ The fullid of the Network, within the network scope
-
global_id
¶ The global id of the Network, this is different for networks used in macronodes, as they still have parents.
-
id
¶ The id of the Network. This is a read only property.
-
namespace
= None¶ The namespace this network lives in, this will be set by the NetworkManager on load
-
nodegroups
¶ Give an overview of the nodegroups in the network
-
remove
(value)[source]¶ Remove an item from the Network.
Parameters: value ( Node
orLink
) – the item to remove
-
networkmanager
Module¶
This module contains the tool manager class
-
class
fastr.core.networkmanager.
NetworkManager
(path)[source]¶ Bases:
fastr.core.objectmanager.ObjectManager
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.networkmanager'¶
-
object_class
¶
-
-
fastr.core.networkmanager.
networklist
= NetworkManager add_ints v0.0 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/networks/add_ints.json¶ The fastr networklist
node
Module¶
A module to maintain a network node.
Exported classes:
Node – A class encapsulating a tool. ConstantNode – A node encapsulating an Output to set scalar values. SourceNode – A class providing a handle to a file.
-
class
fastr.core.node.
AdvancedFlowNode
(tool, id_=None, parent=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Bases:
fastr.core.node.FlowNode
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.node'¶
-
-
class
fastr.core.node.
ConstantNode
(datatype, data, id_=None, parent=None, nodegroup=None)[source]¶ Bases:
fastr.core.node.SourceNode
Class encapsulating one output for which a value can be set. For example used to set a scalar value to the input of a node.
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'ConstantNode.schema.json'¶
-
__getstate__
()[source]¶ Retrieve the state of the ConstantNode
Returns: the state of the object Rtype dict:
-
__init__
(datatype, data, id_=None, parent=None, nodegroup=None)[source]¶ Instantiation of the ConstantNode.
Parameters: - datatype – The datatype of the output.
- data – the prefilled data to use.
- id – The url pattern.
This class should never be instantiated directly (unless you know what you are doing). Instead create a constant using the network class like shown in the usage example below.
usage example:
>>> import fastr >>> network = fastr.Network() >>> source = network.create_source(datatype=fastr.typelist['ITKImageFile'], id_='sourceN')
or alternatively create a constant node by assigning data to an item in an InputDict:
>>> node_a.inputs['in'] = ['some', 'data']
which automatically creates and links a ConstantNode to the specified Input
-
__module__
= 'fastr.core.node'¶
-
__setstate__
(state)[source]¶ Set the state of the ConstantNode by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
data
¶ The data stored in this constant node
-
-
class
fastr.core.node.
FlowNode
(tool, id_=None, parent=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Bases:
fastr.core.node.Node
A Flow Node is a special subclass of Nodes in which the amount of samples can vary per Output. This allows non-default data flows.
-
__abstractmethods__
= frozenset([])¶
-
__init__
(tool, id_=None, parent=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Instantiate a flow node.
Parameters: Returns: the newly created FlowNode
-
__module__
= 'fastr.core.node'¶
-
blocking
¶ A FlowNode is (for the moment) always considered blocking.
Returns: True
-
dimensions
¶ Names of the dimensions in the Node output. These will be reflected in the SampleIdList of this Node.
-
outputsize
¶ Size of the outputs in this Node
-
-
class
fastr.core.node.
InputDict
(*args, **kwds)[source]¶ Bases:
collections.OrderedDict
The container containing the Inputs of Node. Implements helper functions for the easy linking syntax.
-
__module__
= 'fastr.core.node'¶
-
__setitem__
(key, value, dict_setitem=<slot wrapper '__setitem__' of 'dict' objects>)[source]¶ Set an item in the input dictionary. The behaviour depends on the type of the value. For a
BaseInput
, the input will simply be added to the list of inputs. For aBaseOutput
, a link between the output and input will be created.Parameters: - key (str) – id of the input to assign/link
- value (
BaseInput
orBaseOutput
) – either the input to add or the output to link - dict_setitem – the setitem function to use for the underlying OrderedDict insert
-
-
class
fastr.core.node.
MacroNode
(network, id_=None, parent=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Bases:
fastr.core.node.Node
MacroNode encapsulates an entire network in a single node.
-
__abstractmethods__
= frozenset([])¶
-
__getstate__
()[source]¶ Retrieve the state of the MacroNode
Returns: the state of the object Rtype dict:
-
__init__
(network, id_=None, parent=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Parameters: network (Network) – network to create macronode for
-
__module__
= 'fastr.core.node'¶
-
get_output_info
(output)[source]¶ This functions maps the output dimensions based on the input dimensions of the macro. This is cached for speed as this can become rather costly otherwise
Parameters: output – output to get info for Returns: tuple of Dimensions
-
network
¶
-
-
class
fastr.core.node.
Node
(tool, id_=None, node_class=None, parent=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Bases:
fastr.core.dimension.HasDimensions
,fastr.core.updateable.Updateable
,fastr.core.serializable.Serializable
The class encapsulating a node in the network. The node is responsible for setting and checking inputs and outputs based on the description provided by a tool instance.
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'Node.schema.json'¶
-
__eq__
(other)[source]¶ Compare two Node instances with each other. This function ignores the parent and update status, but tests rest of the dict for equality. equality
Parameters: other (Node) – the other instances to compare to Returns: True if equal, False otherwise
-
__init__
(tool, id_=None, node_class=None, parent=None, cores=None, memory=None, walltime=None, nodegroup=None)[source]¶ Instantiate a node.
Parameters: - tool (
Tool
) – The tool to base the node on - id (str) – the id of the node
- node_class (str) – The class of the NodeRun to create (e.g. SourceNodeRun, NodeRun)
- parent (
Network
) – the parent network of the node - cores (int) – number of cores required for executing this Node
- memory (str) – amount of memory required in the form d+[mMgG] where M is for megabyte and G for gigabyte
- walltime (str) – amount of time required in second or in the form HOURS:MINUTES:SECOND
Returns: the newly created Node
- tool (
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.node'¶
-
__repr__
()[source]¶ Get a string representation for the Node
Returns: the string representation Return type: str
-
__setstate__
(state)[source]¶ Set the state of the Node by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
blocking
¶ Indicate that the results of this Node cannot be determined without first executing the Node, causing a blockage in the creation of jobs. A blocking Nodes causes the Chunk borders.
-
dimensions
¶
-
dimnames
¶ Names of the dimensions in the Node output. These will be reflected in the SampleIdList of this Node.
-
fullid
¶ The full defining ID for the Node inside the network
-
get_sourced_nodes
()[source]¶ A list of all Nodes connected as sources to this Node
Returns: list of all nodes that are connected to an input of this node
-
global_id
¶ The global defining ID for the Node from the main network (goes out of macro nodes to root network)
-
id
¶ The id of the Node
-
id_
= None¶ The Node id s a unique string identifying the Node
-
input_groups
¶ - A list of input groups for this Node. An input group is InputGroup
- object filled according to the Node
-
inputs
= None¶ A list of inputs of this Node
-
listeners
¶ All the listeners requesting output of this node, this means the listeners of all Outputs and SubOutputs
-
merge_dimensions
¶
-
name
¶ Name of the Tool the Node was based on. In case a Toolless Node was used the class name is given.
-
nodegroup
¶
-
outputs
= None¶ A list of outputs of this Node
-
outputsize
¶ The size of output of this SourceNode
-
parent
¶ The parent is the Network this Node is part of
-
required_cores
¶ Number of cores required for the execution of this Node
-
required_memory
¶ Amount of memory required for the execution of this Node. Follows the format d+[mMgG] so 500M or 4g would be valid ways to specify 500 megabytes or 4 gigabyte of memory.
-
required_time
¶ Amount of time required for the execution of this Node. Follows the format of a number of second or H:M:S, with H the number of hours, M the number of minutes and S the number of seconds.
-
status
¶
-
tool
¶
-
-
class
fastr.core.node.
OutputDict
(*args, **kwds)[source]¶ Bases:
collections.OrderedDict
The container containing the Inputs of Node. Only checks if the inserted values are actually outputs.
-
__module__
= 'fastr.core.node'¶
-
__setitem__
(key, value, dict_setitem=<slot wrapper '__setitem__' of 'dict' objects>)[source]¶ Set an output.
Parameters: - key (str) – the of the item to set
- value (
BaseOutput
) – the output to set - dict_setitem – the setitem function to use for the underlying OrderedDict insert
-
-
class
fastr.core.node.
SinkNode
(datatype, id_=None, parent=None, nodegroup=None)[source]¶ Bases:
fastr.core.node.Node
Class which handles where the output goes. This can be any kind of file, e.g. image files, textfiles, config files, etc.
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'SinkNode.schema.json'¶
-
__init__
(datatype, id_=None, parent=None, nodegroup=None)[source]¶ Instantiation of the SourceNode.
Parameters: - datatype – The datatype of the output.
- id – the id of the node to create
Returns: newly created sink node
usage example:
>>> import fastr >>> network = fastr.Network() >>> sink = network.create_sink(datatype=fastr.typelist['ITKImageFile'], id_='SinkN')
-
__module__
= 'fastr.core.node'¶
-
datatype
¶ The datatype of the data this sink can store.
-
input
¶ The default input of the sink Node
-
-
class
fastr.core.node.
SourceNode
(datatype, id_=None, parent=None, nodegroup=None)[source]¶ Bases:
fastr.core.node.FlowNode
Class providing a connection to data resources. This can be any kind of file, stream, database, etc from which data can be received.
-
__abstractmethods__
= frozenset([])¶
-
__dataschemafile__
= 'SourceNode.schema.json'¶
-
__eq__
(other)[source]¶ Compare two Node instances with each other. This function ignores the parent and update status, but tests rest of the dict for equality. equality
Parameters: other (Node) – the other instances to compare to Returns: True if equal, False otherwise
-
__getstate__
()[source]¶ Retrieve the state of the SourceNode
Returns: the state of the object Rtype dict:
-
__init__
(datatype, id_=None, parent=None, nodegroup=None)[source]¶ Instantiation of the SourceNode.
Parameters: - datatype – The (id of) the datatype of the output.
- id – The url pattern.
This class should never be instantiated directly (unless you know what you are doing). Instead create a source using the network class like shown in the usage example below.
usage example:
>>> import fastr >>> network = fastr.Network() >>> source = network.create_source(datatype=fastr.typelist['ITKImageFile'], id_='sourceN')
-
__module__
= 'fastr.core.node'¶
-
__setstate__
(state)[source]¶ Set the state of the SourceNode by the given state.
Parameters: state (dict) – The state to populate the object with Returns: None
-
datatype
¶ The datatype of the data this source supplies.
-
dimensions
¶ The dimensions in the SourceNode output. These will be reflected in the SampleIdLists.
-
nodegroup
¶
-
output
¶ Shorthand for
self.outputs['output']
-
set_data
(data, ids=None)[source]¶ Set the data of this source node.
Parameters: - data (dict, OrderedDict or list of urls) – the data to use
- ids – if data is a list, a list of accompanying ids
-
sourcegroup
¶
-
valid
¶ This does nothing. It only overloads the valid method of Node(). The original is intended to check if the inputs are connected to some output. Since this class does not implement inputs, it is skipped.
-
objectmanager
Module¶
This module contains the object manager class
-
class
fastr.core.objectmanager.
ObjectManager
(path)[source]¶ Bases:
fastr.core.basemanager.BaseManager
Class for managing all the objects loaded in the fastr system
-
__abstractmethods__
= frozenset(['object_class', 'get_object_version'])¶
-
__contains__
(key)[source]¶ Check if an item is in the ObjectManager
Parameters: key (str or tuple) – object id or tuple (Objectid, version) Returns: flag indicating the item is in the manager
-
__getitem__
(key)[source]¶ Retrieve a Object from the ObjectManager. You can request by only an id, which results in the newest version of the Object being returned, or request using both an id and a version.
Parameters: key (str or tuple) – object id or tuple (Objectid, version) Returns: the requested Object Raises: FastrObjectUnknownError – if a non-existing Object was requested
-
__init__
(path)[source]¶ Create a ObjectManager and scan path to search for Objects
Parameters: path (str or iterable of str) – the path(s) to scan for Objects Returns: newly created ObjectManager
-
__keytransform__
(key)[source]¶ Key transform, used for allowing indexing both by id-only and by
(id, version)
Parameters: key – key to transform Returns: key in form (id, version)
-
__module__
= 'fastr.core.objectmanager'¶
-
get_object_version
(obj)[source]¶ Get the version of a given object
Parameters: object – the object to use Returns: the version of the object
-
object_class
¶ The class of the objects to populate the manager with
-
pluginmanager
Module¶
This module contains the Manager class for Plugins in the fastr system
-
class
fastr.core.pluginmanager.
BasePluginManager
(path=None, recursive=False)[source]¶ Bases:
fastr.core.basemanager.BaseManager
Baseclass for PluginManagers, need to override the self._plugin_class
-
__abstractmethods__
= frozenset(['plugin_class'])¶
-
__getitem__
(key)[source]¶ Retrieve item from BaseManager
Parameters: key – the key of the item to retrieve Returns: the value indicated by the key Raises: FastrKeyError – if the key is not found in the BaseManager
-
__init__
(path=None, recursive=False)[source]¶ Create a BasePluginManager and scan the give path for matching plugins
Parameters: Returns: newly created plugin manager
Raises: FastrTypeError – if self._plugin_class is set to a class not subclassing BasePlugin
-
__module__
= 'fastr.core.pluginmanager'¶
-
plugin_class
¶ The class from which the plugins must be subclassed
-
-
class
fastr.core.pluginmanager.
LazyModule
(name, parent, plugin_manager)[source]¶ Bases:
module
A module that allows content to be loaded lazily from plugins. It generally is (almost) empty and gets (partially) populated when an attribute cannot be found. This allows lazy loading and plugins depending on other plugins.
-
__getattr__
(item)[source]¶ The getattr is called when getattribute does not return a value and is used as a fallback. In this case we try to find the value normally and will trigger the plugin manager if it cannot be found.
Parameters: item (str) – attribute to retrieve Returns: the requested attribute
-
__module__
= 'fastr.core.pluginmanager'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
class
fastr.core.pluginmanager.
PluginManager
(path=None)[source]¶ Bases:
fastr.core.pluginmanager.BasePluginManager
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.pluginmanager'¶
-
__setitem__
(key, value)[source]¶ Store an item in the BaseManager, will ignore the item if the key is already present in the BaseManager.
Parameters: - name – the key of the item to save
- value – the value of the item to save
Returns: None
-
plugin_class
¶ The plugin manager contains any Plugin subclass
-
-
class
fastr.core.pluginmanager.
PluginSubManager
(parent, plugin_class)[source]¶ Bases:
fastr.core.pluginmanager.BasePluginManager
A PluginManager that is a selection of a parent plugin manger. It uses the PluginsView to only exponse part of the parent PluginManager. This is used to create plugin managers for only certain types of plugins (e.g. IOPlugins) without loading them multiple times.
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.pluginmanager'¶
-
data
¶
-
plugin_class
¶ PluginSubManagers only expose the plugins of a certain class
-
-
class
fastr.core.pluginmanager.
PluginsView
(parent, plugin_class)[source]¶ Bases:
_abcoll.MutableMapping
A collection that acts like view of the plugins of another plugin manager. This is a proxy object that only gives access the plugins of a certain plugin class. It behaves like a mapping and is used as the data object for a PluginSubManager.
-
__abstractmethods__
= frozenset([])¶
-
__init__
(parent, plugin_class)[source]¶ Constructor for the plugins view
Parameters: - parent (BasePluginManager) – the parent plugin manager
- plugin_class (class) – the class of the plugins to expose
-
__module__
= 'fastr.core.pluginmanager'¶
-
-
class
fastr.core.pluginmanager.
plugin_option_type
(filename, name, namespace, id)¶ Bases:
tuple
-
__dict__
= dict_proxy({'__module__': 'fastr.core.pluginmanager', '_make': <classmethod object>, '_replace': <function _replace>, '_asdict': <function _asdict>, '__dict__': <property object>, '__getnewargs__': <function __getnewargs__>, 'id': <property object>, 'name': <property object>, '_fields': ('filename', 'name', 'namespace', 'id'), '__new__': <staticmethod object>, 'namespace': <property object>, 'filename': <property object>, '__slots__': (), '__repr__': <function __repr__>, '__getstate__': <function __getstate__>, '__doc__': 'plugin_option_type(filename, name, namespace, id)'})¶
-
__getnewargs__
()¶ Return self as a plain tuple. Used by copy and pickle.
-
__getstate__
()¶ Exclude the OrderedDict from pickling
-
__module__
= 'fastr.core.pluginmanager'¶
-
static
__new__
(_cls, filename, name, namespace, id)¶ Create new instance of plugin_option_type(filename, name, namespace, id)
-
__repr__
()¶ Return a nicely formatted representation string
-
__slots__
= ()¶
-
filename
¶ Alias for field number 0
-
id
¶ Alias for field number 3
-
name
¶ Alias for field number 1
-
namespace
¶ Alias for field number 2
-
provenance
Module¶
-
class
fastr.core.provenance.
Provenance
(host=None)[source]¶ Bases:
object
The Provenance object keeps track of everything that happens to a data object.
-
__dict__
= dict_proxy({'__module__': 'fastr.core.provenance', 'init_provenance': <function init_provenance>, 'collect_input_argument_provenance': <function collect_input_argument_provenance>, 'data_uri': <staticmethod object>, 'serialize': <function serialize>, 'get_parent_provenance': <staticmethod object>, 'agent': <function agent>, 'entity': <function entity>, '__init__': <function __init__>, 'activity': <function activity>, '__dict__': <attribute '__dict__' of 'Provenance' objects>, '_add_namespace': <function _add_namespace>, '__weakref__': <attribute '__weakref__' of 'Provenance' objects>, '__doc__': '\n The Provenance object keeps track of everything that happens to a data object.\n ', 'collect_provenance': <function collect_provenance>})¶
-
__module__
= 'fastr.core.provenance'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
static
get_parent_provenance
(value)[source]¶ Find the provenance of the parent job
Parameters: value (str) – url for the value for which to find the job
Returns: the provenance of the job that created the value
Raises: - FastrKeyError – if the deferred is not available (yet)
- FastrValueError – if the value is not a valid deferred url
-
samples
Module¶
This package holds the classes for working with samples.
-
class
fastr.core.samples.
ContainsSamples
[source]¶ Bases:
fastr.core.samples.HasSamples
-
__abstractmethods__
= frozenset(['samples'])¶
-
__module__
= 'fastr.core.samples'¶
-
dimensions
¶
-
samples
¶
-
-
class
fastr.core.samples.
HasSamples
[source]¶ Bases:
fastr.core.dimension.HasDimensions
Base class for all classes that supply samples. This base class allows to only define __getitem__ and size and get all other basic functions mixed in so that the object behaves similar to a Mapping.
-
__abstractmethods__
= frozenset(['dimensions', '__getitem__'])¶
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.samples'¶
-
-
class
fastr.core.samples.
SampleBaseId
[source]¶ Bases:
tuple
This class represents a sample id. A sample id is a multi-dimensional id that has a simple, consistent string representation.
-
__add__
(other)[source]¶ Add another SampleId, this allows to add parts to the SampleId in a convenient way.
-
__dict__
= dict_proxy({'__module__': 'fastr.core.samples', '__new__': <staticmethod object>, '__str__': <function __str__>, '__radd__': <function __radd__>, '__repr__': <function __repr__>, '__add__': <function __add__>, '__dict__': <attribute '__dict__' of 'SampleBaseId' objects>, '__doc__': '\n This class represents a sample id. A sample id is a multi-dimensional\n id that has a simple, consistent string representation.\n ', '_element_type': <type 'NoneType'>})¶
-
__module__
= 'fastr.core.samples'¶
-
static
__new__
(*args)[source]¶ Create a new SampleId
Parameters: args (iterator/iterable of element type or element type) – the strings to make sample id for
-
__radd__
(other)[source]¶ Add another SampleId, this allows to add parts to the SampleId in a convenient way. This is the right-hand version of the operator.
-
-
class
fastr.core.samples.
SampleCollection
(dimnames, parent)[source]¶ Bases:
_abcoll.MutableMapping
,fastr.core.dimension.HasDimensions
The SampleCollections is a class that contains the data including a form of ordering. Each sample is reachable both by its SampleId and a SampleIndex. The object is sparse, so not all SampleId have to be defined allowing for non-rectangular data shapes.
Note
This object is meant to replace both the SampleIdList and the ValueStorage.
-
__abstractmethods__
= frozenset([])¶
-
__contains__
(item)[source]¶ Check if an item is in the SampleCollection. The item can be a SampleId or SampleIndex. If the item is a slicing SampleIndex, then check if it would return any data (True) or no data (False)
Parameters: item (SampleId, SampleIndex) – the item to check for Returns: flag indicating item is in the collections Return type: bool
-
__delitem__
(key)[source]¶ Remove an item from the SampleCollection
Parameters: key (SampleId, SampleIndex, tuple of both, or SampleItem) – the key of the item to remove
-
__getitem__
(item)[source]¶ Retrieve (a) SampleItem(s) from the SampleCollection using the SampleId or SampleIndex. If the item is a tuple, it should be valid tuple for constructing either a SampleId or SampleIndex.
Parameters: item (SampleId, SampleIndex, or tuple) – the identifier of the item to retrieve
Returns: the requested item
Return type: Raises: - FastrTypeError – if the item parameter is of incorrect type
- KeyError – if the item is not found
-
__module__
= 'fastr.core.samples'¶
-
__setitem__
(key, value)[source]¶ Set an item to the SampleCollection. The key can be a SampleId, SampleIndex or a tuple containing a SampleId and SampleIndex. The value can be a SampleItem (with the SampleId and SampleIndex matching), a tuple with values (assuming no depending jobs), or a with a list of values and a set of depending jobs.
Parameters: - key (SampleId, SampleIndex, tuple of both, or SampleItem) – the key of the item to store
- value (SampleItem, tuple of values, or tuple of tuple of values and set of depending jobs) – the value of the SampleItem to store
Raises: - FastrTypeError – if the key or value types are incorrect
- FastrValueError – if the id or values are incorrectly formed
-
dimensions
¶
-
fullid
¶ The full defining ID for the SampleIdList
-
ndims
¶ The number of dimensions in this SampleCollection
-
parent
¶ The parent object holding the SampleCollection
-
-
class
fastr.core.samples.
SampleId
[source]¶ Bases:
fastr.core.samples.SampleBaseId
SampleId is an identifier for data using human readable strings
-
__module__
= 'fastr.core.samples'¶
-
-
class
fastr.core.samples.
SampleIndex
[source]¶ Bases:
fastr.core.samples.SampleBaseId
SampleId is an identifier for data using the location in the N-d data structure.
-
__module__
= 'fastr.core.samples'¶
-
__repr__
()[source]¶ Get a string representation for the SampleIndex
Returns: the string representation Return type: str
-
__str__
()[source]¶ Get a string version for the SampleId, joins the SampleId with __ to create a single string version.
Returns: the string version Return type: str
-
expand
(size)[source]¶ Function expanding a slice SampleIndex into a list of non-slice SampleIndex objects
Parameters: size – the size of the collection to slice
-
isslice
¶ Flag indicating that the SampleIndex is a slice (as opposed to a simple single index).
-
-
class
fastr.core.samples.
SampleItem
[source]¶ Bases:
fastr.core.samples.SampleItemBase
-
__module__
= 'fastr.core.samples'¶
-
static
__new__
(index, id_, data, jobs=None, failed_annotations=None)[source]¶ Create a SampleItem. Data should be an OrderedDict of tuples.
Parameters: - index (tuple, slice) – the sample index
- id (
SampleId
) – the sample id - data (SampleValue, Mapping) – the data values
- jobs (set) – set of jobs on which this SampleItems data depends.
- failed_annotations (set) – set of tuples. The tuple is contructed like follows: (job_id, reason).
-
-
class
fastr.core.samples.
SampleItemBase
[source]¶ Bases:
tuple
This class represents a sample item, a combination of a SampleIndex, SampleID, value and required jobs. The SampleItem based on a named tuple and has some extra methods to combine SampleItems easily.
-
__add__
(other)[source]¶ The addition operator combines two SampleItems into a single SampleItems. It merges the data and jobs and takes the index and id of the left-hand item.
Parameters: other (SampleItem) – The other item to add to this one Returns: the combined SampleItem Return type: SampleItem
-
__dict__
= dict_proxy({'index': <property object>, '__module__': 'fastr.core.samples', 'dimensionality': <property object>, 'jobs': <property object>, '__new__': <staticmethod object>, 'data': <property object>, '__doc__': '\n This class represents a sample item, a combination of a SampleIndex,\n SampleID, value and required jobs. The SampleItem based on a named\n tuple and has some extra methods to combine SampleItems easily.\n ', 'combine': <staticmethod object>, '__repr__': <function __repr__>, '__add__': <function __add__>, '__dict__': <attribute '__dict__' of 'SampleItemBase' objects>, 'cardinality': <property object>, '__getnewargs__': <function __getnewargs__>, 'id': <property object>, 'failed_annotations': <property object>})¶
-
__getnewargs__
()[source]¶ Get new args gives the arguments to use to re-create this object, This is used for serialization.
-
__module__
= 'fastr.core.samples'¶
-
static
__new__
(index, id_, data, jobs=None, failed_annotations=None)[source]¶ Create a SampleItem. Data should be an OrderedDict of tuples.
Parameters: - index (tuple, slice) – the sample index
- id (
SampleId
) – the sample id - data (SampleValue, Mapping) – the data values
- jobs (set) – set, tuple or list of jobs on which this SampleItems data depends.
- failed_annotations (set) – set of tuples. The tuple is contructed like follows: (job_id, reason).
-
__repr__
()[source]¶ Get a string representation for the SampleItem
Returns: the string representation Return type: str
-
cardinality
¶ The cardinality of this Sample
-
static
combine
(*args)[source]¶ Combine a number of SampleItems into a new one.
Parameters: *args – the SampleItems to combine
Returns: the combined SampleItem Return type: SampleItem It is possible to both give multiple arguments, where each argument is a SampleItem, or a single argument which is an iterable yielding SampleItems.
# variables a, b, c, d are SampleItems to combine # These are all valid ways of combining the SampleItems comb1 = SampleItem.combine(a, b, c, d) # Using multiple arguments l = [a, b, c, d] comb2 = SampleItem.combine(l) # Using a list of arguments comb3 = SampleItem.combine(l.__iter__()) # Using an iterator
-
data
¶ The data SampleValue of the SampleItem
Returns: The value of this SampleItem Return type: SampleValue
-
dimensionality
¶ The dimensionality of this Sample
-
failed_annotations
¶
-
index
¶ The index of the SampleItem
Returns: The index of this SampleItem Return type: SampleIndex
-
-
class
fastr.core.samples.
SamplePayload
[source]¶ Bases:
fastr.core.samples.SampleItemBase
-
__add__
(other)[source]¶ The addition operator combines two SampleItems into a single SampleItems. It merges the data and jobs and takes the index and id of the left-hand item.
Parameters: other (SampleItem) – The other item to add to this one Returns: the combined SamplePayload Return type: SamplePayload
-
__module__
= 'fastr.core.samples'¶
-
static
__new__
(index, id_, data, jobs=None, failed_annotations=None)[source]¶ Create a SampleItem. Data should be an OrderedDict of tuples.
Parameters: - index (tuple, slice) – the sample index
- id (
SampleId
) – the sample id - data (SampleValue, Mapping) – the data values
- jobs (set) – set of jobs on which this SampleItems data depends.
- failed_annotations (set) – set of tuples. The tuple is contructed like follows: (job_id, reason).
-
serializable
Module¶
This package contains the base class and meta class for all serializable objects in the Fastr system.
-
class
fastr.core.serializable.
PassThroughSerializer
[source]¶ Bases:
object
-
__dict__
= dict_proxy({'__module__': 'fastr.core.serializable', 'dumps': <staticmethod object>, '__dict__': <attribute '__dict__' of 'PassThroughSerializer' objects>, 'loads': <staticmethod object>, '__weakref__': <attribute '__weakref__' of 'PassThroughSerializer' objects>, '__doc__': None})¶
-
__module__
= 'fastr.core.serializable'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
class
fastr.core.serializable.
Serializable
[source]¶ Bases:
object
Superclass for all classes that can be serialized.
-
SERIALIZERS
= {'/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/schemas/FastrInterface.schema.json': <fastr.utils.jsonschemaparser.Blueprinter object>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/schemas/Tool.schema.json': <fastr.utils.jsonschemaparser.Blueprinter object>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/schemas/Network.schema.json': <fastr.utils.jsonschemaparser.Blueprinter object>}¶
-
__dict__
= dict_proxy({'load': <classmethod object>, '__module__': 'fastr.core.serializable', 'SERIALIZERS': {'/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/schemas/FastrInterface.schema.json': <fastr.utils.jsonschemaparser.Blueprinter object>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/schemas/Tool.schema.json': <fastr.utils.jsonschemaparser.Blueprinter object>, '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/schemas/Network.schema.json': <fastr.utils.jsonschemaparser.Blueprinter object>}, 'dump': <function dump>, '_unicode_convert': <classmethod object>, 'get_serializer': <classmethod object>, 'dumpfuncs': {'xml': <module 'fastr.utils.xmltodict' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/utils/xmltodict.pyc'>, 'pickle': <module 'pickle' from '/home/docs/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc'>, 'json': <module 'json' from '/home/docs/.pyenv/versions/2.7.13/lib/python2.7/json/__init__.pyc'>, 'dict': <class 'fastr.core.serializable.PassThroughSerializer'>, 'marshall': <module 'marshal' (built-in)>}, '_dumps': <classmethod object>, '_dump': <classmethod object>, '__dict__': <attribute '__dict__' of 'Serializable' objects>, 'loadf': <classmethod object>, '_unserialize': <classmethod object>, '_load': <classmethod object>, '_dumpf': <classmethod object>, '_loads': <classmethod object>, 'dumpf': <function dumpf>, '__weakref__': <attribute '__weakref__' of 'Serializable' objects>, '__doc__': '\n Superclass for all classes that can be serialized.\n ', 'dumps': <function dumps>, '_loadf': <classmethod object>, '_serialize': <function _serialize>, 'loads': <classmethod object>, '__getstate__': <function __getstate__>, 'createobj': <classmethod object>})¶
-
__module__
= 'fastr.core.serializable'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
classmethod
createobj
(state, _=None)[source]¶ Create object function for generic objects
Parameters: - cls – The class to create
- state – The state to use to create the Link
- network – the parent Network
Returns: newly created Link
-
dump
(file_handle, method='json', **kwargs)[source]¶ Dump the object to a file like object.
Parameters: - file_handle – file descriptor to write the data to
- method (str) – method of final serialization to use (e.g. json, xml, pickle)
- kwargs – extra arguments passed to the final serializer
-
dumpf
(path, method=None, **kwargs)[source]¶ Dump the object to a file
Parameters: - path – path where to write the file
- method (str) – method of final serialization to use (e.g. json, xml, pickle)
- kwargs – extra arguments passed to the final serializer
Note
The dumpf function can determine the method based on the desired output filename. Also, if the filename ends with .gz it will continue search for another extension (so .json.gz could be found) and will then compress the result with gzip.
-
dumpfuncs
= {'xml': <module 'fastr.utils.xmltodict' from '/home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/utils/xmltodict.pyc'>, 'pickle': <module 'pickle' from '/home/docs/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc'>, 'json': <module 'json' from '/home/docs/.pyenv/versions/2.7.13/lib/python2.7/json/__init__.pyc'>, 'dict': <class 'fastr.core.serializable.PassThroughSerializer'>, 'marshall': <module 'marshal' (built-in)>}¶
-
dumps
(method='json', **kwargs)[source]¶ Dump the object to a string
Parameters: - method (str) – method of final serialization to use (e.g. json, xml, pickle)
- kwargs – extra arguments passed to the final serializer
Returns: serialization string
Return type:
-
classmethod
load
(file_handle, method=None, network=None, **kwargs)[source]¶ Load the object from a file-like object
Parameters: - cls – class of the object
- file_handle – file descriptor to write the data to
- method (str) – method of final serialization to use (e.g. json, xml, pickle)
- network – network in which to place the loaded object
- kwargs – extra arguments passed to the final serializer
Returns: newly created object
Warning
Unlike the loadf functions, this function does not automatically detect gzip compression. You read a gzip using the gzip.open method, but not but simply opening a stream and hopeing this function will function.
-
classmethod
loadf
(path, method=None, network=None, **kwargs)[source]¶ Load the object from a file
Parameters: - cls – class of the object
- path – path where to write the file
- method (str) – method of final serialization to use (e.g. json, xml, pickle)
- network – network in which to place the loaded object
- kwargs – extra arguments passed to the final serializer
Returns: newly created object
Note
The loadf function can determine the method of loading based on the filename. Also it can automatically determine whether a file is gzipped.
-
target
Module¶
The module containing the classes describing the targets.
-
class
fastr.core.target.
ProcessUsageCollection
[source]¶ Bases:
_abcoll.Sequence
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.target'¶
-
usage_type
¶ alias of
SystemUsageInfo
-
-
class
fastr.core.target.
SystemUsageInfo
(timestamp, cpu_percent, vmem, rmem, read_bytes, write_bytes)¶ Bases:
tuple
-
__dict__
= dict_proxy({'__module__': 'fastr.core.target', '_make': <classmethod object>, 'timestamp': <property object>, '_replace': <function _replace>, 'read_bytes': <property object>, '_asdict': <function _asdict>, 'rmem': <property object>, '__dict__': <property object>, '__getnewargs__': <function __getnewargs__>, 'write_bytes': <property object>, '_fields': ('timestamp', 'cpu_percent', 'vmem', 'rmem', 'read_bytes', 'write_bytes'), '__new__': <staticmethod object>, 'cpu_percent': <property object>, 'vmem': <property object>, '__slots__': (), '__repr__': <function __repr__>, '__getstate__': <function __getstate__>, '__doc__': 'SystemUsageInfo(timestamp, cpu_percent, vmem, rmem, read_bytes, write_bytes)'})¶
-
__getnewargs__
()¶ Return self as a plain tuple. Used by copy and pickle.
-
__getstate__
()¶ Exclude the OrderedDict from pickling
-
__module__
= 'fastr.core.target'¶
-
static
__new__
(_cls, timestamp, cpu_percent, vmem, rmem, read_bytes, write_bytes)¶ Create new instance of SystemUsageInfo(timestamp, cpu_percent, vmem, rmem, read_bytes, write_bytes)
-
__repr__
()¶ Return a nicely formatted representation string
-
__slots__
= ()¶
-
cpu_percent
¶ Alias for field number 1
-
read_bytes
¶ Alias for field number 4
-
rmem
¶ Alias for field number 3
-
timestamp
¶ Alias for field number 0
-
vmem
¶ Alias for field number 2
-
write_bytes
¶ Alias for field number 5
-
-
class
fastr.core.target.
Target
[source]¶ Bases:
fastr.core.baseplugin.Plugin
The abstract base class for all targets. Execution with a target should follow the following pattern:
>>> with Target() as target: ... target.run_commmand(['sleep', '10']) ... target.run_commmand(['sleep', '10']) ... target.run_commmand(['sleep', '10'])
The Target context operator will set the correct paths/initialization. Within the context command can be ran and when leaving the context the target reverts the state before.
-
__abstractmethods__
= frozenset(['run_command'])¶
-
__metaclass__
¶ alias of
ABCMeta
-
__module__
= 'fastr.core.target'¶
-
-
class
fastr.core.target.
TargetManager
[source]¶ Bases:
fastr.core.pluginmanager.PluginSubManager
Container holding all the ExecutionPlugins known to the Fastr system
-
__abstractmethods__
= frozenset([])¶
-
__init__
()[source]¶ Initialize a ExecutionPluginManager and load plugins.
Parameters: - path – path to search for plugins
- recursive – flag for searching recursively
Returns: newly created ExecutionPluginManager
-
__module__
= 'fastr.core.target'¶
-
tool
Module¶
A module to maintain a tool.
Exported classes:
- Tool – A class encapsulating a tool.
- ParameterDescription – The base class containing the shared description of a parameter (both input and ouput).
- InputParameterDescription – A class containing the description of an input parameter.
- Output ParameterDescription – A class containing the description of an output parameter.
-
class
fastr.core.tool.
Tool
(doc=None)[source]¶ Bases:
fastr.core.serializable.Serializable
The class encapsulating a tool.
-
TOOL_REFERENCE_FILE_NAME
= '__fastr_tool_ref__.json'¶
-
TOOL_RESULT_FILE_NAME
= '__fastr_tool_result.pickle.gz'¶
-
__dataschemafile__
= 'Tool.schema.json'¶
-
__eq__
(other)[source]¶ Compare two Tool instances with each other.
Parameters: other (Tool) – the other instances to compare to Returns: True if equal, False otherwise
-
__init__
(doc=None)[source]¶ Create a new Tool :param doc: path of toolfile or a dict containing the tool data :type doc: str or dict
-
__module__
= 'fastr.core.tool'¶
-
__repr__
()[source]¶ Get a string representation for the Tool. This will show the inputs and output defined in a table-like structure.
Returns: the string representation Return type: str
-
__setstate__
(state)[source]¶ Set the state of the Tool by the given state.
Parameters: state (dict) – The state to populate the object with
List of authors of the tool. These people wrapped the executable but are not responsible for executable itself.
-
cite
= None¶ This holds the citation you should use when publishing something based on this Tool
-
command
= None¶ Command is a dictionary contain information about the command which is called by this Tool: command[‘interpreter’] holds the (possible) interpreter to use command[‘targets’] holds a per os/arch dictionary of files that should be executed command[‘url’] is the webpage of the command to be called command[‘version’] is the version of the command used command[‘description’] can help a description of the command command[‘authors’] lists the original authors of the command
-
command_version
¶
-
static
compare_output_data
(current_output_data, reference_output_data, validation_result, output)[source]¶
-
description
= None¶ Description of the tool and it’s functionality
-
execute
(payload=None, **kwargs)[source]¶ Execute a Tool given the payload for a single run
Parameters: payload – the data to execute the Tool with Returns: The result of the execution Return type: InterFaceResult
-
fullid
¶ The full id of this tool
-
hash
¶
-
help
= None¶ Man page for the Tool. Here usage and examples can be described in detail
-
inputs
¶
-
interface_class
= None¶ Create the Interface based on the class specified in the tool file
-
name
= None¶ Name of the tool, this should be a descriptive, human readable name.
-
namespace
= None¶ The namespace this tools lives in, this will be set by the ToolManager on load
-
node_class
= None¶ Class for of the Node to use
-
ns_id
¶ The namespace and id of the Tool
-
outputs
¶
-
path
¶ The path of the directory in which the tool definition file was located.
-
references
= None¶ A list of documents and in depth reading about the methods used in this tool
-
regex
= None¶ Identifier for the tool
-
requirements
= None¶ Requirements for this Tool
Warning
Not yet implemented
List of tags for this tool
-
target
¶ The OS and arch matched target definition.
-
test_spec
¶ alias of
TestSpecification
-
classmethod
test_tool
(reference_data_dir, tool=None, input_data=None)[source]¶ Execute the tool with the input data specified and test the results against the refence data. This effectively tests the tool execution.
Parameters:
-
tests
= None¶ Test for this tool. A test should be a collection of inputs, parameters and outputs to verify the proper functioning of the Tool.
The format of the tests is a list of namedtuples, that have 3 fields: - input: a dict of the input data - command: a list given the expected command-line arguments - output: a dict of the output data to validate
Warning
Not yet implemented
-
url
= None¶ URL to website where this tool can be downloaded from
-
version
= None¶ Version of the tool, not of the underlying software
-
toolmanager
Module¶
This module contains the tool manager class
-
class
fastr.core.toolmanager.
ToolManager
(path)[source]¶ Bases:
fastr.core.objectmanager.ObjectManager
-
__abstractmethods__
= frozenset([])¶
-
__module__
= 'fastr.core.toolmanager'¶
-
object_class
¶
-
-
fastr.core.toolmanager.
toollist
= ToolManager fastr.Sink v1.0 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/core/ioplugin.pyc fastr.Source v1.0 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/core/ioplugin.pyc fastr.flow.CrossValidation v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/flow/0.1/crossvalidation.xml fastr.math.Add v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/add.xml fastr.math.AddInt v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/addint.xml fastr.math.Divide v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/divide.xml fastr.math.IntegerDivide v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/intdivide.xml fastr.math.Max v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/max.xml fastr.math.Min v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/min.xml fastr.math.Multiply v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/multiply.xml fastr.math.Subtract v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/subtract.xml fastr.math.Sum v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/math/0.1/sum.xml fastr.test.0.1.BetDocker v4.1.9 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/test/0/1/bet_docker.xml fastr.util.AutoPrefixNegateTest v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/util/0.1/auto_prefix_negate.xml fastr.util.AutoPrefixTest v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/util/0.1/auto_prefix.xml fastr.util.Delay v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/util/0.1/delay.xml fastr.util.Fail v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/util/0.1/fail.xml fastr.util.PassThroughAuto v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/util/0.1/passthroughauto.xml fastr.util.Range v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/util/0.1/range.xml fastr.util.StdOut v0.1 : /home/docs/checkouts/readthedocs.org/user_builds/fastr/envs/2.0.0/lib/python2.7/site-packages/fastr-2.0.0-py2.7.egg/fastr/resources/tools/fastr/util/0.1/stdout.xml ¶ The fastr toollist
updateable
Module¶
This module contains both the Updateable class and UpdateableMeta meta-class for objects which support updates within the fastr system
-
class
fastr.core.updateable.
Updateable
[source]¶ Bases:
object
Super class for all classes that can be updated and have a status. These objects can be valid/invalid state. These states are set by the function update. This allows for interactively checking the network.
-
__abstractmethods__
= frozenset(['_update'])¶
-
__dict__
= dict_proxy({'_abc_cache': <_weakrefset.WeakSet object>, '__module__': 'fastr.core.updateable', '__updateinprogress__': <thread.lock object>, '__metaclass__': <class 'fastr.core.updateable.UpdateableMeta'>, '__updating__': True, '__setstate__': <function __setstate__>, '__weakref__': <attribute '__weakref__' of 'Updateable' objects>, '__updatefunc__': <function update>, 'messages': <property object>, 'update': <function update>, '_abc_negative_cache': <_weakrefset.WeakSet object>, '_abc_registry': <_weakrefset.WeakSet object>, 'valid': <property object>, '_abc_negative_cache_version': 34, '__updatetriggers__': [], '__abstractmethods__': frozenset(['_update']), '__dict__': <attribute '__dict__' of 'Updateable' objects>, '__getstate__': <function __getstate__>, '_update': <function _update>, '__doc__': '\n Super class for all classes that can be updated and have a status.\n These objects can be valid/invalid state. These states are set by\n the function update. This allows for interactively checking the network.\n ', '__init__': <function __init__>})¶
-
__getstate__
()[source]¶ Retrieve the state of the object, make sure the status is not part of the description as it will not be valid after re-creating the object.
Returns: the state of the object Rtype dict:
-
__metaclass__
¶ alias of
UpdateableMeta
-
__module__
= 'fastr.core.updateable'¶
-
__setstate__
(state)[source]¶ Set the state of the object by the given state. This adds a clean status field, making sure it is not unintended, outdated information from before serialization.
Parameters: state (dict) – The state to populate the object with
-
__updatefunc__
(key=None, forward=True, backward=False)¶ Default function for updating, it can be called without key to have a new update started with a new key.
Parameters:
-
__updateinprogress__
= <thread.lock object>¶ Lock to avoid multiple updates happening at the same time
-
__updatetriggers__
= []¶ Which methods need to be wrapped to trigger an update. Override this value to have the functions automatically wrapped. E.g.
__update_triggers__ = ['append', 'insert', '__setitem__']
to have these functions wrapped.
-
__updating__
= True¶ Flag to indicate that this object is allowed to update
-
__weakref__
¶ list of weak references to the object (if defined)
-
messages
¶ The messages of the last update
-
update
(key=None, forward=True, backward=False)[source]¶ Default function for updating, it can be called without key to have a new update started with a new key.
Parameters:
-
valid
¶ Flag indicating that the object is valid
-
-
class
fastr.core.updateable.
UpdateableMeta
[source]¶ Bases:
abc.ABCMeta
A metaclass for objects which are updateable and need some methods/properties to trigger an update.
-
__module__
= 'fastr.core.updateable'¶
-
classmethod
calcmro
(mcs, bases)[source]¶ Calculate the Method Resolution Order of bases using the C3 algorithm.
Suppose you intended creating a class K with the given base classes. This function returns the MRO which K would have, excluding K itself (since it doesn’t yet exist), as if you had actually created the class.
Another way of looking at this, if you pass a single class K, this will return the linearization of K (the MRO of K, including itself).
Parameters: bases – the list of bases for which create the MRO Returns: the list representing the entire MRO, except the (non-existing) class itself - Note: Taken from http://code.activestate.com/recipes/577748-calculate-the-mro-of-a-class/
- Created by Steven D’Aprano and licensed under the MIT license
-
classmethod
find_member
(mcs, name, parents, dct)[source]¶ Find a member of the class in the same way as Python would if it had a given dict and set of bases
Parameters: - mcs – metaclass at work
- name – name of the class to be created
- parents – list of the bases for the new class
- dct – the dict of the class being created
Returns: the firstly resolved member or None if nothing found
-
version
Module¶
Module containing the class that represent versions
-
class
fastr.core.version.
Version
[source]¶ Bases:
tuple
Class representing a software version definition. Allows for sorting and extraction of parts.
-
__dict__
= dict_proxy({'status': <property object>, '__module__': 'fastr.core.version', '__new__': <staticmethod object>, 'extra': <property object>, '__str__': <function __str__>, 'major': <property object>, '__dict__': <attribute '__dict__' of 'Version' objects>, 'build': <property object>, '__repr__': <function __repr__>, 'extra_string': <property object>, 'version_matcher': <_sre.SRE_Pattern object at 0x334c740>, 'date_version_matcher': <_sre.SRE_Pattern object>, '__doc__': '\n Class representing a software version definition. Allows for sorting and\n extraction of parts.\n ', 'minor': <property object>, 'suffix': <property object>})¶
-
__module__
= 'fastr.core.version'¶
-
static
__new__
(*version)[source]¶ Class containing a version
Can be constructed by:
Version( 'major.$minor.$extra[0].$extra[1]$seperator$status$build$suffix' ) Version( major, minor, extra, status, build, suffix, seperator ) Version( (major, minor, extra, status, build, suffix, seperator) ) Version( [major, minor, extra, status, build, suffix, seperator] )
Parameters: - major (int) – interger giving major version
- minor (int) – is an integer (required)
- extra (list of int) – is a list of integers
- status (str) – can be “a”, “alpha”, “b”, “beta”, “rc”, or “r”
- build (int) – is an integer
- suffix (str) – can contain any combination of alpha-numeric character and ”._-“
- seperator (str) – is any of ”.”, “-”, or “_”, which is located between $extra and $build
Note
The method based on strings is the recommended method. For strings the major and minor version are required, where for tuple and list constructors all seven elements are optional.
Examples:
>>> a = Version('0.1') >>> print(tuple(a)) (0, 1, None, None, None, '', None) >>> b = Version('2.5.3-rc2') >>> print(tuple(b)) (2, 5, [3], 'rc', 2, '', '-') >>> c = Version('1.2.3.4.5.6.7-beta8_with_suffix') >>> print(tuple(c)) (1, 2, [3, 4, 5, 6, 7], 'beta', 8, '_with_suffix', '-')
-
build
¶ the build number, this is following the status (e.g. for 3.2-beta4, this would be 4)
-
date_version_matcher
= <_sre.SRE_Pattern object>¶
-
extra
¶ extra version extension as a list
-
extra_string
¶ extra version extension as a string
-
major
¶ major version
-
minor
¶ minor version
-
status
¶ the status of the version (a, alpha, b, beta, rc or r)
-
suffix
¶ the remainder of the version which was not formatted in a known way
-
version_matcher
= <_sre.SRE_Pattern object at 0x334c740>¶
-
vfs
Module¶
This module contains the virtual file system code. This is both an internally used object as well as an IOPlugin.