Command Line Tools

Fastr is shipped with a number of command line tools to perform common tasks and greatly simplify things such as debugging. The list of command line tools that is included in Fastr:

command description
cat Print information from a job file
dump Dump the contents of a network run tempdir into a zip for remote assistance
execute Execute a fastr job file
extract_argparse Create a stub for a Tool based on a python script using argparse
provenance Get PROV information from the result pickle.
pylint Dump the contents of a network run tempdir into a zip for remote assistance
run Run a Network from the commandline
sink Command line access to the IOPlugin sink
source Command line access to the IOPlugin source
test Run the tests of a tool to verify the proper function
trace Trace samples/sinks from a run
upgrade Print information from a job file
verify Verify fastr resources, at the moment only tool definitions are supported.
webapp Start the fastr webapp and open in a new browser tab

fastr cat

Extract selected information from the extra job info. The path is the selection of the data to retrieve. Every parts of the path (separated by a /) is seen as the index for the previous object. So for example to get the stdout of a job, you could use ‘fastr cat __fastr_extra_job_info__.json process/stdout’.

usage: fastr cat [-h] __fastr_extra_job_info__.json path

Positional Arguments

__fastr_extra_job_info__.json
 result file to cat
path path of the data to print

fastr dump

Create a dump of a network run directory that contains the most important information for debugging. This includes a serialization of the network, all the job command and result files, the extra job information files and the provenance files. No data files will be included, but note that if jobs get sensitive information passed via the command line this will be included in the job files.

usage: fastr dump [-h] RUNDIR DUMP.zip

Positional Arguments

RUNDIR The run directory to dump
DUMP.zip The file to place the dump in

fastr execute

Execute a job from commandline.

usage: fastr execute [-h] [JOBFILE]

Positional Arguments

JOBFILE File of the job to execute (default ./__fastr_command__.pickle.gz)

fastr extract_argparse

Extract basic information from argparse.

usage: fastr extract_argparse [-h] SCRIPT.py TOOL.xml

Positional Arguments

SCRIPT.py Python script to inspect
TOOL.xml created Tool stub

fastr provenance

Export the provenance information from JSON to other formats or plot the provenance data as a graph.

usage: fastr provenance [-h] [-so SYNTAX_OUT_FILE] [-sf SYNTAX_FORMAT]
                        [-i INDENT] [-vo VISUALIZE_OUT_FILE]
                        [RESULTFILE]

Positional Arguments

RESULTFILE File of the job to execute (default ./__fastr_prov__.json)

Named Arguments

-so, --syntax-out-file
 Write the syntax to file.
-sf, --syntax-format
 

Choices are: [json], provn or xml

Default: “json”

-i, --indent

Indent size of the serialized documents.

Default: 2

-vo, --visualize-out-file
 Visualize the provenance. The most preferred format is svg. You can specify any format pydot supports. Specify the format by postfixing the filename with an extension.

fastr pylint

Run pylint in such a way that the output is written to a file

usage: fastr pylint [-h] --output_file PYLINT.OUT

Named Arguments

--output_file The file to result in

fastr run

Execute a job or network from commandline.

usage: fastr run [-h] NETWORKFILE

Positional Arguments

NETWORKFILE File of the network to execute

fastr sink

executes an ioplugin

usage: fastr sink [-h] -i INPUT [INPUT ...] -o OUTPUT [OUTPUT ...]
                  [-d DATATYPE [DATATYPE ...]]

Named Arguments

-i, --input The url to process (can also be a list)
-o, --output The output urls in vfs scheme (can also be a list and should be the same size as –inurl)
-d, --datatype The datatype of the source/sink data to handle

fastr source

Executes an source command

usage: fastr source [-h] -i INPUT [INPUT ...] -o OUTPUT [-d DATATYPE]
                    [-s SAMPLE_ID]

Named Arguments

-i, --input The url to process (can also be a list)
-o, --output The output url in vfs scheme
-d, --datatype The datatype of the source/sink data to handle
-s, --sample_id
 The sample_id of the source/sink data to handle

fastr test

Run a tests for a fastr resource.

usage: fastr test [-h] {tool,tools,network,networks} ...

Sub-commands:

tool

Test a single tool

fastr test tool [-h] TOOL
Positional Arguments
TOOL Tool to test or directory with tool reference data

tools

Test all tools known to fastr

fastr test tools [-h]

network

Test a single network

fastr test network [-h] NETWORK
Positional Arguments
NETWORK The reference data to test the Network

networks

Test all network references inside subdirectories

fastr test networks [-h] [--result RESULT.json] REFERENCE
Positional Arguments
REFERENCE path of the directory containing subdirectories with reference data
Named Arguments
--result Write the results of the test to a JSON file

fastr trace

Fastr trace helps you inspect the staging directory of the Network run and pinpoint the errors.

usage: fastr trace [-h] [--verbose] [--sinks [SINKS [SINKS ...]]]
                   [--samples [SAMPLES [SAMPLES ...]]]
                   [__sink_data__.json]

Positional Arguments

__sink_data__.json
 

result file to cat

Default: “/home/docs/checkouts/readthedocs.org/user_builds/fastr/checkouts/stable/fastr/doc/__sink_data__.json”

Named Arguments

--verbose, -v

set verbose output for more details

Default: False

--sinks list results for specified sinks
--samples list result for all samples

fastr upgrade

Extract selected information from the extra job info. The path is the selection of the data to retrieve. Every parts of the path (separated by a /) is seen as the index for the previous object. So for example to get the stdout of a job, you could use ‘fastr cat __fastr_extra_job_info__.json process/stdout’.

usage: fastr upgrade [-h] NETWORK.py NEW.py

Positional Arguments

NETWORK.py Network creation file (in python) to upgrade
NEW.py location of the result file

fastr verify

Verify fastr resources, at the moment only tool definitionsare supported.

usage: fastr verify [-h] TYPE path

Positional Arguments

TYPE

Possible choices: tool

Type of resource to verify (e.g. tool)

path path of the resource to verify

fastr webapp

Starts the fastr web client.

usage: fastr webapp [-h] [-d] [-o]

Named Arguments

-d, --debug

Debug mode.

Default: False

-o, --openpage

Open web page after start.

Default: False