Command Line Tools

Fastr is shipped with a number of command line tools to perform common tasks and greatly simplify things such as debugging. The list of command line tools that is included in Fastr:

command

description

cat

Print information from a job file

dump

Dump the contents of a network run tempdir into a zip for remote assistance

execute

Execute a fastr job file

extract_argparse

Create a stub for a Tool based on a python script using argparse

provenance

Get PROV information from the result pickle.

pylint

Tiny wrapper in pylint so the output can be saved to a file (for test automation)

report

Print report of a job result (__fastr_result__.pickle.gz) file

run

Run a Network from the commandline

sink

Command line access to the IOPlugin sink

source

Command line access to the IOPlugin source

test

Run the tests of a tool to verify the proper function

trace

Trace samples/sinks from a run

upgrade

Upgrade a fastr 2.x python file to fastr 3.x syntax

verify

Verify fastr resources, at the moment only tool definitions are supported.

webapp

Start the fastr webapp and open in a new browser tab

fastr cat

Extract selected information from the extra job info. The path is the selection of the data to retrieve. Every parts of the path (separated by a /) is seen as the index for the previous object. So for example to get the stdout of a job, you could use ‘fastr cat __fastr_extra_job_info__.json process/stdout’.

usage: fastr cat [-h] __fastr_extra_job_info__.json path

Positional Arguments

__fastr_extra_job_info__.json

result file to cat

path

path of the data to print

fastr dump

Create a dump of a network run directory that contains the most important information for debugging. This includes a serialization of the network, all the job command and result files, the extra job information files and the provenance files. No data files will be included, but note that if jobs get sensitive information passed via the command line this will be included in the job files.

usage: fastr dump [-h] RUNDIR DUMP.zip

Positional Arguments

RUNDIR

The run directory to dump

DUMP.zip

The file to place the dump in

fastr execute

Execute a job from commandline.

usage: fastr execute [-h] [JOBFILE]

Positional Arguments

JOBFILE

File of the job to execute (default ./__fastr_command__.pickle.gz)

fastr extract_argparse

Extract basic information from argparse.

usage: fastr extract_argparse [-h] SCRIPT.py TOOL.xml

Positional Arguments

SCRIPT.py

Python script to inspect

TOOL.xml

created Tool stub

fastr provenance

Export the provenance information from JSON to other formats or plot the provenance data as a graph.

usage: fastr provenance [-h] [-so SYNTAX_OUT_FILE] [-sf SYNTAX_FORMAT]
                        [-i INDENT] [-vo VISUALIZE_OUT_FILE]
                        [RESULTFILE]

Positional Arguments

RESULTFILE

File of the job to execute (default ./__fastr_prov__.json)

Named Arguments

-so, --syntax-out-file

Write the syntax to file.

-sf, --syntax-format

Choices are: [json], provn or xml

Default: “json”

-i, --indent

Indent size of the serialized documents.

Default: 2

-vo, --visualize-out-file

Visualize the provenance. The most preferred format is svg. You can specify any format pydot supports. Specify the format by postfixing the filename with an extension.

fastr pylint

Run pylint in such a way that the output is written to a file

usage: fastr pylint [-h] --output_file PYLINT.OUT

Named Arguments

--output_file

The file to result in

fastr report

Print a report of a job result file.

usage: fastr report [-h] [JOBFILE]

Positional Arguments

JOBFILE

File of the job to execute (default ./__fastr_command__.pickle.gz)

fastr run

Execute a job or network from commandline.

usage: fastr run [-h] NETWORKFILE

Positional Arguments

NETWORKFILE

File of the network to execute

fastr sink

executes an ioplugin

usage: fastr sink [-h] -i INPUT [INPUT ...] -o OUTPUT [OUTPUT ...]
                  [-d DATATYPE [DATATYPE ...]]

Named Arguments

-i, --input

The url to process (can also be a list)

-o, --output

The output urls in vfs scheme (can also be a list and should be the same size as –inurl)

-d, --datatype

The datatype of the source/sink data to handle

fastr source

Executes an source command

usage: fastr source [-h] -i INPUT [INPUT ...] -o OUTPUT [-d DATATYPE]
                    [-s SAMPLE_ID]

Named Arguments

-i, --input

The url to process (can also be a list)

-o, --output

The output url in vfs scheme

-d, --datatype

The datatype of the source/sink data to handle

-s, --sample_id

The sample_id of the source/sink data to handle

fastr test

Run a tests for a fastr resource.

usage: fastr test [-h] {tool,tools,network,networks} ...

Sub-commands:

tool

Test a single tool

fastr test tool [-h] TOOL
Positional Arguments
TOOL

Tool to test or directory with tool reference data

tools

Test all tools known to fastr

fastr test tools [-h]

network

Test a single network

fastr test network [-h] NETWORK
Positional Arguments
NETWORK

The reference data to test the Network

networks

Test all network references inside subdirectories

fastr test networks [-h] [--result RESULT.json] REFERENCE
Positional Arguments
REFERENCE

path of the directory containing subdirectories with reference data

Named Arguments
--result

Write the results of the test to a JSON file

fastr trace

Fastr trace helps you inspect the staging directory of the Network run and pinpoint the errors.

usage: fastr trace [-h] [--verbose] [--sinks [SINKS [SINKS ...]]]
                   [--samples [SAMPLES [SAMPLES ...]]]
                   [__sink_data__.json]

Positional Arguments

__sink_data__.json

result file to cat

Default: “/home/docs/checkouts/readthedocs.org/user_builds/fastr/checkouts/develop/fastr/doc/__sink_data__.json”

Named Arguments

--verbose, -v

set verbose output for more details

Default: False

--sinks

list results for specified sinks

--samples

list result for all samples

fastr upgrade

Upgrades a python file that creates a Network to the new fastr 3.x syntax. The file will be parsed and the full syntax tree will be transformed to fit the new syntax.

Note

Solves most common problems, but cannot always solve 100% of the issues

usage: fastr upgrade [-h] [--type TYPE] NETWORK.py NEW.py

Positional Arguments

NETWORK.py

Network creation file (in python) to upgrade

NEW.py

location of the result file

Named Arguments

--type

tool of resource to upgrade, one of: network, tool

fastr verify

Verify fastr resources, at the moment only tool definitionsare supported.

usage: fastr verify [-h] TYPE path

Positional Arguments

TYPE

Possible choices: tool

Type of resource to verify (e.g. tool)

path

path of the resource to verify

fastr webapp

Starts the fastr web client.

usage: fastr webapp [-h] [-d] [-o]

Named Arguments

-d, --debug

Debug mode.

Default: False

-o, --openpage

Open web page after start.

Default: False