Tool Repository

A Tool is the definition of a containerized application with defined inputs, outputs, and execution environment details including compute resources required, environment variables, command line arguments, and more.

Create a Tool

Tools define the inputs, parameters, and outputs for the analysis. Tools are available for use in graphical CWL pipelines by any project in the account.

  1. Select System Settings > Tool Repository > + New tool.

  2. Configure tool settings in the tool properties tabs. See Tool Properties.

  3. Select Save.

Tool Properties

The following sections describe the tool properties that can be configured in each tab.

Refer to the CWL CommandLineTool Specification for further explanation about many of the properties described below. Not all features described in the specification are supported.

Information Tab

FieldEntry

Name

The name of the tool.

Categories

One or more tags to categorize the tool. Select from existing tags or type a new tag name in the field.

Icon

The icon for the tool.

Description

Free text description for information purposes.

Status

The release status of the tool.

Docker image

The registered Docker image for the tool.

Regions

The regions supported by linked Docker image.

Tool version

The version of the tool specified by the end user. Could be any string.

Release version

The version number of the tool.

Family

A group of tools or tool versions.

Version comment

A description of changes in the updated version.

Links

External reference links.

Tool Status

The release status of the tool. can be one of "Draft", "Release Candidate", "Released" or "Deprecated".

StatusDescription

Draft

Fully editable draft.

Release Candidate

The tool is ready for release. Editing is locked but the tool can be cloned to create a new version.

Released

The tool is released. Tools in this state cannot be edited. Editing is locked but the tool can be cloned to create a new version.

Deprecated

The tool is no longer intended for use in pipelines. but there are no restrictions placed on the tool. That is, it can still be added to new pipelines and will continue to work in existing pipelines. It is merely an indication to the user that the tool should no longer be used.

Documentation Tab

The Documentation tab provides options for configuring the HTML description for the tool. The description appears in the Tool Repository but is excluded from exported CWL definitions.

General Tool Tab

The General Tool tab provides options to configure the basic command line.

FieldEntry

ID

CWL identifier field

CWL version

The CWL version in use. This field cannot be changed.

Interpreter version

The interpreter version in use.

Base command

Components of the command. Each argument must be added in a separate line.

Standard in

The name of the file that captures Standard In (STDIN) stream information.

Standard out

The name of the file that captures Standard Out (STDOUT) stream information.

Standard error

The name of the file that captures Standard Error (STDERR) stream information.

Requirements

The requirements for triggering an error message.

Hints

The requirements for triggering a warning message.

The Hints/Requirements include CWL features to indicate capabilities expected in the Tool's execution environment.

  • Inline Javascript

    • The Tool contains a property with a JavaScript expression to resolve it's value.

  • Initial workdir

    • The workdir can be any of the following types:

      • String or Expression — A string or JavaScript expression, eg, $(inputs.InputFASTA)

      • File or Dir — A map of one or more files or directories, in the following format: {type: array, items: [File, Directory]}

      • Dirent — A script in the working directory. The Entry name field specifies the file name.

  • Scatter feature — Indicates that the workflow platform must support the scatter and scatterMethod fields.

Tool Arguments Tab

The Tool Arguments tab provides options to configure base command parameters that do not require user input.

Tool arguments may be one of two types:

  • String or Expression — A literal string or JavaScript expression, eg --format=bam.

  • Binding — An argument constructed from the binding of an input parameter.

The following table describes the argument input fields.

FieldEntryType

Value

The literal string to be added to the base command.

String or expression

Position

The position of the argument in the final command line. If the position is not specified, the default value is set to 0 and the arguments appear in the order they were added.

Binding

Prefix

The string prefix.

Binding

Item separator

The separator that is used between array values.

Binding

Value from

The source string or JavaScript expression.

Binding

Separate

The setting to require the Prefix and Value from fields to be added as separate or combined arguments. Tru indicates the fields must be added as separate arguments. False indicates the fields must be added as a single concatenated argument.

Binding

Shell quote

The setting to quote the Value from field on the command line. True indicates the value field appears in the command line. False indicates the value field is entered manually.

Binding

Example

FieldValue

Prefix

--output-filename

Value from

$(inputs.inputSAM.nameroot).bam

Input file

/tmp/storage/SRR45678_sorted.sam

Output file

SRR45678_sorted.bam

Tool Input Tab

The Tool Inputs tab provides options to define the input files and directories for the tool. The following table describes the input and binding fields. Selecting multi value enables type binding options for adding prefixes to the input.

FieldEntry

ID

The file ID.

Label

A short description of the input.

Description

A long description of the input.

Type

The input type, which can be either a file or a directory.

Input options

Checkboxes to add the following options. Optional indicates the input is optional. Multi value indicates there is more than one input file or directory. Streamable indicates the file is read or written sequentially without seeking.

Secondary files

The required secondary files or directories.

Format

The input file format.

Position

The position of the argument in the final command line. If the position is not specified, the default value is set to 0 and the arguments appear in the order they were added.

Prefix

The string prefix.

Item separator

The separator that is used between array values.

Value from

The source string or JavaScript expression.

Load contents

The setting to require the Prefix and Value from fields to be added as separate or combined arguments. True indicates the fields must be added as separate arguments. False indicates the fields must be added as a single concatenated argument.

Separate

The setting to require the Prefix and Value from fields to be added as separate or combined arguments. True indicates the fields must be added as separate arguments. False indicates the fields must be added as a single concatenated argument.

Shell quote

The setting to quote the Value from field on the command line. True indicates the value field appears in the command line. False indicates the value field is entered manually.

Tool Settings Tab

The Tool Settings tab provides options to define parameters that can be set at the time of execution. The following table describes the input and binding fields. Selecting multi value enables type binding options for adding prefixes to the input.

FieldEntry

ID

The file ID.

Label

A short description of the input.

Description

A long description of the input.

Default Value

The default value to use if the tool setting is not available.

Type

The input type, which can be Boolean, Int, Long, Float, Double or String.

Input options

Checkboxes to add the following options. Optional indicates the input is optional. Multi value indicates there can be more than one value for the input.

Position

The position of the argument in the final command line. If the position is not specified, the default value is set to 0 and the arguments appear in the order they were added.

Prefix

The string prefix.

Item separator

The separator that is used between array values.

Value from

The source string or JavaScript expression.

Separate

The setting to require the Prefix and Value from fields to be added as separate or combined arguments. True indicates the fields must be added as separate arguments. False indicates the fields must be added as a single concatenated argument.

Shell quote

The setting to quote the Value from field on the command line. True indicates the value field appears in the command line. False indicates the value field is entered manually.

Tool Outputs Tab

The Tool Outputs tab provides options to define the parameters of output files.

The following table describes the input and binding fields. Selecting multi value enables type binding options for adding prefixes to the input.

FieldEntry

ID

The file ID.

Label

A short description of the input.

Description

A long description of the input.

Default Value

A long description of the input.

Type

The input type, which can be either a file or a directory.

Output options

Checkboxes to add the following options. Optional indicates the input is optional. Multi value indicates here is more than one input file or directory. Streamable indicates the file is read or written sequentially without seeking.

Secondary files

The required secondary files or directories.

Format

The input file format.

Globs

The pattern for searching file names.

Load contents

Automatically loads some contents. The system extracts up to the first 64 KiB of text from the file. Populates the contents field with the first 64 KiB of text from the file.

Output eval

Evaluate an expression to generate the output value.

Tool CWL Tab

The Tool CWL tab displays the complete CWL code constructed from the values entered in the other tabs. the CWL code automatically updates when changes are made in the tool definition tabs, and any changes to the CWL code are reflected in the tool definition tabs.

❗️ Modifying data within the CWL editor can result in invalid code.

Edit a Tool

  1. From the System Settings > Tool Repository page, select a tool.

  2. Select Edit.

Update Tool Status

  1. From the System Settings > Tool Repository page, select a tool.

  2. Select the Information tab.

  3. From the Status drop-down menu, select a status.

  4. Select Save.

Import Tool

In addition to the interactive Tool builder, the platform GUI also supports working directly with the raw definition when developing a new Tool. This provides the ability to write the Tool definition manually or bring an existing Tool's definition to the platform.

A simple example CWL Tool definition is provided below.

#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
label: echo
inputs:
  message:
    type: string
    default: testMessage
    inputBinding:
      position: 1
outputs:
  echoout:
    type: stdout
baseCommand:
- echo

When creating a new Tool, navigate to System Settings > Tool Repository > your_tool > Tool CWL tab to show the raw CWL definition. Here a CWL CommandLineTool definition may be pasted into the editor. After pasting into the editor, the definition is parsed and the other tabs for visually editing the Tool will populate according to the definition contents.

Creating Your First Tool - Tips and Tricks

  • General Tool - includes your base command and various optional configurations.

    • The base command is required for your tool to run, e.g. python /path/to/script.py such that python and /path/to/script.py are added in separate lines.

    • Inline Javascript requirement - must be enabled if you are using Javascript anywhere in your tool definition.

    • Initial workdir requirement - Dirent Type

      • Your tool must point to a script that executes your analysis. That script can either be provided in your Docker image or using a Dirent. Defining a script via Dirent allows you to dynamically modify your script without updating your Docker image. In order to define your Dirent script define your script name under Entry name (e.g. runner.sh) and the script content under Entry. Then, point your base command to that custom script, e.g. bash runner.sh.

❗ What's the difference between Settings and Arguments?

Settings are exposed at the pipeline level with the ability to get modified at launch, while Arguments are intended to be immutable and hidden from users launching the pipeline.

  • How to reference your tool inputs and settings throughout the tool definition?

    • You can either reference your inputs using their position or ID.

      • Settings can be referenced using their defined IDs, e.g. $(inputs.InputSetting)

      • File/Directory inputs can be referenced using their defined IDs, followed by the desired field, e.g. $(inputs.InputFile.path). For additional information please refer to the File CWL documentation.

      • All inputs can also be referenced using their position, e.g. bash script.sh $1 $2

Last updated