CWL

ICA supports running pipelines defined using Common Workflow Language (CWL).

Compute Type

To specify a compute type for a CWL CommandLineTool, use the ResourceRequirement with a custom namespace.

requirements:
    ResourceRequirement:
        https://platform.illumina.com/rdf/ica/resources:type: fpga
        https://platform.illumina.com/rdf/ica/resources:size: small 
        https://platform.illumina.com/rdf/ica/resources:tier: standard

Reference Compute Types for available compute types and sizes.

The ICA Compute Type will be determined automatically based on CWL ResourceRequirement coresMin/coresMax (CPU) and ramMin/ramMax (Memory) values using a "best fit" strategy to meet the minimum specified requirements (refer to the Compute Types table).

For example, take the following ResourceRequirements:

requirements:
    ResourceRequirement:
      ramMin: 10240
      coresMin: 6

This would result in a best fit of standard-large ICA Compute Type request for the tool.

• If the specified requirements can not be met by any of the presets, the task will be rejected and failed.

• FPGA requirements can not be set by means of CWL ResourceRequirements.

• The Machine Profile Resource in the graphical editor will override whatever is set for requirements in the ResourceRequirement.

Considerations

If no Docker image is specified, Ubuntu will be used as default. Both : and / can be used as separator.

CWL Overrides

ICA supports overriding workflow requirements at load time using Command Line Interface (CLI) with JSON input. Please refer to CWL documentation for more details on the CWL overrides feature.

In ICA you can provide the "override" recipes as a part of the input JSON. The following example uses CWL overrides to change the environment variable requirement at load time.

icav2 projectpipelines start cwl cli-tutorial --data-id fil.a725a68301ee4e6ad28908da12510c25 --input-json '{
  "ipFQ": {
    "class": "File",
    "path": "test.fastq"
  },
  "cwltool:overrides": {
  "tool-fqTOfa.cwl": {
    "requirements": {
      "EnvVarRequirement": {
        "envDef": {
          "MESSAGE": "override_value"
          }
        }                                       
       }
      }
    }
}' --type-input JSON --user-reference overrides-example