Launch a Prepackaged DRAGEN Pipeline
Prerequisite - Launch a CWL or Nextflow pipeline to completion using the ICA GUI with the intended set of parameters.
Configure CLI and Identify Pipeline ID
Configure and Authenticate ICA command line interface (CLI).
Obtain a list of your projects with their associated IDs:
Copy ID NAME OWNER
a5690b16-a739-4bd7-a62a-dc4dc5c5de6c Project1 670 fd8ea-2ddb-377d-bd8b-587e7781f2b5
ccb0667b-5949-489a-8902-692ef2f31827 Project2 f1aa8430-7058-4f6c-a726-b75ddf6252eb
No of items : 2
Use the ID of the project from the previous step to enter the project context:
Copy icav2 projects enter a5690b16-a739-4bd7-a62a-dc4dc5c5de6c
Find the pipeline you want to start from the CLI by obtaining a list of pipelines associated with your project:
Copy icav2 projectpipelines list
Copy ID CODE DESCRIPTION
fbd6f3c3-cb70-4b35-8f57-372dce2aaf98 DRAGEN Somatic 3.9 .5 The DRAGEN Somatic tool identifies somatic variants
b4dc6b91-5283-41f6-8095-62a5320ed092 DRAGEN Somatic Enrichment 3 -10-4 The DRAGEN Somatic Enrichment pipeline identifies somatic variants which can exist at low allele frequencies in the tumor sample.
No of items : 2
Find the ID associated with your pipeline of interest.
Identify Input File Parameters
To find the input files parameter, you can use a previously launched projectanalyses with the input command.
Find the previous analyses launched along with their associated IDs:
Copy icav2 projectanalyses list
Copy ID REFERENCE CODE STATUS
3539d676-ae99-4e5f-b7e4-0835f207e425 kyle-test-somatic-2-DRAGEN Somatic 3 _9_5 DRAGEN Somatic 3.9 .5 SUCCEEDED
f11e248e-9944-4cde-9061-c41e70172f20 kyle-test-somatic-1-DRAGEN Somatic 3 _9_5 DRAGEN Somatic 3.9 .5 FAILED
No of items : 2
List the analyses inputs by using the ID found in the previous step:
Copy icav2 projectanalyses input 3539 d676-ae99-4e5f-b7e4-0835f207e425
Copy CODE NAMES DATA ID
BED
CNV_B_Allele_VCF
CNV_Population_B_Allele_VCF
HLA_Allele_Frequency_File
HLA_BED
HLA_reference_file_(protein_FASTA )
Microsatellites_File
Normal_BAM_File
Normal_FASTQ_Files
Panel_of_Normals
Panel_of_Normals_TAR
Reference hg38_altaware_nohla-cnv-anchored.v8.tar fil.35e27101fdec404fb37d08d9adf63307
Systematic_Noise_BED
Tumor_BAM_File
Tumor_FASTQ_Files HCC1187C_S1_L001_R1_001.fastq.gz,HCC1187C_S1_L001_R2_001.fastq.gz fil.e1ec77f2647f45804fe508d9aecb19c4,fil.d89018f0c7784fc4b76708d9adf63307
This will return the Input File Codes, as well as the file names and data IDs of the associated data used to previously launch the pipeline
Identify Configuration Settings
Currently, this step for CWL requires the use of the ICA API to access the configuration settings of a project analyses that ran successfully. It is optional for Nextflow since the XML configuration file can be accessed in the ICA GUI.
Nextflow XML file parameters
Click the previous GUI run, and select the pipeline that was run. On the pipeline page, select the XML Configuration Tab to view the configuration settings.
In the "steps" section of the XML file, you will find various steps labeled with
Copy < pd : tool code = "map_align" >
and subsequent labels of parameters with a similar structure
Copy < pd : parameter code = "enable_map_align" minValues = "0" maxValues = "1" classification = "USER" >
< pd : label >Enable Map/Align</ pd : label >
< pd : description ></ pd : description >
< pd : optionsType >
< pd : option >true</ pd : option >
< pd : option >false</ pd : option >
</ pd : optionsType >
< pd : value ></ pd : value >
The code should be used to generate the later command line parameters e.g.
--parameters enable_map_align:true
API based Configuration Settings (CWL or Nextflow)
Generate JWT Token from API Key or Basic login credentials
Instructions on how to get an API Key https://illumina.gitbook.io/ica/account-management/am-iam#api-keys
If your user has access to multiple domains, you will need to need to add a "?tenant=($domain)" to the request
Copy curl -X 'POST' \
'https://ica.illumina.com/ica/rest/api/tokens' \
-H 'accept: application/vnd.illumina.v3+json' \
-H 'X-API-Key: <APIKEY>' \
-d ''
Copy echo -ne 'testemail@testdomain.com:testpassword' | base64
< BASE64UN+PW>
curl -X 'POST' \
'https://ica.illumina.com/ica/rest/api/tokens' \
-H 'accept: application/vnd.illumina.v3+json' \
-H 'Authorization: Basic <BASE64UN+PW>' \
-d ''
Response to this request will provide a JWT token {"token":($token)}, use the value of the token in further requests
Using the API endpoint /api/projects/{projectID}/analyses/{analysisId}/configurations
to find the configuration file listing out all of required and optional parameters
Copy curl -X 'GET' \
'https://ica.illumina.com/ica/rest/api/projects/e501a0d5-f5e7-458c-a590-586c79bb87e0/analyses/3539d676-ae99-4e5f-b7e4-0835f207e425/configurations' \
-H 'accept: application/vnd.illumina.v3+json' \
-H 'Authorization: Bearer <Token>' \
-H 'X-API-Key: <APIKEY>'
The response JSON to this API will have configuration items listed as
Copy {
"items" : [{
"name" : "DRAGEN_Somatic__enable_variant_caller" ,
"multiValue" : false ,
"values" : [
"true"
]
}]
}
Create Launch Command CWL
Structure of the final command
icav2 projectpipelines start cwl $(pipelineID) --user-reference
Plus input options
Input Options - For CLI, the entire input can be broken down as individual command line arguments
To launch the same analysis as using the GUI, use the same file ID and parameters, if using new data you can use the CLI command icav2 projectdata list
to find new file IDs to launch a new instance of the pipeline Required information in Input - Input Data and Parameters
Command Line Arguments
This option requires the use of --type input STRUCTURED
along with --input
and --parameters
Copy icav2 projectpipelines start cwl fbd6f3c3-cb70-4b35-8f57-372dce2aaf98 \
--user-reference kyle-test-somatic-9 \
--storage-size small \
--type-input STRUCTURED \
--input Reference:fil.35e27101fdec404fb37d08d9adf63307 \
--input Tumor_FASTQ_Files:fil.e1ec77f2647f45804fe508d9aecb19c4,fil.d89018f0c7784fc4b76708d9adf63307 \
--parameters DRAGEN_Somatic__enable_variant_caller: true \
--parameters DRAGEN_Somatic__enable_hrd: false \
--parameters DRAGEN_Somatic__enable_sv: true \
--parameters DRAGEN_Somatic__output_file_prefix:tumor \
--parameters DRAGEN_Somatic__enable_map_align: true \
--parameters DRAGEN_Somatic__cnv_use_somatic_vc_baf: false \
--parameters DRAGEN_Somatic__enable_cnv: false \
--parameters DRAGEN_Somatic__output_format:BAM \
--parameters DRAGEN_Somatic__vc_emit_ref_confidence:BP_RESOLUTION \
--parameters DRAGEN_Somatic__enable_hla: false \
--parameters DRAGEN_Somatic__enable_map_align_output: true
Responses:
Successful Response
Copy analysisStorage.description 1.2 TB
analysisStorage.id 6 e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0
analysisStorage.name Small
analysisStorage.ownerId 8 ec463f6-1acb-341b-b321-043c39d8716a
analysisStorage.tenantId f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3
analysisStorage.tenantName ica-cp-admin
analysisStorage.timeCreated 2021 -11-05T10:28:20Z
analysisStorage.timeModified 2021 -11-05T10:28:20Z
id 51 abe34a-2506-4ab5-adef-22df621d95d5
ownerId 47793 c21-75a6-3aa8-8147-81b354d0af4d
pipeline.analysisStorage.description 1.2 TB
pipeline.analysisStorage.id 6 e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0
pipeline.analysisStorage.name Small
pipeline.analysisStorage.ownerId 8 ec463f6-1acb-341b-b321-043c39d8716a
pipeline.analysisStorage.tenantId f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3
pipeline.analysisStorage.tenantName ica-cp-admin
pipeline.analysisStorage.timeCreated 2021 -11-05T10:28:20Z
pipeline.analysisStorage.timeModified 2021 -11-05T10:28:20Z
pipeline.code DRAGEN Somatic 3.9 .5
pipeline.description The DRAGEN Somatic tool identifies somatic variants which can exist at low allele frequencies in the tumor sample. The pipeline can analyze tumor/normal pairs and tumor-only sequencing data. The normal sample, if present, is used to avoid calls at sites with germline variants or systematic sequencing artifacts. Unlike germline analysis, the somatic platform makes no ploidy assumptions about the tumor sample, allowing sensitive detection of low-frequency alleles.
pipeline.id fbd6f3c3-cb70-4b35-8f57-372dce2aaf98
pipeline.language CWL
pipeline.ownerId e9dd2ff5-c9ba-3293-857e-6546c5503d76
pipeline.tenantId 55 cb0a54-efab-4584-85da-dc6a0197d4c4
pipeline.tenantName ilmn-dragen
pipeline.timeCreated 2021 -11-23T22:55:49Z
pipeline.timeModified 2021 -12-09T16:42:14Z
reference kyle-test-somatic-9-DRAGEN Somatic 3 _9_5-bc56d4b1-f90e-4039-b3a4-b11d29263e4e
status REQUESTED
summary
tenantId b5b750a6-49d4-49de-9f18-75f4f6a81112
tenantName ilmn-cci
timeCreated 2022 -03-16T22:48:31Z
timeModified 2022 -03-16T22:48:31Z
userReference kyle-test-somatic-9
Unsuccessful Response Pipeline ID not formatted correctly
Copy 400 Bad Request : ICA_API_004 : com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type ` java.util.UUID ` from String "8f57-372dce2aaf98" : UUID has to be represented by standard 36 -char representation
at [Source: (io.undertow.servlet.spec.ServletInputStreamImpl); line: 1 , column: 983 ] (through reference chain: com.bluebee.rest.v3.publicapi.dto.analysis.SearchMatchingActivationCodesForCwlAnalysisDto[ "pipelineId" ] ) ( ref. c9cd9090-4ddb-482a-91b5-8471bff0be58 )
Check that the pipeline ID is correct based on icav2 projectpipelines list
File ID not found
Copy 404 Not Found : ICA_GNRC_001 : Could not find data with ID [fil.35dec404fb37d08d9adf63307] ( ref. 91 b70c3c-378c-4de2-acc9-794bf18258ec )
Check that the file ID is correct based on icav2 projectdata list
Parameter not found
Copy 400 Bad Request : ICA_EXEC_007 : The specified variableName [DRAGEN] does not exist. Make sure to use an existing variableName ( ref. ab296d4e-9060-412c-a4c9-562c63450022 )
Create Launch Command Nextflow
When using nextflow to start runs, the input-type parameter is not used, but the --project-id
is required
Structure of the file command icav2 projectpipelines start nextflow $(pipelineID) --user-reference
Plus input options
Copy icav2 projectpipelines start nextflow b4dc6b91-5283-41f6-8095-62a5320ed092 \
--user-reference "somatic-3-10-test5" \
--project-id e501a0d5-f5e7-458c-a590-586c79bb87e0 \
--storage-size Small \
--input ref_tar:fil.35e27101fdec404fb37d08d9adf63307 \
--input tumor_fastqs:fil.e1ec77f2647f45804fe508d9aecb19c4,fil.d89018f0c7784fc4b76708d9adf63307 \
--parameters enable_map_align: true \
--parameters enable_map_align_output: true \
--parameters output_format:BAM \
--parameters enable_variant_caller: true \
--parameters vc_emit_ref_confidence:BP_RESOLUTION \
--parameters enable_cnv: false \
--parameters enable_sv: true \
--parameters repeat_genotype_enable: true \
--parameters enable_hla: false \
--parameters enable_variant_annotation: false \
--parameters output_file_prefix:Tumor
Response status can be used to determine if the pipeline was submitted successfully
status options: REQUESTED,SUCCEEDED,FAILED,ABORTED
Copy analysisStorage.description 1.2 TB
analysisStorage.id 6 e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0
analysisStorage.name Small
analysisStorage.ownerId 8 ec463f6-1acb-341b-b321-043c39d8716a
analysisStorage.tenantId f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3
analysisStorage.tenantName ica-cp-admin
analysisStorage.timeCreated 2021 -11-05T10:28:20Z
analysisStorage.timeModified 2021 -11-05T10:28:20Z
id 9 b8f9e84-2e7f-4adb-92e5-738b032c2328
ownerId 47793 c21-75a6-3aa8-8147-81b354d0af4d
pipeline.analysisStorage.description 1.2 TB
pipeline.analysisStorage.id 6 e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0
pipeline.analysisStorage.name Small
pipeline.analysisStorage.ownerId 8 ec463f6-1acb-341b-b321-043c39d8716a
pipeline.analysisStorage.tenantId f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3
pipeline.analysisStorage.tenantName ica-cp-admin
pipeline.analysisStorage.timeCreated 2021 -11-05T10:28:20Z
pipeline.analysisStorage.timeModified 2021 -11-05T10:28:20Z
pipeline.code DRAGEN Somatic Enrichment 3 -10-4
pipeline.description The DRAGEN Somatic Enrichment pipeline identifies somatic variants which can exist at low allele frequencies in the tumor sample.
pipeline.id b4dc6b91-5283-41f6-8095-62a5320ed092
pipeline.language NEXTFLOW
pipeline.ownerId e9dd2ff5-c9ba-3293-857e-6546c5503d76
pipeline.tenantId 55 cb0a54-efab-4584-85da-dc6a0197d4c4
pipeline.tenantName ilmn-dragen
pipeline.timeCreated 2022 -03-07T18:09:38Z
pipeline.timeModified 2022 -03-08T17:47:20Z
reference somatic-3-10-test5-DRAGEN Somatic Enrichment 3 -10-4-3df131a2-2187-489a-b9f8-140e3ec5efb0
status REQUESTED
tenantId b5b750a6-49d4-49de-9f18-75f4f6a81112
tenantName ilmn-cci
timeCreated 2022 -07-13T22:44:47Z
timeModified 2022 -07-13T22:44:47Z
userReference somatic-3-10-test5
Last updated 5 months ago