Instrument Run Upload
Sample data produced by Illumina sequencing instruments can be made available in ICA. A V2 Sample Sheet is required to be used for the instrument run. The following instruments are currently supported:
- NextSeq 1000/2000
- NovaSeq 6000
There are several software components involved in the end to end data flow for the instrument run upload:
Software | Purpose |
|---|---|
BSSH | Plan and monitor the instrument run, store raw run output files (BCLs) |
Instrument | Login and launch the planned instrument run |
BCLConvert App | Demultiplex the raw instrument run outputs and store the resulting FASTQ files in ICA |
ICA | Store the demultiplexed run data |
BaseSpace Sequence Hub (BSSH) is used to plan and monitor the instrument run upload. Raw BCL run data is streamed to BSSH from the instrument. The FASTQ generation process launched after the instrument run upload completes places the demultiplexed FASTQs into an ICA project where the data is accessible through an externally-mananged project. The FASTQ data can then be linked to other projects to be used as input for analyses.
The BSSH Instrument Run Setup provides an interface for planning an instrument run based on a V2 Sample Sheet.
Navigate to BSSH and change to your desired workgroup or personal context. This may require you login to your domain first.
instrument-run-upload-9
After selecting the appropriate user/workgroup context, navigate to the Runs tab in BSSH. Then click the New Run button and select Instrument Run Setup.
instrument-run-upload-1
Complete the instrument run setup form to create a planned run with the following configuration.
- Set the Analysis Location to "BaseSpace"
- Set the "Type of Analysis" to the "Illumina BCL Convert for ICA" app (recommended to use latest version)
The Illumina BCL Convert for ICA analysis app currently requires an Illumina operator to make the app available for a workgroup.
instrument-run-upload-8
An alternative to using the BSSH Instrument Run Setup interface is to provide a samplesheet directly on the Instrument. Use the following settings in the SampleSheet when configuring manually:
- v2 SampleSheet is required
- In the
[Cloud_Settings]section, the value ofBsshApp(appslug) must be set toillumina-inc.bcl-convert-for-ica.1.0.0
On the instrument, sign in using the desired user/workgroup context then:
- NextSeq 1000/2000 - Locate the run from the list of planned runs in NextSeq 1000/2000 Control Software. Launch the run.
- NovaSeq 6000 - Download the samplesheet using the "Export" button, manually upload the samplesheet to the NovaSeq 6000 during run setup on the instrument. Launch the run.
After an instrument run is started, BSSH is used for monitoring the run through completion. Use the BSSH Runs view to track the run progress.
instrument-run-upload-6
In order to create resources including sample data in ICA, an externally-managed project is created by BSSH. Operations in BSSH, including planning and monitoring instrument runs, are performed in a given user or workgroup context. To preserve permission boundaries between applications, an ICA project is created for each user/workgroup context per region. The ICA project will be named using the convention
BSSH {context name}, where {context-name} is the user or workgroup name operating in BSSH. For workgroups, the project will be created on behalf of the workgroup owner.In the example below, the project is created for a workgroup with name "ICA_BSSH_Demo_WG" where the workgroup owner is "[email protected]".
instrument-run-upload-0
The project Team includes an entry for the workgroup to grant all workgroup users read-only access to the project. While Team members may be assigned the Contributor role in the externally-managed project, they will not have write access (ie, create/delete data, run pipelines) in the project.
instrument-run-upload-4
The project contains the following tags:
- Technical Tags:
bssh.project.user:{user/workgroup ID}
instrument-run-upload-3
The demultiplexed FASTQ data is uploaded to the BSSH externally-managed project. Sample and data entities in ICA are tagged with key BSSH identifiers for traceability to the associated upstream BSSH resources.
The image below shows details of a sample in the externally-managed project, including tags.
instrument-run-upload-5
The image below shows details of FASTQ data in the externally-managed project, selected from the sample details view.
instrument-run-upload-7
See the table below for the tags added to Samples and Data.
BSSH Property | Tag | Sample | Data |
|---|---|---|---|
Run ID | bssh.run.id | | x |
Run Name | bssh.run.name | | x |
Run URL | bssh.run.url | | x |
Run API URL | bssh.run.apiurl | | x |
Biosample ID | bssh.biosample.id | x | |
Biosample Name | bssh.biosample.name | x | |
Biosample URL | bssh.biosample.url | x | |
Biosample API URL | bssh.biosample.apiurl | x | |
BSSH Project ID | bssh.project.id | x | x |
BSSH Project Name | bssh.project.name | x | x |
BSSH Project URL | bssh.project.url | x | x |
BSSH Project API URL | bssh.project.apiurl | x | x |
Dataset ID | bssh.dataset.id | | x |
Dataset ID Token | bssh.dataset.idtoken | | x |
Dataset Name | bssh.dataset.name | | x |
Dataset API URL | bssh.dataset.apiurl | | x |
The results from the FASTQ Generation (bclConvert), including FASTQs and logs files, are stored in the ICA project data. The FASTQ files are linked to the associated Sample resource. The data organization is important to preserve BSSH functionality as BSSH expects specific file/folder paths. The data is read-only for users accessing the data directly through ICA to prevent corrupting BSSH.
Walking through the folder structure, at the top level there is the
datasets/ folder. This serves as a parent container. Under this folder are folders named with a BSSH App Session, representing each execution of the FASTQ Generation app. Inside the app session folder are log files and outputs of the app. The output is stored in the output/ folder. Inside this folder is a fastqs/ folder, containing the outputs from the FASTQ Generation task. The Samples/ folder contains nested folders for the Lanes and Sample IDs which contain the FASTQ files. See the below path for a complete example path of a FASTQ file./datasets/app.27849123/output/fastqs/Samples/Lane_1/ICA_SC_NexteraFlex-NA12878-UDP0002/ICA_SC_NexteraFlex-NA12878-UDP0002_S2_L001_R2_001.fastq.gzWith sample data available in the BSSH externally-managed project, the sample data can be linked to other ICA projects to serve as inputs for running pipelines, etc. Users with access to the BSSH externally-managed project can create links from the destination project Samples and Data views. See the Link Project Data and Link Project Sample documentation for details on linking sample data.
Project Connectors provide a mechanism for automatically linking the sample data to a project. Use the BSSH externally-managed project as the "Source" project when configuring the connector.
A "requeue" is a re-execution of the FASTQ generation (bclConvert) often performed due to an error in the samplesheet provided for the initial execution.
A requeue is performed by manually relaunching the FASTQ Generation app with a corrected samplesheet. Find the original samplesheet by navigating to the Runs tab, click the desired Run to be requeued, then click the Files tab. In the list of files, find the SampleSheet.csv file. Click to open the file, and click the Download button to download locally. After downloading make any desired modifications to the samplesheet to prepare for the requeue.
Once the modified samplesheet is prepared, you'll need to upload the file to an existing BSSH project. Navigate to the "Projects" tab and choose a project to upload the modified samplesheet. Navigate to the "Other Datasets" tab, click the File icon, select Upload > Files.
instrument-run-upload-12
Select "Other" from the list of file types, and use the browser upload UI to select the modified samplesheet.
Find the Illumina BCL Convert for ICA app in the BSSH Apps. Navigate to the Apps tab.
instrument-run-upload-10
In the list of BSSH Apps, select the "BCL Convert for ICA (Beta)" app.
instrument-run-upload-11
Click to Launch the application. Complete the launch form by selecting the instrument run to be requeued and choosing the modified samplesheet in the Override Sample Sheet form field. When ready, click the "Launch Application" button to start the app.
Last modified 2mo ago