Release Notes
- Fixed an issue causing incorrect empty storage configuration dropdown during Project creation when using the “I want to manage my own storage” option for users with access to a single region
- General availability of sequencer integration for Illumina sequencing systems and analysis auto launch
- General usability improvements in the graphical interface, including improved navigation structure and ability to switch between applications via the waffle menu in the header
- Storage Bundle field will be auto-filled based on the Project location that is being chosen if multiple regions are available
- Event Log entries will be paged in the UI and will contain a maximum of 1,000 entries. Exports are limited to the maximum number of entries displayed on the page.
- Read-only temporary credentials will be returned when you are not allowed to modify the contents of a file
- Creating Project notifications for BaseSpace externally managed projects is now supported
- The ICA UI will only allow selection of storage bundles belonging to ICA during Project creation, and the API will only return storage bundles for ICA
- Flow: Allow attached storage for Pipeline steps to be set to 0 to disable provisioning attached storage and improve performance
- Cohorts: GRCh37/hg19-aligned molecular data will get converted to GRCh38/hg38 coordinates to facilitate cross-project analyses and incorporating publicly available data sets.
- API: Project list API now contains a parameter to filter on (a) specific workgroup(s)
- API: Two new API endpoints are added to retrieve regular parameters from a pipeline within or without a Project context
- Optimized price calculations resulting in less overhead and logging
- Improved error handling:
- during Project creation
- of own storage Project creation failures.
- to indicate connection issue with credentials
- for graphical CWL draft Pipelines being updated during an Analysis.
- Fixed an issue causing data records to remain incorrectly in Unarchiving status when an unarchive operation is requested in the US and Germany regions
- Improved error messaging in cases where the AWS path contains (a) special character(s)
- Fixed an issue causing errors when navigating via deep link to the Analysis Details view
- CLI: Unarchival of folders is supported when archive or unarchive actions are not in progress for the folder
- CLI: Improved error message to indicate connection issue with credentials
- Flow: Fixed an issue causing incorrect naming of Analysis tasks generated from CWL ExpressionTools
- Flow: Fixed an issue when cloning Pipelines linked from Entitled Bundles to preserve the original Tenant as the Owning Tenant of the cloned Pipeline instead of the cloning user’s Tenant
- Flow: Fixed an issue causing outputs from CWL Pipelines to not show in the Analysis Details despite being uploaded to the Project Data Analysis output folder when an output folder is empty
- Flow: When a Contributor starts an Analysis, but is removed afterwards, the Analysis still runs as expected
- Flow: Fixed an issue where Analyses fail where Nextflow is run a second time
- Flow: Fixed an issue causing API error when attempting to launch an Analysis with up to 50,000 input files
- Flow: Fixed an issue causing degraded performance in APIs to retrieve Analysis steps in Pipelines with many steps
- Flow: Fixed an issue causing Analysis failure during output upload with error “use of closed network connection”
- Flow: Fixed an issue causing disk capacity alter log to not show when an Analysis fails due to disk capacity and added error message
- Flow: Fixed an issue preventing cross-tenant users from being able to open a shared CWL pipeline
- Base: Improved target Table selection for schedulers to be limited to your own Tables
- Bench: Fixed an issue causing Workspaces to hang in the Starting or Stopping statuses
- Cohorts: Now handles large VCFs/gVCFs correctly by splitting them into smaller files for subsequent annotation by Nirvana
- API: Fixed returning list of unlinked data in a sample that was linked before in GET/api/projects/{projectId}/data
- API: Fixed error for getSampleCreationBatch when using status filter
- Updated guidance for selecting Project Location and availability of Storage Bundles when configuring a new Project
- Included information about receiving and using Entitled Bundles
- Added guidance that a sample must be complete to be linked to a Bundle
- Updated details about paging in the Event Log
- Refined and expanded guidance for sequencer integration and analysis auto-launch, including adding a new tutorial for NovaSeq 6000 and expanding the sequencer reference table
- Added tutorial for using scatter/gather for Nextflow pipelines
- Refined Pipelines documentation around clone function
- Added information that Base Tables must not contain reserved Snowflake terms in their definition
- Added description for Copy Number Variants and Structural Variants tables now viewable in ICA Base for Cohorts
- Added description for hg19 support as an alternative during ingestion for Cohorts
- Updated CLI command index
- Updated CLI binary and checksum files
- Updated information about data archive pricing
- Updated software registration guide
- General updates and improvements
- Added a limit to Event Log and Audit UI screens to show 10,000 records
- API: Parent output folder can be specified in URN format when launching a Workflow session via the API
- Flow: Reduced Analysis delays when system is experiencing heavy load
- Flow: Improved formatting of Pipeline error text shown in Analysis Details view
- Flow: Users can now start Analyses from the Analysis Overview screen
- Flow: Superfluous “Namespace check-0” step was removed to reduce Analysis failures
- Flow: Number of input files for an Analysis is limited to 50,000
- Flow: Auto launched Workflow sessions will fail if duplicate sample IDs are detected under Analysis Settings in the Sample Sheet
- Base: Activity screen now contains the size of the query
Cohorts
- Detect and Lift Genome Build: Cohorts documentation provides set-up instructions to convert hg19/GRCh37 VCFs to GRCh38 before import into Cohorts.
- Attribute Queries: Improved the user experience choosing a range of values for numerical attributes when defining a cohort
- Export Cohort to ICA Project Data: Improved the user experience exporting list of subjects that match cohort definition criteria to their ICA project for further analysis
- Ingest Structural Variants into database
- The Cohorts ingestion pipeline supports structural variant VCFs and will deposit all such variants into an ICA Base table if Base is enabled for the given project
- Structural variants can be ingested and viewed in base tables
- Needle Plot Enhancements
- Users can input a numerical value in the Needle Plot legend to display variants with a specific gnomAD frequency percentage or sample count
- The needle plot combines variants that are observed among subjects in the current project as well as shared and public projects into a single needle, using an additional shape to indicate these occurrences
- Needle Plot legend color changes for Variant severity; pathogenic color coding is the same as the color coding in the visualization; differentiating hue between proteins and variants; and other color coding changes.
- Needle plot tool tips that display additional information on variants and mutations are now larger and modal
- The needle plot now allows to filter by gnomAD allele frequency and sample count in the selected cohort. Variants include links to view a list of all subjects carrying that variant and export that list.
- Remove Samples Individually from Cohorts
- Exclude individual subjects from a cohort and save the refined list
- The subjects view allows users to exclude individual subjects from subsequent analyses and plots and save these changes
- Subject exclusions are reset when editing a cohort
- Subject Selection in Analysis Visualization: Users can follow the link for subject counts in the needle plot to view a list of subjects carrying the selected variant or mutation.
- UI/UX: Start and End time points are available as a date or age with a condition attribute in the subject data summary screen.
- Improved resilience against misconfiguration of the team page when there is an issue with Workgroup availability
- Improved handling of multi-file upload when system is experiencing heavy loads
- Fixed an issue to allow upload of zero-byte files via the UI
- Fixed issue where other Bundles would not be visible after editing and saving your Bundle
- Removed ‘IGV (beta)’ button from ‘View’ drop down when selecting Project Data in UI
- API: Improved error handling for API endpoint: POST /api/projects/{projectId}/analysisCreationBatch
- API: Improved performance of API endpoint: getbestmatchingfornextflow
- Flow: Fixed an issue causing Analysis output mapping to incorrectly use source path as target path
- Flow: Fixed an issue where the UI may display incorrect or invalid parameters for DRAGEN workflows which do not accurately show the true parameters passed. Settings can be confirmed by looking at the DRAGEN analysis log files.
- Base: “Allow jagged rows” setting in the Scheduler has been replaced with “Ignore unknown values” to handle files containing records with more fields than there are Table columns
- Base: Improved Base Activity view loading time
- Base: Fixed an error message when using the API to load data into a Base Table that has been deleted
- Bench: Fixed an issue resulting in incorrect Bench compute pricing calculations
- Bench: Fixed an issue preventing building Docker images from Workspaces in UK, Australia, and India regions
- Bench: Fixed an issue where /tmp path is not writeable in a Workspace
Cohorts:
- Fixed issue where the bubble plot sometimes failed to display results even though the corresponding scatter plot showed data correctly.
- The order of messages and warnings for ingestion jobs was not consistent between the UI and an error report sent out via e-mail.
- The UI now displays any open cohort view tabs using shortened (“…”) names where appropriate
- Issue fixed where ingestions with multiple errors caused halting to the ingestion queue.
- The needle plot sometimes showed only one source for a given variant as opposed to all projects in which the variant had been observed in.
- Issue fixed with unhandled genotype index format in annotation file to base database table conversion
- Status updates via e-mail sometimes contained individual error messages or warnings without a text.
- Fixed issue where items show in needle plot with incorrect numbering on the y-axis.
- Fixed performance issue with subject count.
- Widget bar-chart counts are intermittently cut off over four digits.
- Fixed slowness when switching between tabs in query builder
- Refined and expanded reference documents for sequencer integration and analysis autolaunch, including new pages for sequencer reference and troubleshooting.
- Added tutorial for accessing Base databases and tables via snowSQL.
- Added tutorial for setting up a notification service and Lambda function in AWS for chaining pipelines.
- Included guidance for setting override behavior for file output of advanced output mappings.
- Added instructions to remove specific subjects from a cohort.
- Documented cohort needle plot enhancements: List of subjects and specific frequency data entry.
- Updated cohorts ingestion instruction including metadata documents, GWAS ingestion, OMOP ingestion, Structural Variant ingestion, and hg19 lift-over information.
- Added reference documentation for using Cohorts attributes metadata template.
- Added instruction for cohorts attribute query enhancement on numeric data.
- Described selection of entitlements when creating a Project.
- Described options for starting analyses from Analyses page, from Data page, and from a pipeline.
- Updated FAQ with guidance on analysis run times.
- Updated documentation about network settings for pipelines.
- Described limitation on amount of input files for analyses.
- Explained how to unlink a folder from a project.
- Updated explanation for adding Terms of Use to a Bundle.
- Included details for the Event Log.
- Refined and expanded description of Base Activity screen.
- Expanded description of Bench Activity screen.
- Clarified details about S3 bucket behavior and Bring Your Own Storage bucket (BYOB) compatibility.
- Provided guidance about deleting old versions of Bring Your Own Storage buckets.
- Explained multi-factor authentication compatibility with Bench and provide workaround suggestion.
- Updated CLI command index.
- Fixed a bug in the CLI release history files and updated integrity check instructions.
- General updates and improvements.
- Fixed issue with BaseSpace Free Trial and Professional users storing data in ICA
- Fixed an issue resulting in analysis failures caused by a Kubernetes 404 timeout error
- API: POST /api/projects/{projectId}/data:createDownloadUrls now supports a list of paths (in addition to a list of IDs)
- Each tenant supports a maximum of 30,000 Projects
- .MAF files are now recognized as .TSV files instead of UNKNOWN
- Added VCF.IDX as a recognized file format
- General scalability optimizations and performance improvements
- After creating a new Bundle the user will be taken to the Bundle Overview page
- Fixed an issue preventing the ‘Owning Project’ column from being used outside of Project
- Fixed an issue which prevented changing the format of a file back to UNKNOWN
- Fixed an issue causing inaccurate upload progress to be displayed for UI uploads. The Service Connector or CLI are recommended for large file uploads.
- Fixed an issue showing an incorrect status for data linking batch jobs when data is deleted during the linking job
- Fixed an issue allowing the region of a Project to be changed. Changing the region of a resource is not supported.
- Strengthened data separation and improved resilience against cross-Project metadata contamination
- Service Connector: Fixed an issue allowing download of a Service Connector when no operating system is set
- Service Connector: Cleaned up information available on Service Connectors by removing empty address information fields
- API: Fixed date formatting for GET /api/eventLog (yyyy-MM-dd’T’HH:mm:ss.SSS’Z’)
- API: Fixed an issue where the GET users API was not case sensitive on email address
- API: Fixed an issue causing the metadata model to be returned twice in PSOT /api/projects/{projectId}/samples:search
- API: Fixed the listProjects API 500 response when using the pageoffset query parameter
- API: The searchProjectSamples API returns Sample metadata for Samples shared via a Bundle
- API: Fixed an issue causing createProjectDataDownloadUrls API 400 and 502 errors when server is under load
- Flow: Fixed analysis failures caused by kubernetes 404 timeout error
- Flow: Fixed an issue where Workflwos would prematurely report completion of an Analysis
- Flow: Improved Pipeline retry logic to reduce startup delays
- Flow: Fixed an issue where Nextflow pipelines were created with empty files (Nextflow config is allowed to be empty)
- Flow: Removed the 1,000 input file limitation when starting an Analysis
- Flow: Improved the performance of status update messages for pipelines with many parallel steps
- Flow: Fixed an issue with overlapping fields on the Analysis Details screen
- Flow: Deactivated the Abort button for Succeeded analyses
- Base: Fixed an issue where Pipeline metadata was not captured in the metadata Table generated by the metadata schedule
- Base: Error logging and notification enhancements
- Bench: Fixed an issue where Workspaces could be started twice
- Bench: Fixed an issue where the system checkpoint folder was incorrectly created in Project data when opening a file in a Workspace
- Added reference documentation for deep links to a folder.
- Added descriptions of cursor and offset pagination when using the API.
- Added guidance for Base JSON export.
- Updated supported formats for Base table/query export.
- Expanded explanation of default settings for the CLI.
- Updated description of Base Activity screen.
- Provided documentation for available Bench compute types.
- Updated Nextflow DRAGEN tutorial to include details about how to obtain access to the docker image on ICA.
- Improved documentation about pipeline activation codes.
- Created new page for and updated list of available data file formats.
- Updated guidance for requirements for adding assets to a bundle.
- Explained limit of number of projects and bundles allowed per tenant.
- Updated service connector reference docs.
- Added guidance for Base schedule's optional parameters.
- Explained progress details for service connector.
- Added information about archiving or unarchiving a file.
- Analysis system infrastructure updates
- Added documentation for Analysis auto launch
- Added ability to refresh Batch Jobs updates without needing to leave the Details screen.
- Projects will receive a job queuing priority which can be adjusted by an Administrator.
- The text "Only showing the first 100 projects. Use the search criteria to find your projects or switch to Table view." when performing queries is now displayed both on the top and bottom of the page for more clarity.
- API: Added a new endpoint to retrieve download URLs for data: POST/api/projects/{projectId}/data:createDownloadUrls
- API: Added support for paging of the Project Data/getProjectDataChildren endpoint to handle large amounts of data.
- API: Added anew endpoint to deprecate a bundle (POST /api/bundles/{bundleId}:deprecate)
- API: If the API client provides request header "Accept-Encoding: gzip", then the API applies GZIP compression on the JSON response. This way the size of the response is significantly smaller which improves the download time of the response, resulting in faster end-to-end API calls. In case of compression the API also provides header "Content-Encoding: gzip" in the response, indicating that compression was effectively applied.
- Flow: Optimized Analysis storage billing, resulting in reduced pipeline charges.
- Flow: Internal details of a (non-graphical) pipeline marked ‘Proprietary’ will not be shared with users from a different tenant.
- Flow: A new grid layout is used to display Logs for Analyses with more than 50 steps. The classic view is retained for analyses with 50 steps or less, though you can choose to also use the grid layout by means of a grid button on the top right on the Analysis Log tab.
- CLI: Command to launch a CWL and Nextlfow Pipeline now contains the mount path as a parameter.
- CLI: Version command now contains the build number.
- CLI: Added support for providing the nextflow.config file when creating a Nextflow pipeline via CLI.
- API: HTML documentation for aPipeline can now be returned with the following requests:
- GET /api/pipelines/{pipelineId}/documentation/HTML
- GET /api/projects/{projectId}/pipelines/{pipelineId}/documentation/HTML
- API: Added a new endpoint for creating and starting multiple analyses in batch: POST /api/projects/{projectId}/analysisCreationBatch
- Flow: Linking to individual Analyses and Workflow sessions is now supported by /ica/link/project//analysis/ and /ica/link/project//workflowSession/
- Cohorts: Users can now export subject lists to the ICA Project Data as a file.
- Cohorts: Users can query their ingested data through ICA Base. For users who already have ingested private data into ICA Cohorts, another ingestion will need to happen prior to seeing available database shares. Customers can contact support to have previously ingested data sets available in Base.
- Cohorts: Correlation bubble plot counts now link to a subject/sample list.
- Tooltip in the Project Team page provides information about the status of an invite
- ‘Resend invite’ button in the Project Team page will become available only when the invite is expired instead of from the moment the invite is being send out
- Folders, subfolders and files all contain information about which user created the data
- Files and folders with UTF-8 character are not supported. Please look at the documentation on how to recover from it in case you already have used them.
- Improved performance for creating or hiding a Project in a tenant with many Projects
- Service Connector: Updated information in the Service Connector screen to reflect the name change from "Type of Files" to the more accurate "Assign Format"
- Service Connector: Folders within a Bundle can be downloaded via the Service Connector
- Service Connector: Upload rules can only be modified in the Project where they apply
- Service Connector: A message describes when a file is skipped during upload because it already exists in the Project
- Service Connector: Fixed an issue where opening the Connectivity tab occasionally results in a null pointer error
- Service Connector: Fixed an issue causing excessive logging when downloading files with long file paths
- Service Connector: Fixed an issue where the Service Connector log may contain spurious errors which do not impact data transfers
- Existing storage configurations are displayed and accessible via API and UI
- Newly added storage configurations do no longer remain in ‘Initializing’ state
- Fixed error when creating a volume configuration with more than 63 characters
- Clicking on a Data folder in flat mode will now open the details of the folder
- Only Tools in Released state can be added to a Bundle
- Fixed issue preventing new Bundle versions to be created from Restricted Bundles
- Deprecated Bundles are displayed upon request in card and table view
- Bundles view limited to 100 Bundles
- API: Fixed the API spec for ProjectDataTransfer.getDataTransfers
- API: Fixed an issue with the projectData getChildren endpoint which returned incorrect object and pagination
- API: Fixed an issue where multiple clicks on Create sample batch API endpoint resulted in multiple requests
- API: POST /api/projects/{projectId}/data/{dataId}:scheduleDownload can now also perform folder downloads
- API: Improved information on the Swagger page for GET /api/pipelines, GET/api/projects/{projectId}/pipelines, and GET/api/projects/{projectId}/pipelines/{pipelineId}
- API: Fixed and issue when a user provides the same input multiple times to a multi-value input on an analysis run, that input is only passed to the pipeline once instead of multiple times: POST /api/projects/{projectId}/analysis:nextflow
- CLI: Copying files in the CLI from a local directory on MacOS to your Project can result in both the desired file and the metadata file (beginning with ‘./’) being uploaded. The metadata file can safely be deleted from the Project
- CLI: Hardened protection against accidental file overwriting
- CLI: Improved handling for FUSE when connection to ICA is lost
- CLI: icav2 projectdata mount –list shows updated list of mounted Projects
- CLI: Paging improvements made for project list, projectanalyses list, and projectsdata list
- CLI: When there is no config or session file the user will not be asked to create one for icav2 config reset and icav2 config get
- CLI: Fixed an issue where Bundle data could not be seen through FUSE in Bench
- CLI: Fixed an error message when missing config file upon entering the Project context
- CLI: The unmount is possible without a path and will work via the stored Project ID or with a directory path resulting in an unmount of that path
- CLI: Fixed an error when creating a Pipeline using URN for Project identifier
- CLI: Attempting to delete a file from an externally-managed project returns an error indicating this not allowed
- CLI: Fix to delete session file when config file is not detected
- CLI: Paging option added to projectsamples list data
- CLI: Fixed “Error finding children for data” error in CLI when downloading a folder
- CLI: projectdata list now returns the correct page-size results
- Flow: Fixed handling of special characters in CWL pipeline file names
- Flow: Fixed an issue where task names exceeding 25 characters cause analysis failure in CWL pipelines
- Flow: Fixed an issue which prevented requests for economy tier compute
- Flow: Fixed an issue limiting CWL workflow concurrency to two running tasks
- Flow: Fixed an issue where analysis file inputs specified in the input.json with ‘location’set to an external URL cause to CWL pipelines to fail
- Flow: Fixed an issue resulting in out of sync Pipeline statuses
- Flow: Improved Nextflow engine resiliency, including occurrences where Nextflow pipelines fail with ‘pod 404 not found’ error
- Flow: Fix issue with intermittent system runtime failures incorrectly causing analysis failures
- Flow: Fixed an issue where links to Analysis Details returned errors
- Flow: Enabled scrolling for Pipeline documentation
- Flow: Improved performance for handling analyses with large numbers of inputs
- Flow: Improved handling of hanging Analyses
- Flow: Improved error messages for failed Pipelines
- Flow: Added documentation on how to use XML configuration files for CWL Pipelines
- Flow: Duplicate values for multi-value parameters are no longer automatically removed
- Flow: Correct exit code 0 is shown for successful Pipeline steps
- Base: Fixed an issue so that only users with correct permissions are allowed to retrieve a list of Base tables
- Base: Fixed an issue with metadata scheduler resulting in a null pointer
- Base: An empty Table description will not return an error when requesting to list all Tables in a Project
- Base: Jobs failed with an error containing 'has locked table' are not shown on the Base Job activity list. They can be displayed by selecting the 'Show transient failures' checkbox at Projects > Activity > Base Jobs.
- Base: Users can see Schedulers and their results for the entire tenant if created by a tenant administrator in their project, but not create, edit or run them
- Base: Fixed an issue preventing data format change in a schedule
- Base: Fixed an issue preventing exporting data to Excel format
- Bench: Improved handling to prevent multiple users in a single running Workspace
- Bench: Fixed an issue causing Workspaces to be stuck in "Starting" state
- Bench: Fixed an issue where usage does not showing up on usage CSV-based report
- Bench: Fixed an issue where Bundle data could not be seen via the Fuse driver
- Bench: Users can now consistently exit Workspaces with a single click on the ‘Back’ button.
- Bench: After leaving a Workspace by clicking on the ‘Back’ button, the Workspace will remain in a ‘Running’ state and become available for a new user to access
- Bench: Workspaces in a ‘Stuck’ state can be manually changed to ‘Error’ state, allowing users to restart or delete them
- Cohorts: Fixed issue where file system cleanup not occurring after delete.
- Cohorts: Fixed sign in and authentication issues in APN1 region.
- Cohorts: Fixed issue where gene filter missing when editing a cohort and removing the edited filter and cancelling. The filter was preserved and should not have been.
- Cohorts: Fixed issue where users see an application tile in the Illumina application dashboard selection screen called "Cohort Analysis Module".
- Cohorts: Correlation: Fixed issue, Data type selections shows half when loading the search result
- Cohorts: Fixed issue, Users will see an application tile on the Connected Platform home page screen called “Cohort Analysis Module” if the Cohorts module is added to the domain. Users should not enter the ICA Cohorts through this page. They should enter through ICA."
- Added guidance on using Cohorts data in ICA Base.
- Included video overview of Cohorts Multiomics Cancer workflow.
- Updated steps for Cohorts ingestion.
- Updated steps for Cohorts analysis.
- Provided a command index for the ICA v2 CLI.
- Expanded explanation on default settings for the CLI.
- Clarified details about hidden metadata files behavior during CLI copy for MacOS users.
- Added information about standard and economy tier selection for compute types.
- Updated information in the service connector screen to reflect the name change from "Type of Files" to the more accurate "Assign Format".
- Expanded guidance for XML-based input form required of pipelines defined using the "Code" mode.
- Expanded guidance on importing external docker images in ICA when running a pipeline.
- Included information about tile/grid view for analysis logs.
- Added guidance about balancing analysis queuing priorities when setting up a Project.
- Updated reference docs for workflow session events.
- Provided details about price-per-sample and zero-cost pipelines.
- Added details about which data formats are compatible with UI Preview.
- Updated information about Batch Jobs.
- Added details about syntax for linking to individual analyses and workflow sessions.
- Described pipeline Proprietary flag available.
- Added guidance about overriding CWL workflow requirements.
- Explained how to add assets to a bundle.
- New notifications added for data upload, data created, and analyses with ‘aborted’ state
- Flow: Progress on batch jobs can now be monitored in the Activity screen under the ‘Batch Jobs’ tab
- Base: Shared database activity is available in the Base Activity Logging Screen
Cohorts:
- Enhanced Attribute Queries: Filter logic now allows users to include or exclude filter criteria.
- Ingestion Wizard: Human readable project name added to email notification of ingestion status
- Ingestion Wizard: Notification email now includes the project name and job failure details
- Ingestion Wizard: File lists are now sortable and filterable by type
- Ingestion Wizard: UI enhancements on ingestion wizard selections
- Ingestion Wizard: Based on data type chosen, the list of files for ingestion will be filtered for easier navigation of files to ingest
- Share Cohorts with Collaborators - Cohort definitions are now sharable in a project
- Statistical significance on Cohort Analysis - A p-value calculation is added to the Survival Comparison data points
- Resolved an error when creating a Bucket Name with more than 63 characters
- Fixed an issue causing failure when attempting to link "complex" folders (i.e., contains 100 sub-folders each with sub-folders) via GUI throws a “…transaction is not active…" error
- Fixed an issue causing intermittent failure of "complex" folder (i.e., contains 100 sub-folders each with sub-folders) downloads
- Fixed an issue so that the API to create data download URL correctly returns a 503 instead of a 404 response when the underlying system is experiencing network-related issues
- Fixed an issue where the user is not redirected to the Project overview view upon clicking ‘Hide a Project’
- Fixed an issue which prevented linking and unlinking multiple Bundles to and from a Project in a single operation via the GUI
- Fixed an issue to show records in Bundle Data Transfers when data from a Bundle is accessed via View Tab and when downloaded
- Fixed an issue showing stack trace when attempting to add Secondary Data to Data when not allowed
- Fixed issue where the File Upload selection disappears after navigating back to Data after navigating to the Data History Tab
- Fixed an issue where create folder session and get upload URL APIs are allowed for data accessible via an entitled Bundle
- Fixed an issue where OpenAPI spec for delete project bundle API does not include proper "accept" header when using "Try It" functionality
- Fixed an issue where write-access can be acquired for data accessible via an entitled Bundle
- Fixed an issue where Project owner of an externally managed Project could not open Analysis Overview
- Fixed an issue where multiple OS versions of the Service Connector are incorrectly downloaded at once when a single version is selected
- Fixed an issue when selecting the "empty" option in the Operating System drop-down when downloading a connector
- Fixed an issue where users with Contributor role and no data upload access can add data to samples without error
- Added an event log entry for Service Connector log uploads initiated by Illumina Support
- Flow: Fixed an issue to support Aborting an Analysis immediately after starting it
- Flow: Fixed an issue when publishDir in Nextflow pipelines contains symlinks to data, those data are not included when uploading analysis outputs to a Project
- Flow: Fixed an issue preventing closure of tabs in the Logs section for technical steps in the Analysis Details view
- Flow: Cancel button added to the define CWL Pipeline selection dialogue window, and the order of buttons has been changed
- Fixed an issue causing pipelines to stall for hours after all steps are completed
- Flow: Fixed an issue where Nextflow pipelines fail with ‘pod 404 not found’ error
- Base: Linked files from a Bundle can be inserted in a Base table
- Base: Fixed an error on previewing Base tables that have been shared from a Bundle
- Base: Fixed an issue where Base API to retrieve a list of Base jobs is not paging correctly
- Bench: Fixed an issue where Bench Workspaces could not be opened with a double-click
- Bench: After the “Save and Start” of a new Bench Workspace, the breadcrumb shows the actual name of the Workspace
- Bench: Clarified error messages when creating new Workspaces with file permissions
- Cohorts: Issue fixed where cohort is created during filtering process for RNA Expression on Apply
- Cohorts: Issue fixed where order of items in the Survival Curve comparison legend are shuffled when reloaded
- Cohorts: Issue fixed where long comparison names caused by a string concatenation between cohorts were causing the comparison feature to malfunction
- Cohorts: Issue fixed where comparison name updates not reflected in the UI
- Cohorts: Issue fixed where option still remains checked after Molecular filter removed from the right panel while on the project tab
- Cohorts: Issue fixed where TCGA data not showing Molecular breakdown for selected genes
- Cohorts: Issue fixed where data selections in the correlation feature where the second option visually shows half of the result in the UI
- Cohorts: Issue fixed where "missense_and_ptv_all" missing from PHWAS/GWAS variant types
- Added guidance about modifying existing Tools already added to a Pipeline
- Clarified expected behavior and recommended order of steps when connecting input to Tools and editing Input Details when building graphical CWL pipelines
- Added information about details for monitoring Batch Jobs in Project Activity Screen
- Updated tutorials for building a DRAGEN pipeline using Nextflow
- Added Pipeline “tips and tricks”
- Added guidance on expected behavior when configuring user-managed AWS Buckets with Projects
- Updated list of system event notifications
- Added format auto-detection for VCF (.hard-filtered.vcf.gz, .cnv.vcf.gz) and TAR (.v8.tar, .tar.gz) files
- Pipeline documentation will be available if present when starting a new Analysis
- Added Workflow Session event topics
- Base: Shared databases are visible in Base tables. Shared databases are read-only and queryable tables originating from ICA modules such as Cohorts
- Reduced pipeline spin up time delays due to shared files system provisioning
- Added support for Illumina applications to add custom attributes to data, samples, workflow sessions, and analyses
- API: "creatorId" for Project data records returned in the API
- API: Added instructions for paging parameters on list endpoints in API specification
- API: The edit and delete API endpoint for Terms of Use are no longer available
- Updated ICA CLI to version 2.10.0
- Fixed an issue allowing unauthorized users to access external-managed Project contents via a direct link
- Fixed an issue in the Docker Repository view causing region text and check boxes to be misaligned
- Fixed an issue causing storage configuration status to incorrectly report an error
- Fixed an error preventing the creation of an AWS S3 storage configuration in a Project
- Fixed an issue where the Service Connector status incorrectly showing ‘Currently Connected’ when the associated Connector is not connected
- Fixed an intermittent error when clicking to expand a Connector rule details in the Connectivity View
- Removed red exclamation mark icon next to "Unlinked Files" tab in the Samples view; improperly indicates an error
- Fixed an issue where Bundle Terms of Use acceptance flag unset upon saving bundle version
- Fixed an issue preventing users from viewing details of an Entitled Bundle
- Fixed an issue resulting in an error message when creating Bundle Terms of Use with an existing version
- API: Fixed an issue where the API endpoint to link data to a Bundle to responds with errors when data is part of a Reference Data
- API: Fixed an issue causing 500 API error response when retrieving sample details
- Fixed an issue causing incomplete list of users when adding users within the same tenant to the Project Team
- API: Updated the API spec description for create upload URL and create inline view URL for Project Data
- Fix to prevent editing of an existing active terms of use
- Improved the Terms of Use UI flow for forcing re-acceptance upon update
- API: Added a warning response for API to create custom events when event code exceeds 50 characters
- API: Fixed the create data download URL API to correctly return a 503 instead of a 404 response when the underlying system is experiencing network-related issues
- API: Added API endpoint to hide a Project
- Fixed an issue with Boolean inputs incorrectly displayed as false on Analysis Details view (correct value is actually passed in)
- Fixed an issue where CWL parameters containing only a white space are passed incorrectly to underlying pipeline runner
- API: Fixed an issue where launch Nextflow analysis API returns a 500 error response when including duplicate parameters
- Fixed an issue causing analysis failures due to docker image authorization check timeout
- Fixed a shared storage issue causing analyses to remain In Progress until timeout
- Fixed an intermittent issue causing Analyses to be stuck in progress with inability to view logs in the UI
- Fixed an issue where Analysis output data is not immediately shown in the Analysis Details view upon completion
- Fixed an issue causing Analysis failures showing “…Connection timed out (Connection timed out)…” in stdout logs
- Fixed an issue causing CWL pipeline runner restarts during an Analysis which may lead to unexpected results
- Fixed an issue resulting in stuck analyses when invalid Docker image is specified for a task; timeout reduced to 15 minutes
- Fixed the name of Pipeline running logs tab from "workflow_runner" to "pipeline_runner" in Analysis Details view in the UI
- Fixed missing start time on Analysis tasks and incorrectly reported costs for such tasks
- Fixed an issue resulting in an error message when attempting to create a graphical CWL pipeline with a Tool with a Docker image that is not available in the Project's region
- Fixed an issue causing errors when saving a new Bench Workspace
- Fixed an issue causing out of sync Bench table and card views when a Workspace is created or deleted
- Fixed an issue causing an error when starting Bench Workspace when enabling service account on the Workspace from tenant receiving the Workspace via cross-tenant share
- Added tutorial for using scatter/gather for CWL graphical pipelines
- Added information about Documentation tab in Nextflow pipelines
- Included guidance on creating custom events for projects via the API
- Provided list of ICA-supported data file formats
- Updated links and fixed typos
- ICA Cohorts is released in production and added to the ICA user documentation
- Clicking on a folder in the Data view will open the folder. Once a Folder is opened, the Folder details are available on the top right of the screen.
- Added XML Configuration documentation and examples
- General ICA robustness enhancements
- Limit the Projects Overview view to 100 Projects with an indication that filtering must be used to see excluded Projects to improve UI performance
- Fixed an issue preventing removal of permissions on externally managed projects
- API – Fixed a 500 error response from Search Project Samples API when using a page token parameter
- Fixed an issue allowing Bundles to be saved without mandatory Release Version when back-navigating from the Create New Bundle screen in the graphical UI
- Fix to prevent data added to Bundle via secondary data upload on Data Details view
- API – Fixed an error message when attempting to link data to a Bundle from a Project with Data Sharing disabled
- API – Fixed an issue with tag filtering on getBundleSamples API
- Fixed an error when filtering on size in Docker Repository where size filter is ignored
- Fixed an issue where Analysis stuck in Aborting status when abort request sent during final stages of Analysis (such as Output, Upload)
- Fixed an issue where intermittent Analysis failures are caused by system scaling errors under high load
- Reduced repetitive Nextflow runner log entries from internal system logs to improve readability
- API – Fixed an error in the Launch Nextflow Pipeline API when invalid parameter names are requested
- API – Correctly support multi-value input parameters in launch Analysis APIs
- Deprecated Tools are only visible when ticking the box "Show deprecated Tools" box in both table and card views
- Fixed an issue preventing files linked to a Project from a Bundle as available to load into a Base Table
- When exporting a query result from Base the file receives the name specified by the user and contains the header from the table
- Removed unsupported AVRO and PARQUET formats from Available Formats to load data into Base tables
- Fixed an issue where Bench service account API key is not revoked when a Workspace is stopped
- Added ICA Cohorts Module documentation, including summary of features and example walk-throughs
- Added tutorial for building DRAGEN pipeline using Nextflow on ICA
- Added tutorial for building DRAGEN pipeline using CWL on ICA
- Added XML configuration documentation and examples
- Added metadata model documentation
- Added instructions for setting up ICA v2 Bring Your Own Bucket (BYOB) with SSE-KMS key
- Updated supported ICA regions
- Added data privacy recommendations regarding uploading data files
- Updated bucket name character length constraints
- Added guidance on supported characters in file/folder names
- Updated fields and descriptions under Tool Settings
- Added documentation for linking bundles to a project
- Added support for providing a nextflow.config file with a Nextflow Pipeline to improve pipeline building experience (see Nextflow Configuration docs)
- Each tenant is limited to 100 in-progress Analyses at any given time to prevent resource exhaustion; additional analyses will be queued. Note: For assistance with planning batch jobs which require greater than 100 concurrent analyses, please contact Illumina Tech Support.
- Added support for deep link URLs for direct navigation to child views in UI
- API – Added an API to link Data to Projects in batch
- API - Limit number of tags on data to 100; Excess tags in request will be ignored
- API - Added "slug" attribute to Pipeline, Project, and Data asset types
- API - Added support for the use of Project slug in place of ID in API request bodies
- API - Added “Uniform Resource Name” (URN) attribute to Pipelines and Project object types
- API - Added Project ID to notification payload for Pipeline Succeeded/Failed events
- API - Added owning Project ID for data records in the getAnalysisOutput API response
- API - Indicate originating Workflow/Workflow Session on Analyses returned via API, when applicable
- Added support to use bundled Pipelines and data to create Analyses via API without first linking to the Project
- Added support to automatically mount project data in Bench Workspace with ICA CLI
- Renamed “Delete” to “Manage” in the Samples page dropdown menu
- Removed the IGV button under View when selecting a VCF file
- API – Fixed an issue causing the List Files API to incorrectly respond with an error when passing a folder or file path parameter value containing a leading or training space
- Fixed an issue where files linked through a Project Connector will appear on root level as folder linking is not supported yet in the Project Connector
- Fixed an issue when deleting files from S3 along with records from ICA via either volume delete (with purgeobjectstore=true) or folder delete API calls, if there are more than 1,000 items to delete the system deleted only the first 1,000 items, however, all the data records got deleted from ICA. In this case the data in the user-managed S3 bucket and ICA was out of sync.
- Fixed an incorrect count displayed in UI when scheduling a download for a folder containing sub-folders
- Fixed an issue where scheduling a download of a folder containing sub-folders creates duplicate data transfer operations
- Disabled support for "fileNameExpression" on download rules; if provided when creating a download rule, the value will be ignored
- Fixed an issue allowing public domain users to incorrectly access ICA
- API – Fixed an issue where API to retrieve folder upload session intermittently responding incorrectly with error
- API – Fixed a 500 response from the API to add a pipeline to a bundle when pipeline has already been added
- Fixed an issue where user with "Viewer" permission for Flow in a project cannot link Samples from that Project
- Return the correct error response when creating a new Nextflow or CWL pipeline through the API or CLI when an existing pipeline with the same name already exists
- Improved UI messaging when setting up a Storage Configuration with permission error
- Fixed an issue when attempting to edit a saved incomplete Tool (i.e., missing Docker image)
- API - Fixed API to create notification channel to properly respond with error when attempting to create a duplicate notification channel
- Fixed an issue where boolean Analysis input settings are incorrectly set to false when passed to the Pipeline runner
- Fixed an issue when launching a Nextflow pipeline via API with multi-value inputs the order of the listed multi-value input is not preserved when passed to the pipeline runner
- Fixed an issue where "Re-run" analysis button is broken for Nextflow pipeline analyses
- When different versions of DRAGEN are run on the same host, reset the DRAGEN state so that the next task can correctly configure the FPGA compute instance
- Fixed an issue causing intermittent analysis failures due to system error when evaluating permission to pull Docker image
- Fixed empty start time attribute for Analysis tasks with a short duration (within a few seconds)
- Fixed an issue where analysis start time is set when in fact the analysis is still queued; analysis start time will be set when compute resources (i.e., shared file system) have been requested
- Fixed an issue where log file content for the pipeline runner is overwritten if system error occurs during an Analysis causing a pod restart; multiple log files may be created for the pipeline runner task if such an error occurs
- Fixed an issue causing UI log stream view for Analysis tasks to incorrectly show errors or blank views
- Fixed an issue causing incorrect Analysis task log to be displayed on the UI Logs view
- Fixed an issue where analyses get stuck in Running status if input file paths contain leading or trailing spaces in path for analysis stuck in running status if input files contain leading or trailing spaces
- Fixed an issue causing Analysis to be incorrectly marked as Success due to incorrect evaluation of exit code of an analysis task
- Fixed an issue preventing the same file to be passed to multiple input fields when launching an Analysis
- Workflow failures due to unavailable compute resources will fail quickly with a more descriptive error
- Fixed an issue where analysis output data is not immediately shown in the Analysis Details view after the analysis completes
- Removed the “Print” and “Save” buttons in the Analysis Report
- Fixed an issue preventing Pipeline creation and Tool editing when an invalid Tool is saved in the Tool Repository
- API – Fixed a 500 response from API to list Projects to properly return a 400 response when providing a search filter value with invalid characters
- API - Fixed a 500 response from create download URL API to properly return a 400 response when data is not in "Available" status (ie, Partial/Archived)
- Fixed an issue with loading data into a Base Table created directly in Snowflake with fields containing quotes
- Fixed an issue where data was added to Base Table twice when multiple Scheduled Jobs run simultaneously
- Fixed an issue with previewing Base Tables created directly via Snowflake
- Fixed an issue where status is stuck at "In Progress" for Base Copy Table operations
- Fixed an issue where Base Tables linked to a project from a bundle are shown as Views (read-only)
- Fixed an intermittent issue causing duplicate data in Base Tables
- API - Fixed a 500 response when sending a duplicate requests to load data for a Base Table to properly return a 400 response
- API - Fixed a 500 response from API to load data into Base Table to correctly return a 400 response when attempting to load data with a type mismatch
- API - Corrected Swagger UI responses
- Fix error "Unable to save workspace" when attempting to start a Bench Workspace
- Added documentation for analysis output mappings
- Added documentation for Nextflow configuration
- Added documentation for reference data
- Added documentation for metadata models
- Added Quick Registration Guide
- Added tutorial for creating/running CWL pipeline on CLI
- Added end-to-end tutorial highlighting DRAGEN Germline pipeline
Fixed Issues
- Fixed an intermittent issue where analyses fail due to "Connection timed out" system error (seen in analysis error logs)
- Added retry logic to decrease rate of pipeline failures due to internal system error (e.g. errors in pipeline runner log such as connection timeout, 404 pod not found, 500 errors)
- Fixed an intermittent issue where "Cannot access folder" error is shown in pipeline error logs (most common in long-running analyses)
- Fixed an issue resulting in analysis failures due to system error under high load
- Fixed a compute instance configuration issue impacting DRAGEN analyses
Features and Enhancements
- API - Added method to retrieve an analysis JSON input data
- Added Analysis Details views in the UI
- Added Analysis ID to the project analyses
- Support deep links for direct navigation to Analysis Details view
- Added support for multiple Nextflow versions when creating Nextflow pipeline
- API - Added method to create multiple samples in batches
- API - Improved response time performance of API that launches analysis with many data inputs
Fixed Issues
- Repaired issue where Pipeline-generated Sample Metadata is not picked up by platform
- Restored Time to delete function; now executes file deletes at the specified time
- Bench example code now lists project data and pagination correctly
- Restored ability to link secondary data to primary data
- Fixed issue where batch sample creation jobs get stuck in PARTIALLY_SUCCEEDED status
- Resolved issue where batch sample creation is not completed if data items could not be linked
- API - Fixed issue where mountPaths were ignored when launching analyses via API
- Restored function so that bundle contributors can link assets to their bundle
- Restored data linking to a sample in an ICA project
- Resolved issue where child workflow configurations prevent termination of parent workflow
- Removed incorrect error reported in cases where previous failure occurred
- Restored pipeline linking for bundle contributors
- Improved UI messaging for Storage Configuration permission errors
- API - Updated GET /api/projects/{projectId}/data to returns correct file details