Links

Release Notes

31 January 2023 - ICA v2.12.0

Features and Enhancements

  • Added ability to refresh Batch Jobs updates without needing to leave the Details screen.
  • Projects will receive a job queuing priority which can be adjusted by an Administrator.
  • The text "Only showing the first 100 projects. Use the search criteria to find your projects or switch to Table view." when performing queries is now displayed both on the top and bottom of the page for more clarity.
  • API: Added a new endpoint to retrieve download URLs for data: POST/api/projects/{projectId}/data:createDownloadUrls
  • API: Added support for paging of the Project Data/getProjectDataChildren endpoint to handle large amounts of data.
  • API: Added anew endpoint to deprecate a bundle (POST /api/bundles/{bundleId}:deprecate)
  • API: If the API client provides request header "Accept-Encoding: gzip", then the API applies GZIP compression on the JSON response. This way the size of the response is significantly smaller which improves the download time of the response, resulting in faster end-to-end API calls. In case of compression the API also provides header "Content-Encoding: gzip" in the response, indicating that compression was effectively applied.
  • Flow: Optimized Analysis storage billing, resulting in reduced pipeline charges.
  • Flow: Internal details of a (non-graphical) pipeline marked ‘Proprietary’ will not be shared with users from a different tenant.
  • Flow: A new grid layout is used to display Logs for Analyses with more than 50 steps. The classic view is retained for analyses with 50 steps or less, though you can choose to also use the grid layout by means of a grid button on the top right on the Analysis Log tab.
  • CLI: Command to launch a CWL and Nextlfow Pipeline now contains the mount path as a parameter.
  • CLI: Version command now contains the build number.
  • CLI: Added support for providing the nextflow.config file when creating a Nextflow pipeline via CLI.
  • API: HTML documentation for aPipeline can now be returned with the following requests:
    • GET /api/pipelines/{pipelineId}/documentation/HTML
    • GET /api/projects/{projectId}/pipelines/{pipelineId}/documentation/HTML
  • API: Added a new endpoint for creating and starting multiple analyses in batch: POST /api/projects/{projectId}/analysisCreationBatch
  • Flow: Linking to individual Analyses and Workflow sessions is now supported by /ica/link/project//analysis/ and /ica/link/project//workflowSession/
  • Cohorts: Users can now export subject lists to the ICA Project Data as a file.
  • Cohorts: Users can query their ingested data through ICA Base. For users who already have ingested private data into ICA Cohorts, another ingestion will need to happen prior to seeing available database shares. Customers can contact support to have previously ingested data sets available in Base.
  • Cohorts: Correlation bubble plot counts now link to a subject/sample list.

Bugs Fixed

  • Tooltip in the Project Team page provides information about the status of an invite
  • ‘Resend invite’ button in the Project Team page will become available only when the invite is expired instead of from the moment the invite is being send out
  • Folders, subfolders and files all contain information about which user created the data
  • Files and folders with UTF-8 character are not supported. Please look at the documentation on how to recover from it in case you already have used them.
  • Improved performance for creating or hiding a Project in a tenant with many Projects
  • Service Connector: Updated information in the Service Connector screen to reflect the name change from "Type of Files" to the more accurate "Assign Format"
  • Service Connector: Folders within a Bundle can be downloaded via the Service Connector
  • Service Connector: Upload rules can only be modified in the Project where they apply
  • Service Connector: A message describes when a file is skipped during upload because it already exists in the Project
  • Service Connector: Fixed an issue where opening the Connectivity tab occasionally results in a null pointer error
  • Service Connector: Fixed an issue causing excessive logging when downloading files with long file paths
  • Service Connector: Fixed an issue where the Service Connector log may contain spurious errors which do not impact data transfers
  • Existing storage configurations are displayed and accessible via API and UI
  • Newly added storage configurations do no longer remain in ‘Initializing’ state
  • Fixed error when creating a volume configuration with more than 63 characters
  • Clicking on a Data folder in flat mode will now open the details of the folder
  • Only Tools in Released state can be added to a Bundle
  • Fixed issue preventing new Bundle versions to be created from Restricted Bundles
  • Deprecated Bundles are displayed upon request in card and table view
  • Bundles view limited to 100 Bundles
  • API: Fixed the API spec for ProjectDataTransfer.getDataTransfers
  • API: Fixed an issue with the projectData getChildren endpoint which returned incorrect object and pagination
  • API: Fixed an issue where multiple clicks on Create sample batch API endpoint resulted in multiple requests
  • API: POST /api/projects/{projectId}/data/{dataId}:scheduleDownload can now also perform folder downloads
  • API: Improved information on the Swagger page for GET /api/pipelines, GET/api/projects/{projectId}/pipelines, and GET/api/projects/{projectId}/pipelines/{pipelineId}
  • API: Fixed and issue when a user provides the same input multiple times to a multi-value input on an analysis run, that input is only passed to the pipeline once instead of multiple times: POST /api/projects/{projectId}/analysis:nextflow
  • CLI: Copying files in the CLI from a local directory on MacOS to your Project can result in both the desired file and the metadata file (beginning with ‘./’) being uploaded. The metadata file can safely be deleted from the Project
  • CLI: Hardened protection against accidental file overwriting
  • CLI: Improved handling for FUSE when connection to ICA is lost
  • CLI: icav2 projectdata mount –list shows updated list of mounted Projects
  • CLI: Paging improvements made for project list, projectanalyses list, and projectsdata list
  • CLI: When there is no config or session file the user will not be asked to create one for icav2 config reset and icav2 config get
  • CLI: Fixed an issue where Bundle data could not be seen through FUSE in Bench
  • CLI: Fixed an error message when missing config file upon entering the Project context
  • CLI: The unmount is possible without a path and will work via the stored Project ID or with a directory path resulting in an unmount of that path
  • CLI: Fixed an error when creating a Pipeline using URN for Project identifier
  • CLI: Attempting to delete a file from an externally-managed project returns an error indicating this not allowed
  • CLI: Fix to delete session file when config file is not detected
  • CLI: Paging option added to projectsamples list data
  • CLI: Fixed “Error finding children for data” error in CLI when downloading a folder
  • CLI: projectdata list now returns the correct page-size results
  • Flow: Fixed handling of special characters in CWL pipeline file names
  • Flow: Fixed an issue where task names exceeding 25 characters cause analysis failure in CWL pipelines
  • Flow: Fixed an issue which prevented requests for economy tier compute
  • Flow: Fixed an issue limiting CWL workflow concurrency to two running tasks
  • Flow: Fixed an issue where analysis file inputs specified in the input.json with ‘location’set to an external URL cause to CWL pipelines to fail
  • Flow: Fixed an issue resulting in out of sync Pipeline statuses
  • Flow: Improved Nextflow engine resiliency, including occurrences where Nextflow pipelines fail with ‘pod 404 not found’ error
  • Flow: Fix issue with intermittent system runtime failures incorrectly causing analysis failures
  • Flow: Fixed an issue where links to Analysis Details returned errors
  • Flow: Enabled scrolling for Pipeline documentation
  • Flow: Improved performance for handling analyses with large numbers of inputs
  • Flow: Improved handling of hanging Analyses
  • Flow: Improved error messages for failed Pipelines
  • Flow: Added documentation on how to use XML configuration files for CWL Pipelines
  • Flow: Duplicate values for multi-value parameters are no longer automatically removed
  • Flow: Correct exit code 0 is shown for successful Pipeline steps
  • Base: Fixed an issue so that only users with correct permissions are allowed to retrieve a list of Base tables
  • Base: Fixed an issue with metadata scheduler resulting in a null pointer
  • Base: An empty Table description will not return an error when requesting to list all Tables in a Project
  • Base: Jobs failed with an error containing 'has locked table' are not shown on the Base Job activity list. They can be displayed by selecting the 'Show transient failures' checkbox at Projects > Activity > Base Jobs.
  • Base: Users can see Schedulers and their results for the entire tenant if created by a tenant administrator in their project, but not create, edit or run them
  • Base: Fixed an issue preventing data format change in a schedule
  • Base: Fixed an issue preventing exporting data to Excel format
  • Bench: Improved handling to prevent multiple users in a single running Workspace
  • Bench: Fixed an issue causing Workspaces to be stuck in "Starting" state
  • Bench: Fixed an issue where usage does not showing up on usage CSV-based report
  • Bench: Fixed an issue where Bundle data could not be seen via the Fuse driver
  • Bench: Users can now consistently exit Workspaces with a single click on the ‘Back’ button.
  • Bench: After leaving a Workspace by clicking on the ‘Back’ button, the Workspace will remain in a ‘Running’ state and become available for a new user to access
  • Bench: Workspaces in a ‘Stuck’ state can be manually changed to ‘Error’ state, allowing users to restart or delete them
  • Cohorts: Fixed issue where file system cleanup not occurring after delete.
  • Cohorts: Fixed sign in and authentication issues in APN1 region.
  • Cohorts: Fixed issue where gene filter missing when editing a cohort and removing the edited filter and cancelling. The filter was preserved and should not have been.
  • Cohorts: Fixed issue where users see an application tile in the Illumina application dashboard selection screen called "Cohort Analysis Module".
  • Cohorts: Correlation: Fixed issue, Data type selections shows half when loading the search result
  • Cohorts: Fixed issue, Users will see an application tile on the Connected Platform home page screen called “Cohort Analysis Module” if the Cohorts module is added to the domain. Users should not enter the ICA Cohorts through this page. They should enter through ICA."

Documentation

  • Added guidance on using Cohorts data in ICA Base.
  • Included video overview of Cohorts Multiomics Cancer workflow.
  • Updated steps for Cohorts ingestion.
  • Updated steps for Cohorts analysis.
  • Provided a command index for the ICA v2 CLI.
  • Expanded explanation on default settings for the CLI.
  • Clarified details about hidden metadata files behavior during CLI copy for MacOS users.
  • Added information about standard and economy tier selection for compute types.
  • Updated information in the service connector screen to reflect the name change from "Type of Files" to the more accurate "Assign Format".
  • Expanded guidance for XML-based input form required of pipelines defined using the "Code" mode.
  • Expanded guidance on importing external docker images in ICA when running a pipeline.
  • Included information about tile/grid view for analysis logs.
  • Added guidance about balancing analysis queuing priorities when setting up a Project.
  • Updated reference docs for workflow session events.
  • Provided details about price-per-sample and zero-cost pipelines.
  • Added details about which data formats are compatible with UI Preview.
  • Updated information about Batch Jobs.
  • Added details about syntax for linking to individual analyses and workflow sessions.
  • Described pipeline Proprietary flag available.
  • Added guidance about overriding CWL workflow requirements.
  • Explained how to add assets to a bundle.

13 December 2022 - ICA v2.11.0

Features and Enhancements

  • New notifications added for data upload, data created, and analyses with ‘aborted’ state
  • Flow: Progress on batch jobs can now be monitored in the Activity screen under the ‘Batch Jobs’ tab
  • Base: Shared database activity is available in the Base Activity Logging Screen
Cohorts:
  • Enhanced Attribute Queries: Filter logic now allows users to include or exclude filter criteria.
  • Ingestion Wizard: Human readable project name added to email notification of ingestion status
  • Ingestion Wizard: Notification email now includes the project name and job failure details
  • Ingestion Wizard: File lists are now sortable and filterable by type
  • Ingestion Wizard: UI enhancements on ingestion wizard selections
  • Ingestion Wizard: Based on data type chosen, the list of files for ingestion will be filtered for easier navigation of files to ingest
  • Share Cohorts with Collaborators - Cohort definitions are now sharable in a project
  • Statistical significance on Cohort Analysis - A p-value calculation is added to the Survival Comparison data points

Bugs Fixed

  • Resolved an error when creating a Bucket Name with more than 63 characters
  • Fixed an issue causing failure when attempting to link "complex" folders (i.e., contains 100 sub-folders each with sub-folders) via GUI throws a “…transaction is not active…" error
  • Fixed an issue causing intermittent failure of "complex" folder (i.e., contains 100 sub-folders each with sub-folders) downloads
  • Fixed an issue so that the API to create data download URL correctly returns a 503 instead of a 404 response when the underlying system is experiencing network-related issues
  • Fixed an issue where the user is not redirected to the Project overview view upon clicking ‘Hide a Project’
  • Fixed an issue which prevented linking and unlinking multiple Bundles to and from a Project in a single operation via the GUI
  • Fixed an issue to show records in Bundle Data Transfers when data from a Bundle is accessed via View Tab and when downloaded
  • Fixed an issue showing stack trace when attempting to add Secondary Data to Data when not allowed
  • Fixed issue where the File Upload selection disappears after navigating back to Data after navigating to the Data History Tab
  • Fixed an issue where create folder session and get upload URL APIs are allowed for data accessible via an entitled Bundle
  • Fixed an issue where OpenAPI spec for delete project bundle API does not include proper "accept" header when using "Try It" functionality
  • Fixed an issue where write-access can be acquired for data accessible via an entitled Bundle
  • Fixed an issue where Project owner of an externally managed Project could not open Analysis Overview
  • Fixed an issue where multiple OS versions of the Service Connector are incorrectly downloaded at once when a single version is selected
  • Fixed an issue when selecting the "empty" option in the Operating System drop-down when downloading a connector
  • Fixed an issue where users with Contributor role and no data upload access can add data to samples without error
  • Added an event log entry for Service Connector log uploads initiated by Illumina Support
  • Flow: Fixed an issue to support Aborting an Analysis immediately after starting it
  • Flow: Fixed an issue when publishDir in Nextflow pipelines contains symlinks to data, those data are not included when uploading analysis outputs to a Project
  • Flow: Fixed an issue preventing closure of tabs in the Logs section for technical steps in the Analysis Details view
  • Flow: Cancel button added to the define CWL Pipeline selection dialogue window, and the order of buttons has been changed
  • Fixed an issue causing pipelines to stall for hours after all steps are completed
  • Flow: Fixed an issue where Nextflow pipelines fail with ‘pod 404 not found’ error
  • Base: Linked files from a Bundle can be inserted in a Base table
  • Base: Fixed an error on previewing Base tables that have been shared from a Bundle
  • Base: Fixed an issue where Base API to retrieve a list of Base jobs is not paging correctly
  • Bench: Fixed an issue where Bench Workspaces could not be opened with a double-click
  • Bench: After the “Save and Start” of a new Bench Workspace, the breadcrumb shows the actual name of the Workspace
  • Bench: Clarified error messages when creating new Workspaces with file permissions
  • Cohorts: Issue fixed where cohort is created during filtering process for RNA Expression on Apply
  • Cohorts: Issue fixed where order of items in the Survival Curve comparison legend are shuffled when reloaded
  • Cohorts: Issue fixed where long comparison names caused by a string concatenation between cohorts were causing the comparison feature to malfunction
  • Cohorts: Issue fixed where comparison name updates not reflected in the UI
  • Cohorts: Issue fixed where option still remains checked after Molecular filter removed from the right panel while on the project tab
  • Cohorts: Issue fixed where TCGA data not showing Molecular breakdown for selected genes
  • Cohorts: Issue fixed where data selections in the correlation feature where the second option visually shows half of the result in the UI
  • Cohorts: Issue fixed where "missense_and_ptv_all" missing from PHWAS/GWAS variant types

Documentation

  • Added guidance about modifying existing Tools already added to a Pipeline
  • Clarified expected behavior and recommended order of steps when connecting input to Tools and editing Input Details when building graphical CWL pipelines
  • Added information about details for monitoring Batch Jobs in Project Activity Screen
  • Updated tutorials for building a DRAGEN pipeline using Nextflow
  • Added Pipeline “tips and tricks”
  • Added guidance on expected behavior when configuring user-managed AWS Buckets with Projects
  • Updated list of system event notifications

8 November 2022 - ICA v2.10.0

Features and Enhancements

  • Added format auto-detection for VCF (.hard-filtered.vcf.gz, .cnv.vcf.gz) and TAR (.v8.tar, .tar.gz) files
  • Pipeline documentation will be available if present when starting a new Analysis
  • Added Workflow Session event topics
  • Base: Shared databases are visible in Base tables. Shared databases are read-only and queryable tables originating from ICA modules such as Cohorts
  • Reduced pipeline spin up time delays due to shared files system provisioning
  • Added support for Illumina applications to add custom attributes to data, samples, workflow sessions, and analyses
  • API: "creatorId" for Project data records returned in the API
  • API: Added instructions for paging parameters on list endpoints in API specification
  • API: The edit and delete API endpoint for Terms of Use are no longer available
  • Updated ICA CLI to version 2.10.0

Bugs Fixed

  • Fixed an issue allowing unauthorized users to access external-managed Project contents via a direct link
  • Fixed an issue in the Docker Repository view causing region text and check boxes to be misaligned
  • Fixed an issue causing storage configuration status to incorrectly report an error
  • Fixed an error preventing the creation of an AWS S3 storage configuration in a Project
  • Fixed an issue where the Service Connector status incorrectly showing ‘Currently Connected’ when the associated Connector is not connected
  • Fixed an intermittent error when clicking to expand a Connector rule details in the Connectivity View
  • Removed red exclamation mark icon next to "Unlinked Files" tab in the Samples view; improperly indicates an error
  • Fixed an issue where Bundle Terms of Use acceptance flag unset upon saving bundle version
  • Fixed an issue preventing users from viewing details of an Entitled Bundle
  • Fixed an issue resulting in an error message when creating Bundle Terms of Use with an existing version
  • API: Fixed an issue where the API endpoint to link data to a Bundle to responds with errors when data is part of a Reference Data
  • API: Fixed an issue causing 500 API error response when retrieving sample details
  • Fixed an issue causing incomplete list of users when adding users within the same tenant to the Project Team
  • API: Updated the API spec description for create upload URL and create inline view URL for Project Data
  • Fix to prevent editing of an existing active terms of use
  • Improved the Terms of Use UI flow for forcing re-acceptance upon update
  • API: Added a warning response for API to create custom events when event code exceeds 50 characters
  • API: Fixed the create data download URL API to correctly return a 503 instead of a 404 response when the underlying system is experiencing network-related issues
  • API: Added API endpoint to hide a Project
  • Fixed an issue with Boolean inputs incorrectly displayed as false on Analysis Details view (correct value is actually passed in)
  • Fixed an issue where CWL parameters containing only a white space are passed incorrectly to underlying pipeline runner
  • API: Fixed an issue where launch Nextflow analysis API returns a 500 error response when including duplicate parameters
  • Fixed an issue causing analysis failures due to docker image authorization check timeout
  • Fixed a shared storage issue causing analyses to remain In Progress until timeout
  • Fixed an intermittent issue causing Analyses to be stuck in progress with inability to view logs in the UI
  • Fixed an issue where Analysis output data is not immediately shown in the Analysis Details view upon completion
  • Fixed an issue causing Analysis failures showing “…Connection timed out (Connection timed out)…” in stdout logs
  • Fixed an issue causing CWL pipeline runner restarts during an Analysis which may lead to unexpected results
  • Fixed an issue resulting in stuck analyses when invalid Docker image is specified for a task; timeout reduced to 15 minutes
  • Fixed the name of Pipeline running logs tab from "workflow_runner" to "pipeline_runner" in Analysis Details view in the UI
  • Fixed missing start time on Analysis tasks and incorrectly reported costs for such tasks
  • Fixed an issue resulting in an error message when attempting to create a graphical CWL pipeline with a Tool with a Docker image that is not available in the Project's region
  • Fixed an issue causing errors when saving a new Bench Workspace
  • Fixed an issue causing out of sync Bench table and card views when a Workspace is created or deleted
  • Fixed an issue causing an error when starting Bench Workspace when enabling service account on the Workspace from tenant receiving the Workspace via cross-tenant share

Documentation

  • Added tutorial for using scatter/gather for CWL graphical pipelines
  • Added information about Documentation tab in Nextflow pipelines
  • Included guidance on creating custom events for projects via the API
  • Provided list of ICA-supported data file formats
  • Updated links and fixed typos

17 October 2022 - ICA v2.9.0

Features and Enhancements

  • ICA Cohorts is released in production and added to the ICA user documentation
  • Clicking on a folder in the Data view will open the folder. Once a Folder is opened, the Folder details are available on the top right of the screen.
  • Added XML Configuration documentation and examples
  • General ICA robustness enhancements

Bugs Fixed

  • Limit the Projects Overview view to 100 Projects with an indication that filtering must be used to see excluded Projects to improve UI performance
  • Fixed an issue preventing removal of permissions on externally managed projects
  • API – Fixed a 500 error response from Search Project Samples API when using a page token parameter
  • Fixed an issue allowing Bundles to be saved without mandatory Release Version when back-navigating from the Create New Bundle screen in the graphical UI
  • Fix to prevent data added to Bundle via secondary data upload on Data Details view
  • API – Fixed an error message when attempting to link data to a Bundle from a Project with Data Sharing disabled
  • API – Fixed an issue with tag filtering on getBundleSamples API
  • Fixed an error when filtering on size in Docker Repository where size filter is ignored
  • Fixed an issue where Analysis stuck in Aborting status when abort request sent during final stages of Analysis (such as Output, Upload)
  • Fixed an issue where intermittent Analysis failures are caused by system scaling errors under high load
  • Reduced repetitive Nextflow runner log entries from internal system logs to improve readability
  • API – Fixed an error in the Launch Nextflow Pipeline API when invalid parameter names are requested
  • API – Correctly support multi-value input parameters in launch Analysis APIs
  • Deprecated Tools are only visible when ticking the box "Show deprecated Tools" box in both table and card views
  • Fixed an issue preventing files linked to a Project from a Bundle as available to load into a Base Table
  • When exporting a query result from Base the file receives the name specified by the user and contains the header from the table
  • Removed unsupported AVRO and PARQUET formats from Available Formats to load data into Base tables
  • Fixed an issue where Bench service account API key is not revoked when a Workspace is stopped

Documentation

  • Added ICA Cohorts Module documentation, including summary of features and example walk-throughs
  • Added tutorial for building DRAGEN pipeline using Nextflow on ICA
  • Added tutorial for building DRAGEN pipeline using CWL on ICA
  • Added XML configuration documentation and examples
  • Added metadata model documentation
  • Added instructions for setting up ICA v2 Bring Your Own Bucket (BYOB) with SSE-KMS key
  • Updated supported ICA regions
  • Added data privacy recommendations regarding uploading data files
  • Updated bucket name character length constraints
  • Added guidance on supported characters in file/folder names
  • Updated fields and descriptions under Tool Settings
  • Added documentation for linking bundles to a project

12 September 2022 - ICA v2.8.0 and v2.8.1

Features and Enhancements

  • Added support for providing a nextflow.config file with a Nextflow Pipeline to improve pipeline building experience (see Nextflow Configuration docs)
  • Each tenant is limited to 100 in-progress Analyses at any given time to prevent resource exhaustion; additional analyses will be queued. Note: For assistance with planning batch jobs which require greater than 100 concurrent analyses, please contact Illumina Tech Support.
  • Added support for deep link URLs for direct navigation to child views in UI
  • API – Added an API to link Data to Projects in batch
  • API - Limit number of tags on data to 100; Excess tags in request will be ignored
  • API - Added "slug" attribute to Pipeline, Project, and Data asset types
  • API - Added support for the use of Project slug in place of ID in API request bodies
  • API - Added “Uniform Resource Name” (URN) attribute to Pipelines and Project object types
  • API - Added Project ID to notification payload for Pipeline Succeeded/Failed events
  • API - Added owning Project ID for data records in the getAnalysisOutput API response
  • API - Indicate originating Workflow/Workflow Session on Analyses returned via API, when applicable
  • Added support to use bundled Pipelines and data to create Analyses via API without first linking to the Project
  • Added support to automatically mount project data in Bench Workspace with ICA CLI

Bugs Fixed

  • Renamed “Delete” to “Manage” in the Samples page dropdown menu
  • Removed the IGV button under View when selecting a VCF file
  • API – Fixed an issue causing the List Files API to incorrectly respond with an error when passing a folder or file path parameter value containing a leading or training space
  • Fixed an issue where files linked through a Project Connector will appear on root level as folder linking is not supported yet in the Project Connector
  • Fixed an issue when deleting files from S3 along with records from ICA via either volume delete (with purgeobjectstore=true) or folder delete API calls, if there are more than 1,000 items to delete the system deleted only the first 1,000 items, however, all the data records got deleted from ICA. In this case the data in the user-managed S3 bucket and ICA was out of sync.
  • Fixed an incorrect count displayed in UI when scheduling a download for a folder containing sub-folders
  • Fixed an issue where scheduling a download of a folder containing sub-folders creates duplicate data transfer operations
  • Disabled support for "fileNameExpression" on download rules; if provided when creating a download rule, the value will be ignored
  • Fixed an issue allowing public domain users to incorrectly access ICA
  • API – Fixed an issue where API to retrieve folder upload session intermittently responding incorrectly with error
  • API – Fixed a 500 response from the API to add a pipeline to a bundle when pipeline has already been added
  • Fixed an issue where user with "Viewer" permission for Flow in a project cannot link Samples from that Project
  • Return the correct error response when creating a new Nextflow or CWL pipeline through the API or CLI when an existing pipeline with the same name already exists
  • Improved UI messaging when setting up a Storage Configuration with permission error
  • Fixed an issue when attempting to edit a saved incomplete Tool (i.e., missing Docker image)
  • API - Fixed API to create notification channel to properly respond with error when attempting to create a duplicate notification channel
  • Fixed an issue where boolean Analysis input settings are incorrectly set to false when passed to the Pipeline runner
  • Fixed an issue when launching a Nextflow pipeline via API with multi-value inputs the order of the listed multi-value input is not preserved when passed to the pipeline runner
  • Fixed an issue where "Re-run" analysis button is broken for Nextflow pipeline analyses
  • When different versions of DRAGEN are run on the same host, reset the DRAGEN state so that the next task can correctly configure the FPGA compute instance
  • Fixed an issue causing intermittent analysis failures due to system error when evaluating permission to pull Docker image
  • Fixed empty start time attribute for Analysis tasks with a short duration (within a few seconds)
  • Fixed an issue where analysis start time is set when in fact the analysis is still queued; analysis start time will be set when compute resources (i.e., shared file system) have been requested
  • Fixed an issue where log file content for the pipeline runner is overwritten if system error occurs during an Analysis causing a pod restart; multiple log files may be created for the pipeline runner task if such an error occurs
  • Fixed an issue causing UI log stream view for Analysis tasks to incorrectly show errors or blank views
  • Fixed an issue causing incorrect Analysis task log to be displayed on the UI Logs view
  • Fixed an issue where analyses get stuck in Running status if input file paths contain leading or trailing spaces in path for analysis stuck in running status if input files contain leading or trailing spaces
  • Fixed an issue causing Analysis to be incorrectly marked as Success due to incorrect evaluation of exit code of an analysis task
  • Fixed an issue preventing the same file to be passed to multiple input fields when launching an Analysis
  • Workflow failures due to unavailable compute resources will fail quickly with a more descriptive error
  • Fixed an issue where analysis output data is not immediately shown in the Analysis Details view after the analysis completes
  • Removed the “Print” and “Save” buttons in the Analysis Report
  • Fixed an issue preventing Pipeline creation and Tool editing when an invalid Tool is saved in the Tool Repository
  • API – Fixed a 500 response from API to list Projects to properly return a 400 response when providing a search filter value with invalid characters
  • API - Fixed a 500 response from create download URL API to properly return a 400 response when data is not in "Available" status (ie, Partial/Archived)
  • Fixed an issue with loading data into a Base Table created directly in Snowflake with fields containing quotes
  • Fixed an issue where data was added to Base Table twice when multiple Scheduled Jobs run simultaneously
  • Fixed an issue with previewing Base Tables created directly via Snowflake
  • Fixed an issue where status is stuck at "In Progress" for Base Copy Table operations
  • Fixed an issue where Base Tables linked to a project from a bundle are shown as Views (read-only)
  • Fixed an intermittent issue causing duplicate data in Base Tables
  • API - Fixed a 500 response when sending a duplicate requests to load data for a Base Table to properly return a 400 response
  • API - Fixed a 500 response from API to load data into Base Table to correctly return a 400 response when attempting to load data with a type mismatch
  • API - Corrected Swagger UI responses
  • Fix error "Unable to save workspace" when attempting to start a Bench Workspace

Documentation

  • Added documentation for analysis output mappings
  • Added documentation for Nextflow configuration
  • Added documentation for reference data
  • Added documentation for metadata models
  • Added Quick Registration Guide
  • Added tutorial for creating/running CWL pipeline on CLI
  • Added end-to-end tutorial highlighting DRAGEN Germline pipeline

27 July 2022 - ICA v2.7.1

Bugs Fixed
  • Fixed an intermittent issue where analyses fail due to "Connection timed out" system error (seen in analysis error logs)
  • Added retry logic to decrease rate of pipeline failures due to internal system error (e.g. errors in pipeline runner log such as connection timeout, 404 pod not found, 500 errors)
  • Fixed an intermittent issue where "Cannot access folder" error is shown in pipeline error logs (most common in long-running analyses)
  • Fixed an issue resulting in analysis failures due to system error under high load
  • Fixed a compute instance configuration issue impacting DRAGEN analyses

29 June 2022 - ICA v2.7.0 and CLI v2.4.0

Features and Enhancements
  • API - Added method to retrieve an analysis JSON input data
  • Added Analysis Details views in the UI
  • Added Analysis ID to the project analyses
  • Support deep links for direct navigation to Analysis Details view
  • Added support for multiple Nextflow versions when creating Nextflow pipeline
  • API - Added method to create multiple samples in batches
  • API - Improved response time performance of API that launches analysis with many data inputs
Bugs Fixed
  • Repaired issue where Pipeline-generated Sample Metadata is not picked up by platform
  • Restored Time to delete function; now executes file deletes at the specified time
  • Bench example code now lists project data and pagination correctly
  • Restored ability to link secondary data to primary data
  • Fixed issue where batch sample creation jobs get stuck in PARTIALLY_SUCCEEDED status
  • Resolved issue where batch sample creation is not completed if data items could not be linked
  • API - Fixed issue where mountPaths were ignored when launching analyses via API
  • Restored function so that bundle contributors can link assets to their bundle
  • Restored data linking to a sample in an ICA project
  • Resolved issue where child workflow configurations prevent termination of parent workflow
  • Removed incorrect error reported in cases where previous failure occurred
  • Restored pipeline linking for bundle contributors
  • Improved UI messaging for Storage Configuration permission errors
  • API - Updated GET /api/projects/{projectId}/data to returns correct file details

07 June 2022 - ICA v2.6.0 and CLI v2.3.0

Features and Enhancements
  • Re-labeled "Run" and "Runs" to “Analysis” and “Analyses” in the UI
  • Allow storage configuration to be disconnected from a hidden Project
  • Added support for a read-only view of Entitled Bundles and Bundles shared from another Tenant
  • Changed analysis timeouts for setting up environment, downloading inputs, pipeline runner (inclusive of all child steps), and finalizing output steps to 96 hours
  • API – Add idempotency key header to createAnalysis API
  • API – Analysis logs are available via getAnalysisSteps API; includes streamable via websocket endpoint during execution and downloadable S3 URL upon completion
  • JSON data with array field can be added in a Base table
  • Cohorts – Additional submenu items now available
  • CLI – Added support for setting the current Project ID to the value set in the ICA_PROJECT_UUID environment variable upon starting the CLI
  • CLI – Added support for mounting Project data to local operating system (Read-Only)
  • CLI – Added support for the outputParentFolderId flag for the ‘projectpripeline start’ command
  • CLI – Added support for customizing the filename/path of additional Pipeline files with the pipeline create commands
  • CLI – Added owning Project name/id attributes to the project data list output
  • CLI – Added owning Project name attribute to output for projectdata list command
  • CLI – Added commands to create and refresh API tokens (JWT)
  • CLI – Added support for CLI use without writing to disk by enabling API Key and Server flags for all commands
  • CLI – Added support for linking data to a Sample with projectdata upload command
Bugs Fixed
  • Fixed an issue with UI stability leading to slow response times
  • Fixed an issue causing data uploads via browser to hang without error
  • Fixed an issue with UI automatically refreshing upon load when many Projects are present in the Projects view
  • Fixed an issue where Accept Share pop-up remains on screen after pressing to Accept the share
  • Fixed an issue with uploading nested folders using the Windows Connector
  • Fixed an issue where path separators are misinterpreted in upload rules for Windows Connectors.
  • Fixed an issue where modifying a local file uploaded (tracked) using the Connector causes a Connector failure
  • A User with Contributor Role for Flow and a Viewer Role for a Project can start an Analysis using input data from a Sample
  • API – Fixed an issue where sampleDeepDelete API fails with 500 error response when Sample contains a linked folder in archived status
  • Fixed an issue where ICA_STARTING_USER_TENANT_NAME is not set to the correct value on initial start
  • API – Fixed an issue where upload rules are not deleted via the deleteUploadRule API despite a success response
  • Sensitive fields in metadata models are now shown in italics
  • Fixed an issue where filtering on the Data view gives incorrect results
  • All Contributors in a Bundle are allowed to link assets to the Bundle
  • Content of a Bundle shared via an entitlement in another Tenant is now visible
  • Fixed an externally managed Project Team to only allow adding members with "Viewer" and "Allow Download" permissions
  • API – Fixed an issue with project sample APIs returning 500 error response when sample is not linked to the project specified in the request
  • API – Fixed issues with cursor-based paging on listData and listBundleData APIs returning expected results
  • Fixed an issue where a User can create empty folders inside an externally managed Project with Viewer permission
  • Fix auto-detection of AVRO and PARQUET data formats
  • API – Fixed an issue where the API to link Samples to a Bundle incorrectly returns a 204 response when the Sample is not in Available status and the Sample is not actually linked
  • API - Fixed an issue where changing the Project owner causes unexpected error responses from the listProjects, listProjectPermissions, and getProjectPermission APIs
  • API – Fixed an issue where deleteDownloadRule API returns 204 responses but does not delete the specified download rule
  • Fixed an issue causing delays in operations performed on data
  • Fixed an intermittent permission error when writing to /scratch from a Pipeline
  • Fix to make tags optional (not required) with Pipeline launch requests
  • Fixed an issue causing Analyses to be incorrectly set to SUCCEEDED status when failure occurs
  • API – Possible to create a Pipeline with a metadata model via API
  • Fixed an issue where logs are not streamed during Analysis
  • Fixed an issue when a User tries to start an Analysis with missing required parameters, the analysis is stuck in "awaiting input"
  • Fixed an issue preventing renaming a pipeline after attempting to save and receiving error that a pipeline exists with the same name
  • API – Fixed an issue when a Pipeline is launched via API with no input value provided for a nonmandatory input parameter the analysis status remains in "Awaiting Input" indefinitely
  • API – Fixed an issue causing unpredictable error responses from createAnalysis API when many Analyses are launched in rapid succession
  • API – Added operationID to API spec for the listAnalysisStorages and getAnalysisRawOutput APIs
  • Fixed an issue when Analyses are queued and Aborted before they are Running, the status is not correctly updated in the API/UI, and they appear stuck in “Aborting”
  • API – Fixed an issue with createAnalysis API causing error responses when launching many Analyses in rapid succession
  • Possible to load JSON in Base with a repeated field (array)
  • Tables created by the meta and admin data Schedules are updated after a sync
  • Fixed an issue causing Base Table field names to be changed from lowercase to UPPERCASE during a manual sync
  • Fixed an intermittent issue with Base Administrative/Meta-data Schedules creating files in project Data when schedules are run
  • Fixed an intermittent issue with Base Administrative/Meta-data Schedules creating files in project Data when scheduled
  • Fixed an issue where the User receives an obscure error message when creating a Tool from a Bench Workspace with an image name that already exists
  • Fixed an error message upon attempting to save Bench Workspace when input values are invalid
  • Fixed an issue where the Bench Workspace is temporarily inaccessible and stuck in Starting status
  • CLI – Fixed an issue with list samples command output limited to 20 unique records

April 2022 - ICA v2.5.0

Features and Enhancements
  • The value of the "Show Hidden Projects" filter on the Project Overview page will be saved for each user
  • Change analysis task log names to be more human-readable
  • Analyses will queue when system load is high
  • Add owning project tenant name (ICA_PROJECT_TENANT_NAME) and the tenant name of the user which last started the Workspace (ICA_STARTING_USER_TENANT_NAME) environment variables to Bench Workspaces
  • Add Cohorts URL environment variable to Bench Workspaces
  • API - Add support for Cross Origin Resource Sharing (CORS) to the API
Bugs Fixed
  • Fixed an error message in UI when attempting to link a Sample that cannot be linked
  • API - Fix issue where API to update Sample metadata fields does not update the fields
  • Fixed error message when creating a Bundle with a name conflict
  • Fixed error message when a user attempts to view Bundle details without permission
  • API - Fix Project Bundle APIs to properly link, retrieve, and unlink entitled Bundles
  • Fixed to block ability to link data with Contributor role and no upload permission on the target project
  • Fixed an issue where the Owner of an externally managed Project cannot add Team members with Viewer or Download permission
  • Fixed field for providing an e-mail address when adding a Team member to a Project to be required
  • Removed temporary directory creation by service connector
  • Fix for configuring service connector to use proxy settings on Linux clients
  • Fixed an issue where data downloaded through Service Connector is incomplete
  • Fixed an issue causing errors in Service Connector when a directory is deleted in the UI but still present on the local machine. Data added to the folder on the local machine will not be uploaded if the corresponding folder is deleted in the UI.
  • Fix to display an error message when saving a Pipeline with invalid fields
  • Fix issue where CWL file/directory inputs are not mounted when provided within nested structures (ie, CommandInputRecordSchema, CommandInputArraySchema)
  • Fix to show error when attempting to run a graphical CWL pipeline with an invalid definition
  • Fix error when attempting to save a graphical CWL Pipeline with a Tool with an output folder and no specified format
  • API - Fix to ignore input JSON field when launching graphical CWL pipelines via API
  • API - Fix ignored default compute type (machine profile) defined in the Pipeline definition when launching a graphical CWL Pipeline via API
  • Fixed Base Query result export file format set to UNKNOWN
  • Fixed Base activity log to show Table associated with log event
  • API - Ignore unknown attributes in request bodies
  • API - Fixed API error responses formatted as HTML instead of JSON in error scenarios
  • API - Fix list analyses API not returning the "items[].analysisStorage" field

April 2022 - ICA v2.4.0, v2.4.1, v2.4.2

Features and Enhancements
  • Samples may only be linked when status is Available
  • Added Data ID to grid view in UI
  • Added ability to select and copy tag values to the clipboard
  • Added an indication of source domain in list of reference data sets when adding new data to Reference Data
  • Updated the service connector client icon
  • Added ability to filter projects by workgroup, including projects where user is a member of a workgroup added to the project
  • Added ability to specify output directory within a project when launching a pipeline
  • Added ability to Favorite Projects and filter the Projects view by Favorites
  • Remove obsolete commented text from auto-generated CWL Tool definitions
  • Scheduler displays the number of files to be loaded into a Base Table
  • Added an indication that Base Table previews display up to 50 columns and 100 rows
  • API - New endpoint to manage SecondaryData on Data objects
  • API - Added support for "mounts" parameter when launching a pipeline to provide input mount paths for file and folder inputs
  • API - Added fields to API error response schema: HTTP Method, User ID, Tenant Name, Tenant ID
  • API – Added new endpoints for retrieving Entitled Bundles and to manage Project Bundles
Bugs Fixed
  • Docker images available via entitled bundles are listed in green text in the Docker Repository view
  • Fixed an issue in OpenAPI spec causing invalid cURL requests in 'Try it Out' functionality for the create upload URL API
  • Fixed an issue that resulted in empty Workgroup listing via the API and when adding Workgroups to a Project Team
  • Fixed an intermittent error when navigating to Bundles view
  • Fixed an issue where Tools available via Bundles are not displayed in the Tool Repository
  • Fixed an issue where example JSON generation for root Metadata Model fails when the root Metadata Model is modified
  • Fixed an issue where incorrect reporting of compute start times resulted in incorrect compute billing
  • Fixed an issue where Nextflow analyses get stuck "In Progress" when output folder is deleted during an ongoing pipeline run
  • Fixed an issue where incorrect casing on the scatter method 'dotproduct' in CWL auto-generated from graphical pipelines results in failures in pipelines with scatter-gather steps
  • CLI - Fixed an issue with startAnalysis APIs no longer accepting the 'Accept' header with value "application/problem+json,application/vnd.illumina.v3+json”
  • API/CLI - Fixed issue where graphical CWL pipelines launched via API/CLI do not use machine profiles (compute types) specified in the pipeline definition
  • Fixed an issue where pipelines within shared bundles could not execute across regions
  • Fixed an issue where exported Base Tables with null values populate a CSV with 'N' values instead of empty values
  • Fixed an error when loading data into Base Tables with an invalid number of columns
  • Fixed an issue causing View Results in Base Query History to return an expiration error
  • Fixed an issue that caused JSON import into Base Tables to fail when sub-field (record type) is used in the Table schema
  • Updated the error message when invalid SQL is used in a Base Query
  • Fixed an issue with JSON import to Base Tables failing when reference data fields are enabled
  • Fixed an issue where updated Base Table fields in the UI do not update the underlying Snowflake Table
  • Fixed an issue resulting in "Unable to create file error" message when creating a Tool in a Bench Workspace
  • Fixed an issue where Bench Workspaces cannot be deleted
  • Fixed an intermittent issue where Bench Workspaces unexpectedly fail in an error state
  • Updated messaging for users without access to Variant Viewer and Cohorts
  • API - Fixed an issue where the API did not accept multiple 'Accept' header values
  • API - Fixed an issue where the create folder upload sessions API returns date and time with incorrect formatting
  • API - Fixed the 500 response on APIs when the user has not yet accessed the UI
  • API - Fixed missing version header in OpenAPI spec for APIs to link a pipeline to a Project or Bundle
  • API - Fixed the 500 response from create project API when the 'storageConfigurationSubfolder' value is invalid
  • API - Fixed a 500 response from the API to link data to a bundle when linking a Folder with child data records
  • API - Fixed an issue where the API to create File data record creates the data record at the root of the project data when a trailing forward slash ('/') is missing after the folder name
  • API - Fixed an issue where API to retrieve pipeline reference sets does not display linked reference data
  • API - Fixed an issue where APIs to initiate analysis perform validation of required properties in request body

March 2022 - ICA v2.3.1

Bugs Fixed
  • Fix issue where pipeline launch is blocked when a single file is used as the input to multiple pipeline inputs

March 2022 - ICA v2.3.0

Features and Enhancements
  • A Base query is now possible with null marker 'nan'
  • A user can search on Secondary Data tab
  • A user is allowed to run a pipeline from a shared project if the Billing mode is set as 'Tenant'
  • Add "application" field to Project API model to indicate whether a project is externally managed.
  • An error is shown when a user tries to build a workspace that is not running.
  • An error is thrown when the user uses the same name of an existing project connector rule in the same project
  • Base query can be exported as a view
  • Bug fix for allowing multiple files with same name but different paths to be passed as input; causes conflict when downloading inputs to pipeline filesystem, one file will be overwritten (random) causing unpredictable results
  • Data filters and columns now also apply to technical tags
  • Externally managed projects are now supported via API
  • Meta data schedule can be created on tenant level
  • New ICA notification for Data/Sample linked
  • Preview of the content of a Base table is available with null marker nan
  • Project connector permissions are aligned with manual linking permissions
  • Remove obsolete "BlueVantage" option from the Source drop down selections during Base schedule creation
  • Revert cwltool upgrade
  • Samples linked via a project connector are visible in the 'receiving' project
  • The 'Type of Files' field in the configuration of Upload Rules in a Connector is changed to 'Assign Format' so that its purpose is clear
  • The project connector rules are available across tenants when the user has Contributor rights
  • The user is able to use the filters 'Type' and 'Status' and sort the files
  • The yaks-scheduler used to check the pod-to-node fit, but will now defer to the kubernetes default-scheduler as the pod-placement authority. The assumption is that when the pod is not scheduled, there are no nodes available, and a node will be provisioned.
  • When a user links a sample from another project to the current project, the data will not be automatically linked to the current project as well. The user should explicitly add the data to the current project as well if the user wants to use it.
  • [Base] When selecting 'Include reference', you will see the record first and when you unfold the record, you will see the fields belonging to the record
  • When trying to populate the Sample Metadata, we now always use the Project Metadata Model if there is no Pipeline Metadata Model.
  • [API] Add ability to filter by region for list projects API
  • [API] Add psToken authorization type to enable creating JWT from psToken via the create token API
  • [Bench] Add ICA_PROJECT_UUID environment variable to the Jupyter Notebook Bench image.
Bugs Fixed
  • Fix 400 error response when launching an analysis with a pipeline from a bundle
  • Fix Base Query results file export set to format UNKNOWN
  • Fix Bench pricing calculation accuracy
  • Fix for Base table export unexpected failure
  • Fix for auto-linking pipeline outputs files to a sample when input files provided are linked to the sample
  • Fix for failed Base table import when imported data contains "nan" (null marker) for a column of type integer
  • Fix for linking Base Query templates to Bundles