Data
The Data section shows files and folders stored in the project.
ICA supports UTF-8 characters in file and folder names for data. Following recommended practices, users are encouraged to follow additional guidelines detailed below. (For more information about recommended approaches to file naming that can be applicable across platforms, please refer to this AWS S3 documentation.)
Characters generally considered "safe":
- Alphanumeric characters
- 0-9
- a-z
- A-Z
- Special characters
- Exclamation point
! - Hyphen
- - Underscore
_ - Period
. - Asterisk
* - Single quote
' - Open parenthesis
( - Closed parenthesis
)
Length of file name (minus prefixes and delimiters) generally should be limited to 32 characters.
Data privacy should be carefully considered when adding data in ICA, either through storage configurations (ie, AWS S3) or ICA data upload. Be aware that when adding data from cloud storage providers by creating a storage configuration, ICA will provide access to the data. In general, users should ensure the storage configuration source settings are correct and uploads do not include unintended data in order to avoid unintentional privacy breaches. More guidance can be found in the ICA Security and Compliance section.
On the Projects > your_project > Data page, you can view information on or preview files.
To view file details:
- 1.Select a file to view by clicking on the filename, this will show the file details
To view file contents:
- 1.Select a file to view by clicking on the filename, this will show the file details
- 2.Select View. This will preview the file.
You can also preview the file content by selecting the checkbox at the begining of the line and then selecting View from the top menu.
To hyperlink directly to data, use the following syntax:
https://<ServerURL>/ica/link/project/<ProjectID>/data/<FolderID> and https://<ServerURL>/ica/link/project/<ProjectID>/analysis/<AnalysisID> .To link directly to data, use the following syntax:
https://<ServerURL>/ica/link/project/<ProjectID>/data/<FolderID> and https://<ServerURL>/ica/link/project/<ProjectID>/analysis/<AnalysisID>.Variable | Location |
|---|---|
ServerURL | see browser addres bar |
projectID | At YourProject > Details > URN > urn:ilmn:ica:project:ProjectID#MyProject |
FolderID | At YourProject > Data > folder > folder details > ID |
AnalysisID | At YourProject > Flow > Analyses > YourAnalysis > ID |
Normal permission checks still apply with these links. If you try to follow a link to data to which you do not have access, you will be returned to the main project screen or login screen, depending on your permissions.
Uploading data to the platform makes it available for consumption by analysis workflows and tools. There are multiple methods to upload data.
Uploads via the UI are limited to 5TiB
Use the following instructions to upload data manually via the drag-and-drop interface in the platform UI.
- 1.Go to Projects > your_project > Data.
- 2.To add data, use one of the following methods. Make sure the Illumina Connected Analytics tab is open in the browser while data uploads.
- Drag a file from your system into Choose a file or drag it here box.
- Select the Choose a file or drag it here box, and then choose a file. Select Open to upload the file.
Your file or files are added to the Data page when upload completes.
data-0
Some small files can be downloaded directly from within the UI. Things like .txt and .csv files can be viewed by clicking on the filename when in the project Data section. On the View tab, the file can be viewed directly (larger files may take some time to load) and the Download button will allow you to download the file directly from the UI.
download-ui-1
You can trigger an asynchronous download via service connector using the Schedule for Download button with one or more files selected.
- 1.Select a file or files to download.
- 2.Select Schedule for Download.
- 3.Select a connector, and then select Schedule for Download.
You can view the progress of the download or stop the download on the Activity page for the project.
The data records contained in a project can be exported in CSV, JSON, and excel format.
- 1.Select one or more files to export.
- 2.Select Export.
- 3.Select the following export options:
- To export only the selected file, select the Selected rows as the Rows to export option. To export all files on the page, select Current page.
- To export only the columns present for the file, select the Visible columns as the Columns to export option.
- 4.Select the export file type
Uploaded files are automatically added with the standard storage tier. You can use files in the standard tier in your analysis.
To manually archive or delete files, do as follows:
- 1.Select the checkbox next to the file or files to delete or archive.
- 2.Select Manage, and then select one of the following options:
- Archive — Move the file or files to long-term storage.
- Unarchive — Return the file or files from long-term storage. Unarchiving can take up to 48 hours, regardless of file size. Unarchived files can be used in analysis.
- Delete — Remove the file completely.
When attempting concurrent archiving or unarchiving of the same file, a message will inform you to wait for the currently running (un)archiving to finish first.
When Secondary Data is added to a data record, those secondary data records are mounted in the same parent folder path as the primary data file when the primary data file is provided as an input to a pipeline. Secondary data is intended to work with the CWL secondaryFiles feature. This is commonly used with genomic data such as BAM files with companion BAM index files (refer to https://www.ncbi.nlm.nih.gov/tools/gbench/tutorial6/ for an example). 
Linking a folder links only the data within the folder at the time the link is created. Data added within the source folder after the link is created will not be automatically linked to the destination project.
You can perform analysis on data from other projects by linking data from that project.
- 1.Select Projects > your_project > Data > Manage, and then select Link Data.
- 2.To view data by project, select Add filter, and then select Owning Project. If you only know to which project the data is linked, you can choose linked project to filter on.
- 3.Select the checkbox next to the file or files to add.
- 4.Select Select Data.
Tip: if you have selected multiple owning projects, you can add the owning project column to see which project owns the data.
Your files are added to the Data page. To view the linked data file, select Add filter, and then select Links.
If you link a folder instead of individual files, a warning is displayed indicating that, depending on the size of the folder, linking may take considerable time. The linking process will run in the background and the progress can be monitored on the Projects > your_project > activity > Batch Jobs screen.
- To see more details, double-click the batch job.
- To see how many individual files are already linked, double-click the item.
To unlink the data, select the folder containing the files or the individual files themselves (limited to 100 at a time) and select Manage > Unlink Data. As during linking a folder, when unlinking, the progress can be monitored at Projects > your_project > activity > Batch Jobs.
Last modified 2mo ago