Import New Samples
Import New Samples
Platform Core Cohorts can pull any molecular data available in a Platform Core project, as well as additional sample- and subject-level metadata information such as demographics, biometrics, sequencing technology, phenotypes, and diseases.
To import a new data set, select Import Jobs from the left navigation tab underneath Cohorts, and click the Import Files button. The Import Files button is also available under the Data Sets left navigation item.
Confirm that the project shown is the Platform Core project that contains the molecular data you would like to add to Platform Core Cohorts.
Choose a data type among
Germline variants
Somatic mutations
RNAseq
GWAS
Choose a new study name by selecting the radio button Create new study and entering a Study Name.
To add new data to an existing Study, select the radio button Select from list of studies and select an existing Study Name from the dropdown.
To add data to existing records or add new records, select Job Type, Append.
Append does not wipe out any data ingested previously and can be used to ingest the molecular data in an incremental manner.
To replace data, select Job Type, Replace. If you are ingesting data again, use the Replace job type.
Enter an optional Study description.
Select the metadata model (default: Cohorts; alternatively, select OMOP version 5.4 if your data is formatted that way.)
Select the genome build your molecular data is aligned to (default: GRCh38/hg38)
For RNAseq, specify whether you want to run differential expression (see below) or only upload raw TPM.
Click Next.
Navigate to VCFs located in the Project Data.
Select each single-sample VCF or multi-sample VCF to ingest. For GWAS, select CSV files produced by Regenie.
As an alernative to selecting individual files, you can also opt to select a folder instead. Toggle the radio button on Step 2 from "Select files" to "Select folder".
This option is currently only available for germline variant ingestion: any combination of small variants, structural variation, and/or copy number variants.
Platform Core Cohorts will scan the selected folder and all sub-folders for any VCF files or JSON files and try to match them against the Sample ID column in the metadata TSV file in Step 3.
Files not matching sample IDs will be ignored; allowed file extensions for VCF files after the sample ID are: *.vcf.gz, *.hard-filtered.vcf.gz, *.cnv.vcf.gz, and *.sv.vcf.gz .
Files not matching sample IDs will be ignored; allowed file extensions for JSON files after the sample ID are: .json,.json.gz, *.json.bgz, *.json.gzip.
Click Next.
Navigate to the metadata (phenotype) data tsv in the project Data.
Select the TSV file or files for ingestion.
Click Finish.
Variant file formats
Platform Core Cohorts supports VCF files formatted according to VCF v4.2 and v4.3 specifications. VCF files require at least one of the following header rows to identify the genome build:
##reference=file://... --- needs to contain a reference to hg38/GRCh38 in the file path or name (numerical value is sufficient)
##contig=<ID=chr1,length=248956422> --- for hg38/GRCh38
##DRAGENCommandLine= ... --ht-reference
Platform Core Cohorts accepts VCFs aligned to hg38/GRCh38 and hg19/GRCh37. If your data uses hg19/GRCh37 coordinates, Cohorts will convert these to hg38/GRCh38 during the ingestion process [see Reference 1]. Harmonizing data to one genome build facilitates searches across different private, shared, and public projects when building and analyzing a cohort. If your data contains a mixture of samples mapped to hg38 and hg19, please ingest these in separate batches, as each import job into Cohorts is limited to one genome build.
As an alternative to VCFs, Platform Core Cohorts accepts the JSON output of Illumina Nirvana for hg38/GRCh38-aligned data for small germline variants and somatic mutations, copy number variations and other structural variants.
RNAseq file format
Platform Core Cohorts can process gene- and transcript-level quantification files produced by the Illumina DRAGEN RNA pipeline. The file naming convention needs to match .quant.genes.sf for genes and .quant.sf for transcript-level TPM.
Please also see the online documentation for the Illumina DRAGEN RNA Pipeline for more information on output file formats.
GWAS file format
Platform Core Cohorts currently supports upload of SNV-level GWAS results produced by Regenie and saved as CSV files.
Metadata and File Types
Field
Description
Project name
The Platform Core project for your cohort analysis (cannot be changed.)
Study name
Create or select a study. Each study represents a subset of data within the project.
Description
Short description of the data set (optional).
Job type
Append: Appends values to any existing values. If a field supports only a single value, the value is replaced.
Replace: Overwrites existing values with the values in the uploaded file.
Subject metadata files
Subject metadata file(s) in tab-delimited format. For Append and Replace job types, the following fields are required and cannot be changed: - Sample identifier - Sample display name - Subject identifier - Subject display name - Sex
As an alternative to Platform Core Cohorts metadata file format, you can provide files formatted according to the OMOP common data model 5.4. Cohorts currently ingests data for these OMOP 5.4 tables, formatted as tab-delimited files:
PERSON (mandatory),
CONCEPT (mandatory if any of the following is provided),
CONDITION_OCCURRENCE (optional),
DRUG_EXPOSURE (optional), and
PROCEDURE_OCCURRENCE (optional.)
Additional files such as measurement and observation will be supported in a subsequent release of Cohorts.
References
[1] VcfMapper: https://stratus-documentation-us-east-1-public.s3.amazonaws.com/downloads/cohorts/main_vcfmapper.py
[2] crossMap: https://crossmap.sourceforge.net/
[3] liftOver: https://genome.ucsc.edu/cgi-bin/hgLiftOver
[4] Chain files: ftp://ftp.ensembl.org/pub/assembly_mapping/homo_sapiens/
Last updated
Was this helpful?