Links

Notifications

Notifications are used to subscribe to events in the platform and trigger the delivery of a message to an external delivery target. The following table describes available system events to subscribe to:
Description
Code
Details
Payload
Analysis failure
ICA_EXEC_001
Emitted when an analysis fails
Analysis
Analysis success
ICA_EXEC_002
Emitted when an analysis succeeds
Analysis
Analysis status change
ICA_EXEC_028
Emitted when an state transition on an analysis occurs
Analysis
Base Job failure
ICA_BASE_001
Emitted when a Base job fails
BaseJob
Base Job success
ICA_BASE_002
Emitted when a Base job succeeds
BaseJob
Data transfer success
ICA_DATA_002
Emitted when a data transfer is marked as Succeeded
DataTransfer
Data transfer stalled
ICA_DATA_025
Emitted when data transfer hasn't progressed in the past 2 minutes
DataTransfer
Data <action>
ICA_DATA_100
Emitted when data transfer status changes. Subscribing to this serves as a wildcard for for all data actions and covers actions that have no separate code.
ProjectData
Data linked to project
ICA_DATA_104
Emitted when a file is linked to a project
ProjectData
Data deleted
ICA_DATA_106
Emitted when data is deleted
ProjectData
Data created
ICA_DATA_107
Emitted when data is created
ProjectData
Data uploaded
ICA_DATA_108
Emitted when data is uploaded
ProjectData
Data updated
ICA_DATA_109
Emitted when data is updated
ProjectData
Data archived
ICA_DATA_110
Emitted when data is archived
ProjectData
Sample completed
ICA_SMP_002
Emitted when a sample is marked as completed
ProjectSample
Sample linked to a project
ICA_SMP_003
Emitted when a sample is linked to a project
ProjectSample
Workflow session start
ICA_WFS_001
Emitted when workflow is started
WorkflowSession
Workflow session failure
ICA_WFS_002
Emitted when workflow fails
WorkflowSession
Workflow session success
ICA_WFS_003
Emitted when workflow succeeds
WorkflowSession

Delivery Targets

Event notifications can be delivered to the following delivery targets:
Delivery Target
Description
Value
Mail
E-mail delivery
E-mail Address
Sqs
AWS SQS Queue
AWS SQS Queue URL
Sns
AWS SNS Topic
AWS SNS Topic ARN
Http
Webhook (POST request)
URL

Amazon Resource Policy Settings

In order to allow the platform to deliver events to Amazon SQS or SNS delivery targets, a cross-account policy needs to be added to the target Amazon service.
{
"Version":"2012-10-17",
"Statement":[
{
"Effect":"Allow",
"Principal":{
"AWS":"arn:aws:iam::<platform_aws_account>:root"
},
"Action":"<action>",
"Resource": "<arn>"
}
]
}
Substitute the variables in the example above according to the table below.
Variable
Description
platform_aws_account
The platform AWS account ID: 079623148045
action
For SNS use SNS:Publish. For SQS, use SQS:SendMessage
arn
The Amazon Resource Name (ARN) of the target SNS topic or SQS queue
See examples for setting policies in Amazon SQS and Amazon SNS

Amazon SNS Topic

To create a subscription to deliver events to an Amazon SNS topic, one can use either GUI or API endpoints.
To create a subscription via GUI select Projects > your_project > Project Settings > Notifications and then select New ICA subscription. Then select an event from the dropdown menu, insert optional filter, select the channel type (SNS), and then insert the ARN from the target SNS topic.
Settings for SNS subscription
To create a subscription via API use the endpoint /api/notificationChannel to create a channel and then /api/projects/{projectId}/notificationSubscriptions to create a notification subscription.

Amazon SQS Queue

To create a subscription to deliver events to an Amazon SQS queue, one can use either GUI or API endpoints.
To create a subscription via GUI select Notifications menu within a project and then select New ICA subscription. Then select an event from the dropdown menu, insert optional filter, select the channel type (SQS), and then insert the ARN from the target SQS topic.
To create a subscription via API use the endpoint /api/notificationChannel to create a channel and then /api/projects/{projectId}/notificationSubscriptions to create a notification subscription.
Messages delivered to AWS SQS contain the following event body attributes:
Attribute
Description
correlationId
GUID used to identify the event
timestamp
Date when the event was sent
eventCode
Event code of the event
description
Description of the event
payload
Event payload
The following example is a Data Updated event payload sent to an AWS SQS delivery target (condensed for readability):
{
"correlationId": "2471d3e2-f3b9-434c-ae83-c7c7d3dcb4e0",
"timestamp": "2022-10-06T07:51:09.128Z",
"eventCode": "ICA_DATA_100",
"description": "Data updates",
"payload": {
"id": "fil.8f6f9511d70e4036c60908daa70ea21c",
...
}
}

Filtering

Notification subscriptions will trigger for all events matching the configured event type. A filter may be configured on a subscription to limit the matching strategy to only those event payloads which match the filter.
The filter expressions leverage the JsonPath library for describing the matching pattern to be applied to event payloads. The filter must be in the format [?(<expression>)].

Example

The Analysis Success event delivers a JSON event payload matching the Analysis data model (as output from the API to retrieve a project analysis).
{
"id": "0c2ed19d-9452-4258-809b-0d676EXAMPLE",
"timeCreated": "2021-09-20T12:23:18Z",
"timeModified": "2021-09-20T12:43:02Z",
"ownerId": "15d51d71-b8a1-4b38-9e3d-74cdfEXAMPLE",
"tenantId": "022c9367-8fde-48fe-b129-741a4EXAMPLE",
"reference": "210920-1-CopyToolDev-9d78096d-35f4-47c9-b9b6-e0cbcEXAMPLE",
"userReference": "210920-1",
"pipeline": {
"id": "20261676-59ac-4ea0-97bd-8a684EXAMPLE",
"timeCreated": "2021-08-25T01:49:41Z",
"timeModified": "2021-08-25T01:49:41Z",
"ownerId": "15d51d71-b8a1-4b38-9e3d-74cdfEXAMPLE",
"tenantId": "022c9367-8fde-48fe-b129-741a4EXAMPLE",
"code": "CopyToolDev",
"description": "CopyToolDev",
"language": "CWL",
"pipelineTags": {
"technicalTags": ["Demo"]
}
},
"status": "SUCCEEDED",
"startDate": "2021-09-20T12:23:21Z",
"endDate": "2021-09-20T12:43:00Z",
"summary": "",
"finishedSteps": 0,
"totalSteps": 1,
"tags": {
"technicalTags": [],
"userTags": [],
"referenceTags": []
}
}
The below examples demonstrate various filters operating on the above event payload:
  • Filter on a pipeline, with a code that starts with ‘Copy’. You’ll need a regex expression for this:
    [?($.pipeline.code =~ /Copy.*/)]
  • Filter on status (note that the Analysis success event is only emitted when the analysis is successful):
    [?($.status == 'SUCCEEDED')]
  • Filter on pipeline, having a technical tag “Demo":
    [?('Demo' in $.pipeline.pipelineTags.technicalTags)]
  • Combination of multiple expressions using &&. It's best practice to surround each individual expression with parentheses:
    [?(($.pipeline.code =~ /Copy.*/) && $.status == 'SUCCEEDED')]

Custom Events

Custom events enable triggering notification subscriptions using event types beyond the system-defined event types. When creating a custom subscription, a custom event code may be specified to use within the project. Events may then be sent to the specified event code using a POST API with the request body specifying the event payload.
Currently custom events can only be defined using the API. In order to create a custom event for your project please follow the steps below:
  1. 1.
    Create a new custom event POST {ICA_URL}/ica/rest/api/projects/{projectId}/customEvents a. Your custom event code must be 1-20 characters long, e.g. 'ICA_CUSTOM_123'. b. That event code will be used to reference that custom event type.
  2. 2.
    Create a new notification channel POST {ICA_URL}/ica/rest/api/notificationChannels a. If there is already a notification channel created with the desired configuration within the same project, it is also possible to get the existing channel ID using the call GET {ICA_URL}/ica/rest/api/notificationChannels.
  3. 3.
    Create a notification subscription POST {ICA_URL}/ica/rest/api/projects/{projectId}/customNotificationSubscriptions. a. Use the event code created in step 1. b. Use the channel ID from step 2.
Once the steps above have been completed successfully, the call from the first step POST {ICA_URL}/ica/rest/api/projects/{projectId}/customEvents could be reused with the same event code to continue sending events through the same channel and subscription.
Following is a sample Python function used inside an ICA pipeline to post custom events for each failed metric:
def post_custom_event(metric_name: str, metric_value: str, threshold: str, sample_name: str):
api_url = f"{ICA_HOST}/api/projects/{PROJECT_ID}/customEvents"
headers = {
"Content-Type": "application/vnd.illumina.v3+json",
"accept": "application/vnd.illumina.v3+json",
"X-API-Key": f"{ICA_API_KEY}"
}
content = {\"code\": \"ICA_CUSTOM_123\", \"content\": { \"metric_name\": metric_name, \"metric_value\": metric_value,\"threshold\": threshold, \"sample_name\": sample_name}}
json_data = json.dumps(content)
response = requests.post(api_url, data=json_data, headers=headers)
if response.status_code != 204:
print(f"[EVENT-ERROR] Could not post metric failure event for the metric {metric_name} (sample {sample_name}).")