Bench workspaces require setting a Docker image to use as the image for the workspace. Platform Core provides a default Docker image with JupyterLab installed.

JupyterLab supports Jupyter Notebook documents (.ipynb). Notebook documents consist of a sequence of cells which may contain executable code, markdown, headers, and raw text.

The JupyterLab Docker image contains the following environment variables:

Included in the default JupyterLab Docker image is a python library with APIs to perform actions in Platform Core, such as add data, launch pipelines, and operate on Base tables. The python library is generated from the using .

The Platform Core Python library API documentation can be found in folder /etc/ica/data/ica_v2_api_docs within the JupyterLab Docker image.

See the for examples on using the Platform Core Python library.

Platform Core provides a tool called Bench for interactive data analysis. This is a sandboxed workspace which runs a docker image with access to the data and pipelines within a project. This workspace runs on the Amazon S3 system and comes with associated processing and provisioning costs. It is therefore best practice to not keep your Bench instances running indefinitely, but stopping them when not in use.

Access

To access Bench, the following requirements must be met:

• Bench is included in all subscriptions.

• The project owner needs to enable Bench for their project.

• The project owner needs to give to Individual users of that project.

After creating a project, go to Projects > your_project > Bench > Workspaces page and click the Enable button. The entitlements you have determine the available resources for your Bench workspaces. If you have multiple entitlements, all the resources of your individual entitlements are taken into account. Once bench is enabled, users with matching have access to the Bench module in that project.

Once Bench has been enabled for your project, the combination of roles and teams settings determines if a user can access Bench.

• Tenant administrators and project owners are always able to access Bench and perform all actions.

• The teams settings page at Projects > your_project > Project Settings > Team determines the role for the user/workgroup.

  • No Access means you have no access to the Bench workspace for that project.

This tutorial shows you how to

• • the execution

The JupyterLab environment is by default configured with 3 additional kernels

• PySpark –

• PySpark –

• PySpark –

When one of the above kernels is selected, the spark context is automatically initialised and can be accessed using the

Bench Workspaces use a FUSE driver to mount project data directly into a workspace file system. There are both read and write capabilities with some limitations on write capabilities that are enforced by the underlying AWS S3 storage.

As a user, you are allowed to do the following actions from Bench (when having the correct user permissions compared to the workspace permissions) or through the CLI:

• Copy project data

• Delete project data

Introduction

The Pipeline Development Kit in Bench makes it easy to create Nextflow pipelines for Platform Core Flow. This kit consists of a number of development tools which are installed in /data/.software (regardless of which Bench image is selected) and provides the following features:

• Import to Bench

  • From public nf-core pipelines

  • From existing Platform Core Flow Nextflow pipelines

• Run in Bench

• Modify and re-run in Bench, providing fast development iterations

• Deploy to Flow

• Launch validation in Flow

• Recommended workspace size: Nf-core Nextflow pipelines typically require 4 or more cores to run.

• The pipeline development tools require

  • Conda which is automatically installed by “pipeline-dev” if conda-miniconda.installer.ica-userspace.sh is present in the image.

Pipeline development tools work best when the following items are defined:

• Nextflow profiles:

  • test profile, specifying inputs appropriate for a validation run

  • docker profile, instructing NextFlow to use Docker

• nextflow_schema.json, as described

Platform Core Flow adds one additional constraint. The output directory out is the only one automatically copied to the Project data when a Platform Core Flow analysis completes. The -outdir parameter recommended by nf-core should therefore be set to--outdir=out when running as a Flow pipeline.

These are installed in /data/.software (which should be in your $PATH), the pipeline-dev script is the front-end to the other pipeline-dev-* tools.

Pipeline-dev fulfils a number of roles:

• Checks that the environment contains the required tools (conda, nextflow, etc) and offers to install them if needed.

• Checks that the fast data mounts are present (/data/mounts/project etc.) – it is useful to check regularly, as they get unmounted when a workspace is stopped and restarted.

• Redirects stdout and stderr to .pipeline-dev.log, with the history of log files kept as .pipeline-dev.log.<log date>.

A pipeline-dev project relies on the following Folder structure, which is auto-generated when using the pipeline-dev import* tools.

• Project base folder

  • nextflow-src: Platform-agnostic Nextflow code, for example the github contents of an nf-core pipeline, or your usual nextflow source code.

    • main.nf

Optional parameters --local / --sge can be added to force the execution on the local workspace node, or on the workspace cluster (when available). Otherwise, the presence of a cluster is automatically detected and used.

The script then launches nextflow. The full nextflow command line is printed and launched.

In case of errors, full logs are saved as .pipeline-dev.log

Nextflow can run processes with and without Docker images. In the context of pipeline development, the pipeline-dev tools assume Docker images are used, in particular during execution with the nextflow --profile docker.

In NextFlow, Docker images can be specified at the process level

• This is done with the container "<image_name:version>" directive, which can be specified.

  • in nextflow config files (preferred method when following the nf-core best practices).

  • or at the start of each process definition.

Resources such as #cpu and memory can be specified as described See or our for details about Nextflow-Docker syntax.

Bench can push/pull/create/modify Docker images, as described in .

This command does the following:

1. Generate the JSON file describing the Platform Core Flow user interface.

   • If ica-flow-config/inputForm.json doesn’t exist: generate it from nextflow-src/nextflow_swagger.json .

2. Generate the JSON file containing the validation launch inputs.

Output Example:

The pipeline name, id and URL are printed out, and if your environment allows, Ctrl+Click/Option+Click/Right click can open the URL in a browser.

Opening the URL of the pipeline and clicking on Start Analysis shows the generated user interface:

The ica-flow-config/launchPayload_inputFormValues.json file generated in the previous step is submitted to Platform Core Flow to start an analysis with the same validation inputs as “nextflow --profile test”.

Output Example:

The analysis name, id and URL are printed out, and if your environment allows, Ctrl+Click/Option+Click/Right click can open the URL in a browser.

Introduction

DRAGEN can run in Bench workspaces

• On FPGA-instances, DRAGEN can run in FPGA mode (hardware-accelerated) or software mode. This can be useful when comparing performance gains by hardware acceleration or to distribute concurrent processes between the FPGA and cpu.

• On non-FPGA instances DRAGEN can only run in software mode.

The DRAGEN command line parameters to specify the location of the licence file are different.

FPGA mode uses

LICENSE_PARAMS="--lic-instance-id-location /opt/dragen-licence/instance-identity.protected --lic-credentials /opt/dragen-licence/instance-identity.protected/dragen-creds.lic"

Software mode uses

DRAGEN software is provided in specific Bench images with names starting with Dragen. For example (available versions may vary):

• Dragen 4.4.1 - Minimal provides DRAGEN 4.4.1 and SSH access

• Dragen 4.4.6 provides DRAGEN 4.4.6, SSH and JupyterLab.

The instance type is selected during workspace creation (Projects > your_project > Bench > Workspaces). The amount of RAM available on the instance is critical. 256GiB RAM is a safe choice to run DRAGEN in production. All FPGA2 instances offer 256GiB or more of RAM.

When running in Software mode, use (348GiB RAM) or (144 GiB RAM) to ensure enough RAM is available for your runs.

Using an fpga2-medium .

Using a standard-xlarge .

Software mode is activated with the DRAGEN --sw-mode parameter.

Running Jobs in a Bench SGE Cluster

Once a cluster is started, the cluster manager can be accessed from the workspace node.

Job resources

Every cluster member has a certain capacity which is determined by the selected Resource model for the cluster member.

The following complex values have been added to the SGE cluster environment and are requestable.

• static_cores (default: 1)

• static_mem (default: 2G)

These values are used to avoid oversubscription of a node which can result in Out-Of-Memory or unresponsiveness. You need to ensure these limits are not exceeded.

To ensure stability of the system, some headroom is deducted from the total node capacity.

These two values are used by the SGE auto scaler when running in dynamic mode. The SGE auto scaler will summarise all pending jobs and their requested resources to determine the scale up/down operation within the defined range.

Cluster members will remain in the cluster for at least 300 seconds. The Auto scaler only executes one scale up/down operation at a time and is stabilised before taking on a new operation.

The operation of the auto scaler can be monitored in the log file /data/logs/sge-scaler.log

Submitting a single job

Submitting a job array

Listing all members of the cluster

listing all jobs in the cluster

Showing the details of a job.

Deleting a job.

Showing the details of an executed job.

SGE command line options and configuration details can be found .

Bench has the ability to handle containers inside a running workspace. This allows you to install and package software more easily as a container image and provides capabilities to pull and run containers inside a workspace.

Bench offers a container runtime as a service in your running workspace. This allows you to do standardized container operations such as pulling in images from public and private registries, build containers at runtime from a Dockerfile, run containers and eventually publish your container to a registry of choice to be used in different Platform Core products such as Platform Core Flow.

The Container Service is accessible from your Bench workspace environment by default.

The container service uses the workspace disk to store any container images you pulled in or created.

To interact with the Container Service, a container remote client CLI is exposed automatically in the /data/.local/bin folder. The Bench workspace environment is preconfigured to automatically detect where the Container Service is made available using environment variables. These environment variables are automatically injected into your environment and are not determined by the Bench Workspace Image.

Introduction

This tutorial shows you how to

• import an existing ICA Flow pipeline with a supporting validation analysis

• run the pipeline in Bench

  • monitor the execution

• Iterative development: and validate in Bench

  • Modify code

  • Modify Docker image contents ( or method)

Make sure you have access in ICA Flow to:

• the pipeline you want to work with

• an analysis exercising this pipeline, preferably with a short execution time, to use as validation test

For this tutorial, the instance size depends on the flow you import, and whether you use a Bench cluster:

• When using a cluster, choose standard-small or standard-medium for the workspace master node

• Otherwise, choose at least standard-large if you re-import a pipeline that originally came from nf-core, as they typically need 4 or more CPUs to run.

• Select the "single user workspace" permissions (aka "Access limited to workspace owner "), which allows us to deploy pipelines

The starting point is the analysis id that is used as pipeline validation test (the pipeline id is obtained from the analysis metadata).

If no --analysis-id is provided, the tool lists all the successfull analyses in the current project and lets the developer pick one.

• If conda and/or nextflow are not installed, pipeline-dev will offer to install them.

• A folder called imported-flow-analysis is created.

• Pipeline Nextflow assets are downloaded into the nextflow-src sub-folder.

The following command runs this test profile. If a Bench cluster is active, it runs on your Bench cluster, otherwise it runs on the main workspace instance:

When a pipeline is running on your Bench cluster, a few commands help to monitor the tasks and cluster. In another terminal, you can use:

• qstat to see the tasks being pending or running

• tail /data/logs/sge-scaler.log.<latest available workspace reboot time> to check if the cluster is scaling up or down (it currently takes 3 to 5 minutes to get a new node)

• The output of the pipeline is in the outdir folder

• Nextflow work files are under the work folder

• Log files are .nextflow.log* and output.log

Nextflow files (located in the nextflow-src folder) are easy to modify. Depending on your environment (ssh access / docker image with JupyterLab or VNC, with and without Visual Studio code), various source code editors can be used.

After modifying the source code, you can run a validation iteration with the same command as before:

Modifying the Docker image is the next step.

Nextflow (and ICA) allow the Docker images to be specified at different places:

• in config files such as nextflow-src/nextflow.config

• in nextflow code files:

grep container may help locate the correct files:

Use case: Update some of the software (mimalloc) by compiling a new version

With the appropriate permissions, you can then "docker login" and "docker push" the new image.

With the appropriate permissions, you can then "docker login" and "docker push" the new image.

Update the nextflow code and/or configs to use the new image

Validate your changes in Bench:

After generating a few ICA-specific files (JSON input specs for Flow launch UI + list of inputs for next step's validation launch), the tool identifies which previous versions of the same pipeline have already been deployed (in ICA Flow, pipeline versioning is done by including the version number in the pipeline name, so that's what is checked here).

It then asks if we want to update the latest version or create a new one.

At the end, the URL of the pipeline is displayed. If you are using a terminal that supports it, Ctrl+click or middle-click can open this URL in your browser.

This launches an analysis in ICA Flow, using the same inputs as the pipeline's "test" profile.

Some of the input files will have been copied to your ICA project to allow the launch to take place. They are stored in the folder /data/project/bench-pipeline-dev/temp-data.

This tutorial shows you how to start a new pipeline from scratch

Bench images are Docker containers tailored to run in Platform Core with the necessary permissions, configuration and resources. For more information of Docker images, please refer to https://docs.docker.com/reference/dockerfile/

The following steps are needed to get your bench image running in Platform Core.

Requirements

You need to have Docker installed in order to build your images.

For your Docker bench image to work in Platform Core, they must run on Linux X86 architecture, have the correct user id and initialization script in the Docker file.

Scripts

The following scripts must be part of your Docker bench image. Please refer to the examples from the Illumina website for more details.

Init Script (Dockerfile)

This script copies the ica_start.sh file which takes care of the Initialization and termination of your workspace to the location in your project from where it can be started by Platform Core when you request to start your workspace.

# Init script invoked at start of a bench workspace
COPY --chmod=0755 --chown=root:root ${FILES_BASE}/ica_start.sh /usr/local/bin/ica_start.sh

User (Dockerfile)

The user settings must be set up so that bench runs with UID 1000.

To do a clean shutdown, you can capture the sigterm which is transmitted 30 seconds before the workspace is terminated.

Once you have Docker installed and completed the configuration of your Docker files, you can build your bench image.

1. Open the command prompt on your machine.

2. Navigate to the root folder of your Docker files.

3. Execute docker build -f Dockerfile -t mybenchimage:0.0.1 . with mybenchimage being the name you want to give to your image and 0.0.1 replaced with the version number which you want your bench image to be. For more information on this command, see

1. Open Platform Core and log in.

2. Go to Projects > your_project > Data.

3. For small Docker images, upload the docker image file which you generated in the previous step. For large Docker images use the to better performance and reliability to import the Docker image.

4. Select the uploaded image file and perform Manage > Change Format.

1. Navigate to Projects > your_project > Bench > Workspaces.

2. Create a new workspace with + Create Workspace or edit an existing workspace.

3. Fill in the bench workspace details according to .

4. Save your changes.

Once your bench image has been started, you can access it via console, web or both, depending on your configuration.

• Web access (HTTP) is done from either Projects > your_project > Bench > Workspaces > your_Workspace > Access tab or from the link provided at provided in your running workspace at Projects > your_project > Bench > Workspaces > your_Workspace > Details tab > Access section.

• Console access (SSH) is performed from your command prompt by going to the path provided in your running workspace at Projects > your_project > Bench > Workspaces > your_Workspace > Details tab > Access section.

To execute , your workspace needs a way to run them such as the inclusion of an SSH daemon, be it integrated into your web access image or into your console access. There is no need to download the workspace command-line interface, you can run it from within the workspace.

• The bench image will be instantiated as a container which will be forcedly started as user with UID 1000 and GID 100.

• You cannot elevate your permissions in a running workspace.

Only the following folders are writeable:

• /data

• /tmp

All other folders are mounted as read-only.

For inbound access, the following ports on the container are publicly exposed, depending on the selection made at startup.

• Web: TCP/8888

• Console: TCP/2222

For outbound access, a workspace can be started in two modes:

• Public: Access to public IP’s is allowed using TCP protocol.

• Restricted: Access to list of URLs are allowed.

At runtime, the following Bench-specific environment variables are made available to the workspace instantiated from the Bench image.

Following files and folders will be provided to the workspace and made accessible for reading at runtime.

At runtime, Platform Core-related software will automatically be made available at /data/.software in read-only mode.

New versions of Platform Core software will be made available after a restart of your workspace.

When a bench workspace is instantiated from your selected bench image, the following script is invoked: /usr/local/bin/ica_start.sh

This script is the main process in your running workspace and cannot run to completion as it will stop the workspace and instantiate a restart (see ).

This script can be used to invoke other scripts.

When you stop a workspace, a TERM signal is sent to the main process in your bench workspace. You can trap this signal to handle the stop gracefully (see and shut down child processes of the main process. The workspace will be forcedly shut down after 30 seconds if your main process hasn’t stopped within the given period.

If you get the error "docker buildx build" requires exactly 1 argument when trying to build your docker image, then a possible cause is missing the last . of the command.

When you stop the workspace when users are still actively using it, they will receive a message showing a Server Connection Error.

The main concept in Bench is the Workspace. A workspace is an instance of a Docker image that runs the framework which is defined in the image (for example JupyterLab, R Studio). In this workspace, you can write and run code and graphically represent data. You can use API calls to access data, analyses, Base tables and queries in the platform. Via the command line, R-packages, tools, libraries, IGV browsers, widgets, etc. can be installed.

You can create multiple workspaces within a project and each workspace runs on an individual node and is available in different resource sizes. Each node has local storage capacity, where files and results can be temporarily stored and exported from to be permanently stored in a Project. The size of the storage capacity can range from 1GB – 16TB.

Create Workspace

1. Click Projects > Your_Project > Bench > Workspaces > + Create Workspace

2. Complete the following fields and save the changes.

The workspace can be edited on the workspace Details tab once it is stopped. The changes will be applied when the workspace is restarted.

When Access limited to workspace owner is selected, only the workspace owner can access the workspace. Everything created in that workspace will belong to the workspace owner.

• Bench administrators are able to create, edit and delete workspaces and start and stop workspaces. If their permissions match or exceed those of the workspace, they can also access the workspace contents.

• Contributors are able to start and stop workspaces and if their permissions match or exceed those of the workspace, they can also access the workspace contents.

The setting determines if someone is an administrator or contributor, while the dedicated indicate what the workspace itself can and cannot do within your project. For this reason, the users need to meet or exceed the required permissions to enter this workspace and use it.

The permissions that a Bench workspace can receive are the following:

• Upload rights

• Download rights (required)

• Project (No Access - Dataprovider - Viewer - Contributor)

• Flow (No Access - Viewer - Contributor)

Based on these permissions, you will be able to upload or download data to your Platform Core project (upload and download rights) and will be allowed to take actions in the Project, Flow and Base modules related to the granted permission.

If you encounter issues when uploading/downloading data in a workspace, the security settings for that workspace may be set to not allow uploads and downloads. This can result in RequestError: send request failed and read: connection reset by peer. This is by design in restricted workspaces and thus limits data access to your project via /data/project to prevent the extraction of large amounts of (proprietary) data.

To delete a workspace with the GUI, go to Projects > your_project > Bench > Workspaces > your_workspace and click “Delete”. Note that the delete option is only available when the workspace is stopped.

The workspace will not be accessible anymore, nor will it be shown in the list of workspaces. The content of it will be deleted so if there is any information that should be kept, you can either put it in a docker image which you can use to start from next time, or export it using the API.

The workspace is not always accessible. It needs to be started before it can be used. From the moment a workspace is Running, a node with a specific capacity is assigned to this workspace. From that moment on, you can start working in your workspace.

To start the workspace, follow the next steps:

1. Go to Projects > your_project > Bench > Workspaces > your_workspace > Details

2. Click on Start Workspace button

3. On the top of the details tab, the status changes to “Starting”. When you click on the >_Access tab, the message “The workspace is starting” appears.

4. Wait until the status is “Running” and the “Access” tab can be opened. This can take some time because the necessary resources have to be provisioned.

You can refresh the workspace status by selecting the round refresh symbol at the top right.

Once a workspace is running, it can be manually stopped or it will be automatically shut down after the amount of time configured in the field. Even with automatic shutdown, it is still best practice to stop your workspace run when you no longer need it to save costs.

GUI

When you exit a workspace, you can choose to stop the workspace or keep it running. Keeping the workspace running means that it will continue to use resources and incur associated costs. To stop the workspace, select stop in the displayed dialog. You can also stop a workspace by opening it and selecting stop at the top right.

General

Stopping the workspace will stop the notebook, but will not delete local data. Content will no longer be accessible and no actions can be performed until it is restarted. Any work that has been saved will stay stored.

The project/tenant administrator can enter and stop workspaces for their project/tenant even if they did not start those workspaces at Projects > your_project > Bench > Workspaces > your_workspace > Details. Be careful not to stop workspaces that are processing data. For security reasons, a log entry is added when a project/tenant administrator enters and exits a workspace.

You can see who is using a workspace in the workspace list view.

Once the Workspace is running, the default applications are loaded. These are defined by the start script of the docker image.

The docker images provided by Illumina will load JupyterLab by default. It also contains Tutorial notebooks that can help you get started. Opening a new terminal can be done via the Launcher, + button above the folder structure.

To ensure that packages (and other objects, including data) are permanently installed on a Bench image, a new Bench image needs to be created, using the BUILD option in Bench. A new image can only be derived from an existing one. The build process uses the DOCKERFILE method, where an existing image is the starting point for the new Docker Image (The FROM directive), and any new or updated packages are additive (they are added as new layers to the existing Docker file).

In order to create a derived image, open up the image that you would like to use as the basis and select the Build tab.

• Name: By default, this is the same name as the original image and it is recommended to change the name.

• Version: Required field which can by any value.

• Description: The description for your docker image (for example, indicating which apps it contains).

• Code: The Docker file commands must be provided in this section.

The first 4 lines of the Docker file must NOT be edited. It is not possible to start a docker file with a different FROM directive. The main docker file commands are RUN and COPY. More information on them is available in the official Docker documentation.

Once all information is present, click the Build button. Note that the build process can take a while. Once building has completed, the docker image will be available on the Data page within the Project. If the build has failed, the log will be displayed here and the log file will be in the Data list.

From within the workspace it is possible to create a tool from the Docker image.

1. Click the Manage > Create CWL Tool button in the top right corner of the workspace.

2. Give the tool a name.

3. Replace the description of the tool to describe what it does.

4. Add a version number for the tool.

The building can take a while. When it has completed, the tool will be available in the Tool Repository.

To export data from your workspace to your local machine, it is best practice to move the data in your workspace to the /data/project/ folder so that it becomes available in your project under projects > your_project > Data. Although this storage is slow, it offers read and write access and access to the content from within ICA.

• For fast read-only access, link folders with the workspace-ctl data create-mount --mode read-only.

• For fast read/write access, link which are visible, but whose contents are not accessible from Platform Core. Use the workspace-ctl data create-mount --mode read-write to do so. You can not have fast read-write access to indexed folders as the indexing mechanism on those would deteriorate the performance.

Every workspace you start has a read-only /data/.software/ folder which contains the icav2 command-line interface (and readme file).

The last tab of the workspace is the activity tab. On this tab all actions performed in the workspace are shown. For example, the creation of the workspace, starting or stopping of the workspace,etc. The activities are shown with their date, the user that performed the action and the description of the action. This page can be used to check how long the workspace has run.

In the general Activity page of the project, there is also a Bench activity tab. This shows all activities performed in all workspaces within the project, even when the workspace has been deleted. The Activity tab in the workspace only shows the action performed in that workspace. The information shown is the same as per workspace, except that here the workspace in which the action is performed is listed as well.

Managing a Bench cluster

Introduction

Workspaces can have their own dedicated cluster which consists of a number of nodes. First the workspace node, which is used for interacting with the cluster, is started. Once the workspace node is started, the workspace cluster can be started.

The cluster consists of 2 components

• The manager node which orchestrates the workload across the members.

• Anywhere between 0 and up to maximum 50 member nodes.

Clusters can run in two modes.

• Static - A static cluster has a manager node and a static number of members. At start-up of the cluster, the system ensures the predefined number of members are added to the cluster. These nodes will keep running as long as the entire cluster runs. The system will not automatically remove or add nodes depending on the job load. This gives the fastest resource availability, but at additional cost as unused nodes stay active, waiting for work.

• Dynamic - A dynamic cluster has a manager node and a dynamic number of workers up to a predefined maximum (with a hard limit of 50). Based on the job load the system will scale the number of members up or down. This saves resources as only as much worker nodes as needed to perform the work are being used.

You manage Bench Clusters via the Platform Core UI in Projects > your_project > Bench > Workspaces > your_workspace > Details.

The following settings can be defined for a bench cluster:

Once the is started, the cluster can be started at Projects > your_project > Bench > Workspaces > your_workspace > Details and the cluster can be stopped without stopping the workspace. Stopping the workspace will also stop all clusters in that workspace.

Data in a bench workspace can be divided into three groups:

• Workspace data is accessible in read/write mode and can be accessed from all workspace components (workspace node, cluster manager node, cluster member nodes ) at /data. The size of the workspace data is defined at the creation of the workspace but can be increased when editing a workspace in the Platform Core UI. This is persistent storage and data remains when a workspace is shut down.

• Project data can be accessed from all workspace components at /data/project. Every component will have their own dedicated mount to the project. Depending on the project data permissions you will be able to access it in either Read-Only or Read-Write mode.

All mounts occur in /data/mounts/, see and .

Managing these mounts is done via the workspace cli /data/.local/bin/workspace-ctl in the workspace. Every node will have his dedicated mount.

For fast data access, bench offers a mount solution to expose project data on every component in the workspace. This mount provides read-only access to a given location in the project data and is optimized for high read throughput per single file with concurrent access to files. It will try to utilise the full bandwidth capacity of the node.

All mounts occur in path /data/mounts/

For fast read-only access, link folders with the workspace-ctl data create-mount --mode read-only.

The following is a list of available bench CLI commands and their options. You can execute these from within a running bench workspace.

Please refer to the examples from the for more details.