The main concept in Bench is the Workspace. Multiple workspaces can be created within the project. A workspace is an instance of a Docker image that runs the framework defined in the image, such as JupyterLab, R Studio, etc. It is in this workspace that code can be written and run. Via API calls, data, analyses, Base tables and queries in the platform can be accessed and manipulated. Via the command line, R-packages, tools, libraries, IGV browsers, widgets, etc. can be installed. Data can also be graphically presented.
Each workspace runs on an individual node and is available in different resource sizes. In addition, each node will have a local storage capacity, where files and results can be stored temporarily, before they are stored permanently in a Project. The size of the storage capacity can range from 10GB – 64TB.
In order to use Bench, a workspace needs to be created. This workspace defines which docker image will be used, which node and storage size will be used, etc.
If this is the first time you are using a workspace in a Project, click
Enableto create new Bench Workspaces.
To create a new workspace:
- 1.Click + New Workspace
- 2.Complete the following fields:
- Name: (required) must be a unique name.
- Docker image: (required) The list of docker images includes base images from ICA and images uploaded to the docker repository for that domain.
- Storage size: (required) Represents the size of the storage available on the workspace. A storage from 10GB to 64TB can be provided.
- Resource model: (required) Size of the machine on which the workspace will run and whether or not the machine should contain a Graphics Processing Unit (GPU).
- Access mode: (required) Type of access to the internet for this workspace
- Open: Internet access is allowed
- Restricted: Creates a workspace with no internet access. Access to the ICA Project Data is still available in this mode.
example.com www.example.com subdomain.example.com sub-domain.example.com sub_domain.example.com example.co.uk subdomain.example.co.uk sub-domain.example.co.uk
- Whitelisted URLs: Specify URLs that are allowed in a restricted workspace. Separate URLS with a new line. Only domains and subdomains in the specified URL will be allowed.
- URLs must comply with the following:
- URLs can be between 1 and 263 characters including dot (
- URLs can begin with a leading dot (
- Domain and Sub-domains:
- Can include alphanumeric characters (Letters A-Z and digits 0-9). Case insensitive.
- Can contain hyphens (
-) and underscores (
_), but not as a first or last character.
- Length between 1 and 63 characters.
- Dot (
.) must be placed after a domain or sub-domain.
- Example URLs accepted:
pypi.org files.pythonhosted.org repo.anaconda.com conda.anaconda.org github.com cran.r-project.org bioconductor.org www.npmjs.com mvnrepository.com
- Example data science specific whitelist compatible with restricted Bench workspaces. Note there are two required URLs to allow for Python pip installs:
- Workspace Permissions: Users need to have the permissions matching what you set here to run the workspace and your workspace can operate with these permissions.
- Description: A place to provide additional information about the workspace
- 3.Click “Save”
The workspace can be edited afterwards when it is stopped, on the Details tab within the workspace. The changes will be applied when the workspace is restarted.
Workspace permissions allow users to use a Bench workspace without the need of exposing their own API key. Instead, an API key belonging to the Bench workspace is used through an environment variable. Only bench administrators will be able to create a workspace. The dedicated permissions on workspace level will indicate what it can and cannot do within your project and are the minimal permissions that any team member needs to enter this workspace and to use it, except for the workspace administrator who can always access the workspace.
If one of your permissions is not high enough as bench contributor, you will see the following message "You are not allowed to use this workspace as your user permissions are not sufficient compared to the permissions of this workspace".
The permissions that a Bench workspace can receive are the following:
- Upload rights
- Download rights (required)
- Project (No Access - Dataprovider - Viewer - Contributor)
- Flow (No Access - Viewer - Contributor)
- Base (No Access - Viewer - Contributor)
Based on these permissions, you will be able to upload or download data to your ICA project (upload and download rights) and will be allowed to take actions in the Project, Flow and Base modules related to the granted permission.
In case of an old workspace, you have the possibility to enable these workspace permissions. If enabled, you will see the same behavior as described above. If not, the workspace will continue working as before.
To delete a workspace, go to Projects > your_project > Bench > Workspaces > your_workspace and click “Delete”. Note that the delete option is only available when the workspace is stopped.
The workspace will not be accessible anymore, nor will it be shown in the list of workspaces. The content of it will be deleted so if there is any information that should be kept, you can either put it in a docker image, that you then use as a basis to start from next time, or export it using the API.
The workspace is not always accessible; it needs to be started before it can be used. From the moment a workspace is Running, a node with a specific capacity is assigned to this workspace. From that moment on, you can start working in your workspace. As long as the workspace is running, the resources provided for this workspace will be charged.
To start the workspace, follow the next steps:
- 1.Go to Projects > your_project > Bench > Workspaces > your_workspace > Details
- 2.Click on Start button
- 3.On the top of the details tab, the status changes to “Starting”. When you click on the >_Access tab, the message “The workspace is starting” appears.
- 4.Wait until the status is “Running” and the “Access” tab can be opened. This can take some time because the necessary resources have to be provisioned.
When you exit a workspace, a reminder is shown that keeping workspaces running will keep incurring costs. To stop a workspace, select stop in the displayed dialog or stop the workspace by opening it and selecting stop at the top right.
NOTE: The delete option is only visible for Administrators.
The project/tenant administrator can enter and stop workspaces for their project/tenant even if they did not start those workspaces at Projects > your_project > Bench > Workspaces > your_workspace > Details. Be careful not to stop workspaces that are processing data. For security reasons, a log entry is added when a project/tenant administrator enters and exits a workspace.
To see if somebody is actively working in a workspace, go to Projects > your_project > Bench > Workspaces and look at the workspace. If there is a user in the workspace, they will be shown on the bottom right of the workspace.
Alternatively, you can see who is using a workspace in the workspace list view or in the workspace itself, near the workspace actions at the top.
When you stop a workspace, that workspace is paused. Content will no longer be accessible and no actions can be performed until it is started again. Any work that has been saved will stay stored and storage will continue to be charged until the workspace is deleted.
Once the Workspace is running, the Access loads the default applications. These are defined by the start script of the docker image.
The docker images provided by Illumina will load JupyterLab by default. It also contains Tutorial notebooks that can help you get started. Opening a new terminal can be done via the Launcher, + button above the folder structure.
To ensure that packages (and other objects, including data) are permanently installed on a Bench image, a new Bench image needs to be created, using the BUILD option in Bench. A new image can only be derived from an existing one. The build process uses the DOCKERFILE method, where an existing image is the starting point for the new Docker Image (The FROM directive), and any new or updated packages are additive (they are added as new layers to the existing Docker file).
NOTE: The Dockerfile commands are all run as ROOT, so it is possible to delete or interfere with an image in such a way that the image is no longer running correctly. The image does not have access to any underlying parts of the platform so will not be able to harm the platform, but inoperable Bench images will have to be deleted or corrected.
In order to create a derived image, open up the image that you would like to use as the basis and select the Build tab.
- Name: By default, the same name as the original image; it is recommended to change the name.
- Version: Required field, can by any value.
- Description: The description for your docker image (for example, indicating which apps it contains).
- Code: The Docker file commands must be provided in this section.
The first 4 lines of the Docker file must NOT be edited. It is not possible to start a docker file with a different FROM directive. Themain docker file commands are RUN and COPY. More information on them is available in the official Docker documentation.
Once all information is present, click the Build button. Note that the build process can take a while. Once building has completed, the docker image will be available on the Data page within the Project. If the build has failed, the log will be displayed here and the log file will be in the Data list.
From within the workspace it is possible to create a docker image and a tool from it at the same time.
- 1.Click the + Create Tool button in the top right corner of the workspace.
- 2.Give the tool a name.
- 3.Replace the description of the tool to describe what it does.
- 4.Add a version number for the tool.
- 5.Click the Image tab.
- Here the IMAGE that accompanies the TOOL will be created.
- Change the name for the image.
- Change the version.
- Replace the description to describe what the image does.
- Below the line where it says “#Add your commands below.” write the code necessary for running this docker image.
- 6.Click the General Tool tab. This tab and all next tabs will look familiar from Flow. Enter the information required for the tool in each of the tabs. For more detailed instruction check out the Tool creation section in the Flow documentation.
- 7.Click the Save button in the upper, right-hand corner to start the build process.
The building can take a while. When it has completed, the tool will be available in the Tool Repository.
The tab mechanism allows showing a web application running on the same Bench instance to be displayed in a new window.
To create a new tab, click the (+) button next to the Activity tab. This opens a dialog box to indicate
- 1.Name: Can be any name, spaces are allowed.
- 2.URL: The app URL without the leading forward slash, but with the following slash.
Clicking Save will save the link for the tab and clicking the tab will open the new application. Clicking on the Access tab will open up the default application again.
The last tab of the workspace is the activity tab. On this tab all actions performed in the workspace are shown. For example, the creation of the workspace, starting or pausing of the workspace,etc. The activities are shown with their date, the user that performed the action and the description of the action. This page can be used to check how long the workspace has run.
In the general Activity page of the project, there is also a Bench activity tab. This shows all activities performed in all workspaces within the project, even when the workspace has been deleted. The Activity tab in the workspace only shows the action performed in that workspace. The information shown is the same as per workspace, except that here the workspace in which the action is performed is listed as well.