Nextflow training

Enter this command in your terminal:

Or, if you prefer curl:

Then ensure that the downloaded binary is executable:

AND put the nextflow executable into your $PATH (e.g. ~/usr/local/bin or ~/bin/)

Ensure you have Docker Desktop running on your machine. Download Docker here.

You can view the training material here: training.seqera.io/

To download the material use git clone github.com/seqeralabs/nf-training-public.git.

Then cd into the nf-training directory.

Check the correct installation of nextflow by running the following command:

This should show the current version, system and runtime.

To run Gitpod:

gitpod.io/#https://github.com/seqeralabs/nf-training-public

(which is our GitHub repository URL, prefixed with gitpod.io/#).

Once you have signed in, Gitpod should load (skip prebuild if asked).

You should now see something similar to the following:

The sidebar allows you to customize your Gitpod environment and perform basic tasks (copy, paste, open files, search, git, etc.). Click the Explorer button to see which files are in this repository.

The terminal allows you to run all the programs in the repository. For example, both nextflow and docker are installed and can be executed.

The main window allows you to view and edit files. Clicking on a file in the explorer will open it within the main window. You should also see the nf-training material browser (training.seqera.io/).

To test that the environment is working correctly, type the following into the terminal:

This should come up with the Nextflow version and runtime information:

See gitpod.io for more details.

You can reopen an environment from gitpod.io/workspaces. Find your previous environment in the list, then select the ellipsis (three dots icon) and select Open.

If you have saved the URL for your previous Gitpod environment, you can simply open it your browser to open the previous environment.

Alternatively, you can start a new workspace by following the Gitpod URL: gitpod.io/#https://github.com/seqeralabs/nf-training-public

If you have lost your environment, you can find the main scripts used in this tutorial in the nf-training directory to resume with a new environment.

To save any file from the explorer panel, right-click the file and select Download.

The training course can be accessed in your browser from training.seqera.io/.

In practice, a Nextflow pipeline is made by joining together different processes. Each process can be written in any scripting language that can be executed by the Linux platform (Bash, Perl, Ruby, Python, etc.).

Processes are executed independently and are isolated from each other, i.e. they do not share a common (writable) state. The only way they can communicate is via asynchronous first-in, first-out (FIFO) queues, called channels in Nextflow.

Any process can define one or more channels as an input and output. The interaction between these processes, and ultimately the pipeline execution flow itself, is implicitly defined by these input and output declarations.

While a process defines what command or script has to be executed, the executor determines how that script is actually run in the target platform.

If not otherwise specified, processes are executed on the local computer. The local executor is very useful for pipeline development and testing purposes, but for real world computational pipelines, a High Performance Computing (HPC) or cloud platform is often required.

In other words, Nextflow provides an abstraction between the pipeline’s functional logic and the underlying execution system (or runtime). Thus, it is possible to write a pipeline which runs seamlessly on your computer, a cluster, or the cloud, without being modified. You simply define the target execution platform in the configuration file.

Nextflow implements a declarative DSL that simplifies the writing of complex data analysis workflows as an extension of a general purpose programming language.

This approach makes Nextflow flexible — it provides the benefits of a concise DSL for the handling of recurrent use cases with ease, and the flexibility and power of a general purpose programming language to handle corner cases, in the same computing environment. This would be difficult to implement using a purely declarative approach.

In practical terms, Nextflow scripting is an extension of the Groovy programming language which, in turn, is a super-set of the Java programming language. Groovy can be thought of as "Python for Java", in that it simplifies the writing of code and is more approachable.

Now copy the above example into your favourite text editor and save it to a file named hello.nf.

Execute the script by entering the following command in your terminal:

The output will look similar to the text shown below:

The standard output shows (line by line):

It’s worth noting that the process CONVERTTOUPPER is executed in parallel, so there’s no guarantee that the instance processing the first split (the chunk 'Hello ') will be executed before the one processing the second split (the chunk 'world!').

Thus, it is perfectly possible that your final result will be printed out in a different order:

To better understand how Nextflow is dealing with the data in this pipeline, below is a DAG-like figure to visualise all the inputs, outputs, channels and processes:

A container can be run using the following command:

Try for example the following publicly available container (if you have docker installed):

The pull command allows you to download a Docker image without running it. For example:

The above command downloads a Debian Linux image. You can check it exists by using:

Launching a BASH shell in the container allows you to operate in an interactive mode in the containerized operating system. For example:

Once launched, you will notice that it is running as root (!). Use the usual commands to navigate the file system. This is useful to check if the expected programs are present within a container.

To exit from the container, stop the BASH session with the exit command.

Docker images are created by using a so-called Dockerfile, which is a simple text file containing a list of commands to assemble and configure the image with the software packages required.

Here, you will create a Docker image containing cowsay and the Salmon tool.

Use your favorite editor (e.g., vim or nano) to create a file named Dockerfile and copy the following content:

Build the Dockerfile image by using the following command:

Where "my-image" is the user-specified name tag for the Dockerfile, present in the current directory.

When it completes, verify that the image has been created by listing all available images:

You can try your new container by running this command:

Add the Salmon package to the Docker image by adding the following snippet to the Dockerfile:

Save the file and build the image again with the same command as before:

You will notice that it creates a new Docker image with the same name but with a different image ID.

Check that Salmon is running correctly in the container as shown below:

You can even launch a container in an interactive mode by using the following command:

Use the exit command to terminate the interactive session.

Create a genome index file by running Salmon in the container.

Try to run Salmon in the container with the following command:

The above command fails because Salmon cannot access the input file.

This happens because the container runs in a completely separate file system and it cannot access the hosting file system by default.

You will need to use the --volume command-line option to mount the input file(s) e.g.

Or set a folder you want to mount as an environmental variable, called DATA:

Now check the content of the transcript-index folder by entering the command:

Publish your container in the Docker Hub to share it with other people.

Create an account on the hub.docker.com website. Then from your shell terminal run the following command, entering the user name and password you specified when registering in the Hub:

Tag the image with your Docker user name account:

Finally push it to the Docker Hub:

After that anyone will be able to download it by using the command:

Note how after a pull and push operation, Docker prints the container digest number e.g.

This is a unique and immutable identifier that can be used to reference a container image in a univocally manner. For example:

The simplest way to run a Nextflow script with a Docker image is using the -with-docker command-line option:

As seen in the last section, you can also configure the Nextflow config file (nextflow.config) to select which container to use instead of having to specify it as a command-line argument every time.

Singularity images are created using a Singularity file in a similar manner to Docker but using a different syntax.

Once you have saved the Singularity file. You can create the image with these commands:

Note: the build command requires sudo permissions. A common workaround consists of building the image on a local workstation and then deploying it in the cluster by copying the image file.

Once done, you can run your container with the following command

By using the shell command you can enter in the container in interactive mode. For example:

Once in the container instance run the following commands:

An easier way to create a Singularity container without requiring sudo permission and boosting the containers interoperability is to import a Docker container image by pulling it directly from a Docker registry. For example:

The above command automatically downloads the Debian Docker image and converts it to a Singularity image in the current directory with the name debian-jessie.simg.

Nextflow allows the transparent usage of Singularity containers as easy as with Docker.

Simply enable the use of the Singularity engine in place of Docker in the Nextflow configuration file by using the -with-singularity command-line option:

As before, the Singularity container can also be provided in the Nextflow config file. We’ll see how to do this later.

The authors of Singularity, SyLabs have their own repository of Singularity containers.

In the same way that we can push Docker images to Docker Hub, we can upload Singularity images to the Singularity Library.

A Conda environment is defined using a YAML file, which lists the required software packages. The first thing you need to do is to initiate conda for shell interaction, and then open a new terminal by running bash.

Then write your YAML file. For example:

Given the recipe file, the environment is created using the command shown below:

You can check the environment was created successfully with the command shown below:

This should look something like this:

To enable the environment you can use the activate command:

Nextflow is able to manage the activation of a Conda environment when its directory is specified using the -with-conda option (using the same path shown in the list function. For example:

This makes easier to manage different environments for the processes in the workflow script.

See the docs for details.

Another way to build conda-like environments is through a Dockerfile and micromamba.

micromamba is a fast and robust package for building small conda-based environments.

This saves having to build a conda environment each time you want to use it (as outlined in previous sections).

To do this, you simply require a Dockerfile and you use micromamba to install the packages. However, a good practice is to have a YAML recipe file like in the previous section, so we’ll do it here too.

Then, we can write our Dockerfile with micromamba installing the packages from this recipe file.

The above Dockerfile takes the parent image 'mambaorg/micromamba', then installs a conda environment using micromamba, and installs salmon, fastqc and multiqc.

Try executing the RNA-Seq pipeline from earlier (script7.nf). Start by building your own micromamba Dockerfile (from above), save it to your docker hub repo, and direct Nextflow to run from this container (changing your nextflow.config).

A queue channel is an asynchronous unidirectional FIFO queue that connects two processes or operators.

A queue channel is implicitly created by process output definitions or using channel factories such as Channel.of or Channel.fromPath.

Try the following snippets:

Try to execute this snippet. You can do that by creating a new .nf file or by editing an already existing .nf file.

A value channel (a.k.a. singleton channel) by definition is bound to a single value and it can be read unlimited times without consuming its contents. A value channel is created using the value factory method or by operators returning a single value, such as first, last, collect, count, min, max, reduce, and sum.

To better understand the difference between value and queue channels, save the snippet below as example.nf.

When you run the script, it prints only 2, as you can see below:.

To understand why, we can inspect the queue channel and running Nextflow with DSL1 gives us a more explicit comprehension of what is behind the curtains.

Then:

This will print:

We have the value 1 as the single element of our queue channel and a poison pill, which will tell the process that there’s nothing left to be consumed. That’s why we only have one output for the example above, which is 2. Let’s inspect a value channel now.

Then:

This will print:

There is no poison pill, and that’s why we get a different output with the code below, where ch2 is turned into a value channel through the first operator.

Besides, in many situations, Nextflow will implicitly convert variables to value channels when they are used in a process invocation. For example, when you invoke a process with a pipeline parameter (params.example) which has a string value, it is automatically cast into a value channel.

The value factory method is used to create a value channel. An optional not null argument can be specified to bind the channel to a specific value. For example:

The factory Channel.of allows the creation of a queue channel with the values specified as arguments.

The first line in this example creates a variable ch which holds a channel object. This channel emits the values specified as a parameter in the of method. Thus the second line will print the following:

The method Channel.of works in a similar manner to Channel.from (which is now deprecated), fixing some inconsistent behaviors of the latter and provides better handling when specifying a range of values. For example, the following works with a range from 1 to 23 :

The method Channel.fromList creates a channel emitting the elements provided by a list object specified as an argument:

The fromPath factory method creates a queue channel emitting one or more files matching the specified glob pattern.

This example creates a channel and emits as many items as there are files with a csv extension in the /data/meta folder. Each element is a file object implementing the Path interface.

Learn more about the glob patterns syntax at this link.

Use the Channel.fromPath method to create a channel emitting all files with the suffix .fq in the data/ggal/ directory and any subdirectory, in addition to hidden files. Then print the file names.

The fromFilePairs method creates a channel emitting the file pairs matching a glob pattern provided by the user. The matching files are emitted as tuples, in which the first element is the grouping key of the matching pair and the second element is the list of files (sorted in lexicographical order).

It will produce an output similar to the following:

Use the fromFilePairs method to create a channel emitting all pairs of fastq read in the data/ggal/ directory and print them. Then use the flat:true option and compare the output with the previous execution.

The Channel.fromSRA method makes it possible to query the NCBI SRA archive and returns a channel emitting the FASTQ files matching the specified selection criteria.

The query can be project ID(s) or accession number(s) supported by the NCBI ESearch API.

For example, the following snippet will print the contents of an NCBI project ID:

This should print:

Multiple accession IDs can be specified using a list object:

It’s straightforward to use this channel as an input using the usual Nextflow syntax. The code below creates a channel containing two samples from a public SRA study and runs FASTQC on the resulting files. See:

If you want to run the pipeline above and do not have fastqc installed in your machine, don’t forget what you learned in the previous section. Run this pipeline with -with-docker biocontainers/fastqc:v0.11.5, for example.

The splitText operator allows you to split multi-line strings or text file items, emitted by a source channel into chunks containing n lines, which will be emitted by the resulting channel. See:

You can define the number of lines in each chunk by using the parameter by, as shown in the following example:

An optional closure can be specified in order to transform the text chunks produced by the operator. The following example shows how to split text files into chunks of 10 lines and transform them into capital letters:

You can also make counts for each line:

Finally, you can also use the operator on plain files (outside of the channel context):

The splitCsv operator allows you to parse text items emitted by a channel, that are CSV formatted.

It then splits them into records or groups them as a list of records with a specified length.

In the simplest case, just apply the splitCsv operator to a channel emitting a CSV formatted text files or text entries. For example, to view only the first and fourth columns:

When the CSV begins with a header line defining the column names, you can specify the parameter header: true which allows you to reference each value by its column name, as shown in the following example:

Alternatively, you can provide custom header names by specifying a list of strings in the header parameter as shown below:

You can also process multiple csv files at the same time:

Finally, you can also operate on csv files outside the channel context:

Try inputting fastq reads into the RNA-Seq workflow from earlier using .splitCSV.

Parsing tsv files works in a similar way, simply add the sep:'\t' option in the splitCsv context:

Try using the tab separation technique on the file "data/meta/regions.tsv", but print just the first column, and remove the header.

We can also easily parse the JSON file format using the following groovy schema:

This can also be used as a way to parse YAML files:

The best way to store parser scripts is to keep them in a Nextflow module file.

See the following Nextflow script:

For this script to work, a module file called parsers.nf needs to be created and stored in a modules folder in the current directory.

The parsers.nf file should contain the parseJsonFile function.

Nextflow will use this as a custom function within the workflow scope.

Script parameters (params) can be defined dynamically using variable values. For example:

It can be tricky to write a script uses many Bash variables. One possible alternative is to use a script string delimited by single-quote characters

However, this blocks the usage of Nextflow variables in the command script.

Another alternative is to use a shell statement instead of script and use a different syntax for Nextflow variables, e.g., !{..}. This allows the use of both Nextflow and Bash variables in the same script.

The process script can also be defined in a completely dynamic manner using an if statement or any other expression for evaluating a string value. For example:

The val qualifier allows you to receive data of any type as input. It can be accessed in the process script by using the specified input name, as shown in the following example:

In the above example the process is executed three times, each time a value is received from the channel num and used to process the script. Thus, it results in an output similar to the one shown below:

The path qualifier allows the handling of file values in the process execution context. This means that Nextflow will stage it in the process execution directory, and it can be accessed in the script by using the name specified in the input declaration.

The input file name can also be defined using a variable reference as shown below:

The same syntax is also able to handle more than one input file in the same execution and only requires changing the channel composition.

Write a script that creates a channel containing all read files matching the pattern data/ggal/*_1.fq followed by a process that concatenates them into a single file and prints the first 20 lines.

A key feature of processes is the ability to handle inputs from multiple channels. However, it’s important to understand how channel contents and their semantics affect the execution of a process.

Consider the following example:

Both channels emit three values, therefore the process is executed three times, each time with a different pair:

What is happening is that the process waits until there’s a complete input configuration, i.e., it receives an input value from all the channels declared as input.

When this condition is verified, it consumes the input values coming from the respective channels, spawns a task execution, then repeats the same logic until one or more channels have no more content.

This means channel values are consumed serially one after another and the first empty channel causes the process execution to stop, even if there are other values in other channels.

So what happens when channels do not have the same cardinality (i.e., they emit a different number of elements)?

For example:

In the above example, the process is only executed twice because the process stops when a channel has no more data to be processed.

However, what happens if you replace value x with a value channel?

Compare the previous example with the following one :

This is because value channels can be consumed multiple times and do not affect process termination.

Write a process that is executed for each read file matching the pattern data/ggal/*_1.fq and use the same data/ggal/transcriptome.fa in each execution.

The each qualifier allows you to repeat the execution of a process for each item in a collection every time new data is received. For example:

In the above example, every time a file of sequences is received as an input by the process, it executes three tasks, each running a different alignment method set as a mode variable. This is useful when you need to repeat the same task for a given set of parameters.

Extend the previous example so a task is executed for each read file matching the pattern data/ggal/*_1.fq and repeat the same task with both salmon and kallisto.

The val qualifier specifies a defined value in the script context. Values are frequently defined in the input and/or output declaration blocks, as shown in the following example:

The path qualifier specifies one or more files produced by the process into the specified channel as an output.

In the above example the process RANDOMNUM creates a file named result.txt containing a random number.

Since a file parameter using the same name is declared in the output block, the file is sent over the receiver_ch channel when the task is complete. A downstream process declaring the same channel as input will be able to receive it.

When an output file name contains a wildcard character (* or ?) it is interpreted as a glob path matcher. This allows us to capture multiple files into a list object and output them as a sole emission. For example:

Prints the following:

Some caveats on glob pattern behavior:

Remove the flatMap operator and see out the output change. The documentation for the flatMap operator is available at this link.

When an output file name needs to be expressed dynamically, it is possible to define it using a dynamic string that references values defined in the input declaration block or in the script global context. For example:

In the above example, each time the process is executed an alignment file is produced whose name depends on the actual value of the x input.

So far we have seen how to declare multiple input and output channels that can handle one value at a time. However, Nextflow can also handle a tuple of values.

The input and output declarations for tuples must be declared with a tuple qualifier followed by the definition of each element in the tuple.

Modify the script of the previous exercise so that the bam file is named as the given sample_id.

Given each process is being executed in separate temporary work/ folder (e.g., work/f1/850698…​; work/g3/239712…​; etc.), we may want to save important, non-intermediary, and/or final files in a results folder.

To store our workflow result files, we need to explicitly mark them using the directive publishDir in the process that’s creating the files. For example:

The above example will copy all blast script files created by the BLASTSEQ task into the directory path my-results.

You can use more than one publishDir to keep different outputs in separate directories. For example:

The above example will create an output structure in the directory my-results, that contains a separate sub-directory for each given sample ID, each containing the folders counts and outlooks.

The view operator prints the items emitted by a channel to the console standard output, appending a new line character to each item. For example:

It prints:

An optional closure parameter can be specified to customize how items are printed. For example:

It prints:

The map operator applies a function of your choosing to every item emitted by a channel and returns the items obtained as a new channel. The function applied is called the mapping function and is expressed with a closure as shown in the example below:

A map can associate a generic tuple to each element and can contain any data.

Use fromPath to create a channel emitting the fastq files matching the pattern data/ggal/*.fq, then use map to return a pair containing the file name and the path itself, and finally, use view to print the resulting channel.

The mix operator combines the items emitted by two (or more) channels into a single channel.

The flatten operator transforms a channel in such a way that every tuple is flattened so that each entry is emitted as a sole element by the resulting channel.

The above snippet prints:

The collect operator collects all of the items emitted by a channel in a list and returns the object as a sole emission.

It prints a single value:

The groupTuple operator collects tuples (or lists) of values emitted by the source channel, grouping the elements that share the same key. Finally, it emits a new tuple object for each distinct key collected.

Try the following example:

It shows:

This operator is useful to process a group together with all the elements that share a common property or grouping key.

Use fromPath to create a channel emitting all of the files in the folder data/meta/, then use a map to associate the baseName prefix to each file. Finally, group all files that have the same common prefix.

The join operator creates a channel that joins together the items emitted by two channels with a matching key. The key is defined, by default, as the first element in each item emitted.

The resulting channel emits:

The branch operator allows you to forward the items emitted by a source channel to one or more output channels.

The selection criterion is defined by specifying a closure that provides one or more boolean expressions, each of which is identified by a unique label. For the first expression that evaluates to a true value, the item is bound to a named channel as the label identifier. For example:

Components defined in the module script can be imported into other Nextflow scripts using the include statement. This allows you to store these components in a separate file(s) so that they can be re-used in multiple workflows.

Using the hello.nf example, we can achieve this by:

Create a modules.nf file with the previously defined processes from hello.nf. Then remove these processes from hello.nf and add the include definitions shown above.

We now have modularized processes which makes the code more reusable and cleaner to read.

If a Nextflow module script contains multiple process definitions they can also be imported using a single include statement as shown in the example below:

When including a module component it is possible to specify a name alias using the as declaration. This allows the inclusion and the invocation of the same component multiple times using different names:

Save the previous snippet as hello.2.nf, and predict the output to screen.

Another way to deal with outputs in the workflow scope is to use pipes |.

Try changing the workflow script to the snippet below:

Here we use a pipe which passed the output as a channel to the next process.

A workflow component can declare one or more input channels using the take statement. For example:

The input for the workflow can then be specified as an argument:

A workflow can declare one or more output channels using the emit statement. For example:

As a result, we can use the my_pipeline.out notation to access the outputs of my_pipeline in the invoking workflow.

We can also declare named outputs within the emit block.

The result of the above snippet can then be accessed using my_pipeline.out.my_data.

Within a main.nf script we also can have multiple workflows. In which case we may want to call a specific workflow when running the code. For this we use the entrypoint call -entry <workflow_name>.

The following snippet has two named workflows (my_pipeline_one and my_pipeline_two):

You can choose which pipeline runs by using the entry flag:

A module script can define one or more parameters or custom functions using the same syntax as with any other Nextflow script. Using the minimal examples below:

Running main.nf should print:

As highlighted above, the script will print Hola mundo! instead of Hello world! because parameters are inherited from the including context.

The addParams option can be used to extend the module parameters without affecting the external scope. For example:

Executing the main script above should print:

A Nextflow configuration file is a simple text file containing a set of properties defined using the syntax:

Configuration properties can be used as variables in the configuration file itself, by using the usual $propertyName or ${expression} syntax.

Configuration files use the same conventions for comments used in the Nextflow script:

Configuration settings can be organized in different scopes by dot prefixing the property names with a scope identifier or grouping the properties in the same scope using the curly brackets notation. This is shown in the following example:

The scope params allows the definition of workflow parameters that override the values defined in the main workflow script.

This is useful to consolidate one or more execution parameters in a separate file.

Save the first snippet as nextflow.config and the second one as params.nf. Then run:

Execute is again specifying the foo parameter on the command line:

Compare the result of the two executions.

The env scope allows the definition of one or more variables that will be exported into the environment where the workflow tasks will be executed.

Save the above snippet as a file named my-env.config. Then save the snippet below in a file named foo.nf:

Finally, execute the following command:

process directives allow the specification of settings for the task execution such as cpus, memory, container, and other resources in the pipeline script.

This is useful when prototyping a small workflow script.

However, it’s always a good practice to decouple the workflow execution logic from the process configuration settings, i.e. it’s strongly suggested to define the process settings in the workflow configuration file instead of the workflow script.

The process configuration scope allows the setting of any process directives in the Nextflow configuration file. For example:

The above config snippet defines the cpus, memory and container directives for all processes in your workflow script.

The process selector can be used to apply the configuration to a specific process or group of processes (discussed later).

The syntax for setting process directives in the configuration file requires = (i.e. assignment operator), whereas it should not be used when setting the process directives within the workflow script.

This is especially important when you want to define a config setting using a dynamic expression using a closure. For example:

Directives that require more than one value, e.g. pod, in the configuration file need to be expressed as a map object.

Finally, directives that are to be repeated in the process definition, in the configuration files need to be defined as a list object. For example:

The container image to be used for the process execution can be specified in the nextflow.config file:

The use of unique "SHA256" docker image IDs guarantees that the image content does not change over time, for example:

To run a workflow execution with Singularity, a container image file path is required in the Nextflow config file using the container directive:

The following protocols are supported:

The above configuration instructs Nextflow to use the Singularity engine to run your script processes. The container is pulled from the Docker registry and cached in the current directory to be used for further runs.

Alternatively, if you have a Singularity image file, its absolute path location can be specified as the container name either using the -with-singularity option or the process.container setting in the config file.

Try to run the script as shown below, changing the nextflow.config file to the one above using singularity:

The use of a Conda environment can also be provided in the configuration file by adding the following setting in the nextflow.config file:

You can specify the path of an existing Conda environment as either directory or the path of Conda environment YAML file.

Use the scope process to define the resource requirements for all processes in your workflow applications. For example:

In real-world applications, different tasks need different amounts of computing resources. It is possible to define the resources for a specific task using the select withName: followed by the process name:

Run the RNA-Seq script (script7.nf) from earlier, but specify that the quantification process requires 2 CPUs and 5 GB of memory, within the nextflow.config file.

When a workflow application is composed of many processes, listing all of the process names and choosing resources for each of them in the configuration file can be difficult.

A better strategy consists of annotating the processes with a label directive. Then specify the resources in the configuration file used for all processes having the same label.

The workflow script:

The configuration file:

Containers can be set for each process in your workflow. You can define their containers in a config file as shown below:

Read more about config process selectors at this link.

Create an account and login into Tower.

1. Create a new token

You can access your tokens from the Settings drop-down menu :

2. Name your token

3. Save your token safely

4. Export your token

Once your token has been created, open a terminal and type:

Where eyxxxxxxxxxxxxxxxQ1ZTE= is the token you have just created.

To submit a pipeline to a Workspace using the Nextflow command-line tool, add the workspace ID to your environment. For example:

The workspace ID can be found on the organization’s Workspaces overview page.

5. Run Nextflow with tower

Run your Nextflow workflows as usual with the addition of the -with-tower command:

You will see and be able to monitor your Nextflow jobs in Tower.

To configure and execute Nextflow jobs in Cloud environments, visit the Compute environments section.

Run the RNA-Seq script7.nf using the -with-tower flag, after correctly completing the token settings outlined above.

To run using the GUI, there are three main steps:

1. Create an account and login into Tower, available free of charge, at tower.nf.

2. Create and configure a new compute environment.

3. Start launching pipelines.

To learn more about using the Tower API, visit the API section in this documentation.

You can create your own organization and participant workspace by following the docs at tower.

Tower allows the creation of multiple organizations, each of which can contain multiple workspaces with shared users and resources. This allows any organization to customize and organize the usage of resources while maintaining an access control layer for users associated with a workspace.

Any user can be added or removed from a particular organization or a workspace and can be allocated a specific access role within that workspace.

The Teams feature provides a way for organizations to group various users and participants together into teams. For example, workflow-developers or analysts, and apply access control to all the users within this team collectively.

For further information, please refer to the User Management section.