Use Coarrays

Intel® Fortran supports coarray programs that run using shared memory on a multicore or multiprocessor system. Coarray programs can also be built to run using distributed memory across a Linux* or Windows* cluster.

Coarrays are only supported on 64-bit architectures. For more details, see the product system requirements in the Release Notes.

Coarrays, a data sharing concept standardized in Fortran 2008 and extended in Fortran 2018 and 2023, enable parallel processing using multiple copies of a single program. Each copy, called an image, has ordinary local variables and also shared variables called coarrays or covariables.

A covariable, which can be either an array or a scalar, is a variable whose storage spans all the images of the team that was current when the covariable was established. In this Partitioned Global Address Space (PGAS) model, each image can access its own piece of a covariable as a local variable and can access those pieces that live on other images using coindices, which are enclosed in square brackets.

For more information on how to write programs using coarrays, see books on the Fortran 2008 and later language versions, or the ISO Fortran 2008 and later standards versions.

Use Coarray Program Syntax

The additional syntax required by Fortran 2008 coarrays includes:

The following Fortran 2018 coarray extensions are also supported:

Use the Coarray Compiler Options

You must specify the -coarray (Linux) or /Qcoarray (Windows) compiler option (hereafter referred to as [Q]coarray) to enable the compiler to recognize coarray syntax. If you do not specify this compiler option, a program that uses coarray syntax or features produces a compile-time error.

Only one [Q]coarray option is valid on the command line. If multiple coarray compiler options are specified, the last one specified is used. An exception to this rule is the [Q]coarray compiler option using keyword single; if specified, this option takes precedence regardless of where it appears on the command line.

The following describes the option keywords:

No special procedure is necessary to run a program that uses coarrays. You simply run the executable file.

The underlying parallelization implementation uses the Intel® MPI Library. Installation of the compiler automatically installs the necessary runtime libraries to run on shared memory. Products supporting clusters will also install the necessary runtime libraries to run on distributed memory. Use of coarray applications with any other Intel® MPI Library implementation, or with OpenMP*, is not supported.

Note

The conda package for the Intel® Fortran Compiler runtime no longer has a runtime dependency on the Intel® MPI Library, which is needed to enable coarrays. If you maintain a conda package that has a runtime dependency on the Intel Fortran Compiler runtime and your application uses the Intel® MPI Library, you need to explicitly add the impi_rt conda package for the Intel® MPI Library to the list of runtime dependencies in your project's meta.yaml file.

By default, the number of images created is equal to the number of execution units on the current system. You can override this by specifying a number using the [Q]coarray-num-images compiler option on the command line that compiles the main program. You can also specify the number of images at execution time in the environment variable FOR_COARRAY_NUM_IMAGES.

Use a Configuration File

Using a configuration file by specifying compiler option [Q]coarray-config-file is useful if you are looking for more control of your image placement, especially in multi-node systems.

The main reason is if you want to take advantage of Intel® MPI Library features in the coarray environment. To do so, specify the command line segments used by mpiexec -config filename in a file named filename and pass that file name to the Intel® MPI Library using the [Q]coarray-config-file compiler option.

If the [Q]coarray-num-images compiler option also appears on the command line, it will be overridden by what is in the configuration file.

Rules for using an Intel® MPI Library configuration files are as follows:

Use Configuration Environment Variables

Intel Fortran uses Intel® MPI Library as the transport layer for the coarray feature. Intel® MPI Library can be tuned to a particular usage pattern with environment variables.

Intel Fortran has chosen to set some Intel® MPI Library control variables to values that are good for most users and many patterns of coarray usage. However, you may want to experiment with other variables that Intel® Fortran does not set. They are not set by Intel Fortran because they may reduce performance with other usage patterns or because they may cause errors when used with older versions of Intel® MPI Library.

Applications running on shared memory with Intel® MPI Library version 2019 Update 5 or greater may benefit from setting the following variable to shm.

I_MPI_FABRICS

Note

When the above environment variables are set on Linux systems, there may be hangs on Red Hat 7.2 and Ubuntu because they cause increased use of shared memory. Therefore, please note that you may need to increase the size of /dev/shm to avoid Linux bus error (SIGBUS).

Applications which over-subscribe (have more coarray images running than there are actual processors in the machine) may benefit from setting the following variable to 1:

I_MPI_WAIT_MODE

Applications which over-subscribe a great deal (more than four images per processor) may benefit from setting the following variable to 3:

I_MPI_THREAD_YIELD

This is just an introduction to using configuration environment variables. There are many other environment variables that you can use to tune Intel® MPI Library. For more information, see:

Examples

Linux

Windows

See Also