NCL Home> Application examples> Data Analysis || Data files for some examples

Example pages containing: tips | resources | functions/procedures

Task parallelism

This applications page will show various ways that task parallelism can be accomplished within NCL. In this case, task parallelism refers to the ability to call multiple NCL scripts at once, or the ability to call multiple NCL function or procedure calls at once from within a single script. The latter capability is on a list of NCL enhancements to be added in future releases. The two examples below are examples of how to call multiple NCL scripts at once by invoking python from NCL, taking advantage of python's parallel task managing capabilities.

task_parallelism_driver_1.ncl / In this example a ncl script passes a list of ncl scripts to a python script that are to be run at once and managed. The python script loads the commonly available subprocess, sys, time and os modules. Two options are set at the top of the python script. The MAX_CONCURRENT variable is used to set the number of scripts that can run at once. The POLL_INTERVAL variable is used to set the amount of time between when python checks what scripts are actively running.
task_parallelism_driver_2.ncl / This example builds upon example one, adding options that are passed from the NCL driver script to external scripts via environmental variables. Environmental variables passed to the python script include the number of tasks to be run at once, along with the amount of time to wait between checks on the number of scripts running. Other environmental variables are set for use within the called NCL scripts. Note that to set environmental variables from within a NCL script export has to be called. getenv can be used to retrieve a shell environmental variable from within a NCL script.