To start a job, ECFLOW uses the content of the ECF_JOB_CMD variable. By modifying this variable, it is possible to control where and how the job will run. The command should use the ECFLOW variables ECF_JOB and ECF_JOBOUT. ECF_JOB contains the name of the job file and ECF_JOBOUT contains the name of the file that should contain the output. For a ksh or bash UNIX script the default command is:
%ECF_JOB_CMD% 1> %ECF_JOBOUT% 2>&1 & |
To run the tasks on a remote machine you can use the UNIX command rsh (or ssh). We would like the name of the host to be defined by an ECFLOW variable called HOST. We assume that all the files are visible on all the hosts, using NFS. You can then redefine the ECF_JOB_CMD as follows for ksh:
edit ECF_JOB_CMD "rsh %HOST% '%ECF_JOB% > %ECF_JOBOUT% 2>&1 &'" |
As ecFlow makes use of standard UNIX permissions you may experience problems using rsh. Make sure that the file $HOME/.rhosts contains a line with your username and the machine where your ECFLOW is running.
If your login shell is csh, you can define ECF_JOB_CMD as:
edit ECF_JOB_CMD "rsh %HOST% '%ECF_JOB% >& %ECF_JOBOUT%'" |
You can also submit tasks directly to the relevant queuing system on the target machine. In fact at ECMWF, we have written a UNIX script to submit tasks to multiple systems and multiple queuing systems (ecf_submit). An example ecf_submit script is included with the ecFlow release.
edit ECF_JOB_CMD "ecf_submit %USER% %SCHOST% %ECF_JOB% %ECF_JOBOUT%" |
Alongside this we include into our 'ecf' scripts a generic script header containing typical queuing commands (such as wall clock time and priority), e.g.
Contents of sample qsub.h
# QSUB -q %QUEUE% # QSUB -u %USER% # QSUB -s /bin/ksh # QSUB -r %TASK%_%FAMILY1:NOT_DEF% # QSUB -o %LOGDIR%%ECF_NAME%.%ECF_TRYNO% # QSUB -lh %THREADS:1% |
The ecf_submit script can replace these generic queuing commands with the relevant commands for the host to which the task is submitted and submit the task relevant way, e.g. for a PBS system it replaces the QSUB commands with the equivalent PBS commands.