Springe direkt zu Inhalt

Management of Resources

The batch system manages resources, such as CPUs and memory, allocates these to waiting jobs, and uses a method called fairshare scheduling to ensure that computing time is shared fairly among all the users and contributes to the priority of jobs. Because the memory on the nodes is shared amoung the jobs, it is very important to estimate memory requirements as accurately as possible. A further mechanism, called Backfill, is employed to allow short-running jobs to fill in gaps in the scheduling plan and thus optimise resource usage.

The priority of jobs is regulated by fairshare scheduling. Whether job A is started before job B depends mainly on two factors:

  • when the job was submitted
  • how many shares are available to the user

In general, a job which is submitted to the batch system before another will also be started before the other. However, a user consumes so-called shares when his or her job runs based on the amount of CPU time, GPU time and RAM used. The more shares a user has, the more his or her jobs will be preferred. As time passes, a user is given new shares. In this manner, equitable access to resources is achieved.

This mechanism only plays a role when jobs compete for resources. If enough resources are available, a job will also be started, even if the owner no longer has any shares.

The priority of a job is mainly determined by the following factors, in order of importance

  • User Shares - The shares a user has in the sense of fairshare scheduling constitutes the most important factor. The more shares a users has, the higher the priority of his or her job will be. Consuming CPU time, GPU time or RAM reduces the number of shares a user has and thus lowers the priority of any waiting jobs.
  • Age of Job - How long the job has been waiting in the queue is another important factor. However the value is limited and reaches a maximum after a certain time.
  • Size of Job - Larger jobs in term of number of nodes requested will have their priority increased slightly.

The factors affecting the priority of the currently waiting jobs can be viewed with the following command, which sorts the results according to total priority:

sprio -Sy