Meta-scheduling

Meta-scheduling (also called super-scheduling) is a computer software technique for optimising computational workloads by coordinating multiple underlying job schedulers. A meta-scheduler sits above the individual schedulers within a distributed environment — such as a computing grid or a multi-site high-performance computing facility — and provides an aggregated view of available resources, allowing batch jobs to be directed to the most appropriate location for execution.

The term is also used in the context of embedded real-time systems for a related but distinct concept: generating schedules that cover multiple anticipated operating scenarios or modes, so the system can switch between pre-computed schedules at runtime.

Grid computing context

In grid computing, organisations may run different job schedulers at different sites (for example PBS, SLURM, or LSF). A meta-scheduler provides a unified interface that abstracts these differences, enabling users to submit jobs without specifying which underlying system will execute them. The meta-scheduler selects the target based on factors such as current queue depth, resource availability, job requirements and policy constraints. This approach increases overall utilisation and simplifies access for users working across multiple administrative domains.

Implementations

The following is a partial list of open-source and commercial meta-schedulers used in grid and cluster computing:

• GridWay, developed under the Globus Alliance framework
• Community Scheduler Framework, by Platform Computing and Jilin University
• Moab Cluster Suite and Maui Cluster Scheduler, from Adaptive Computing
• Accelerator Plus, which routes jobs via host jobs in an underlying workload manager to achieve high throughput
• SynfiniWay's meta-scheduler

Embedded systems context

In embedded real-time systems, particularly those using time-triggered architectures such as multi-core systems-on-chip (MPSoCs), meta-scheduling refers to the generation of a set of pre-computed schedules — one per anticipated operating scenario or mode — that the system can select between at runtime without incurring the overhead of dynamic scheduling.

This scenario-based meta-scheduling (SBMeS) approach is applicable to reconfigurable systems and those with variable workloads. By preparing schedules in advance for each expected mode of operation, the system can respond to environmental or workload changes by switching to a different pre-validated schedule rather than recomputing one dynamically. This reduces scheduling overhead and can improve fault recovery behaviour.

References

• B. Sorkhpour, R. Obermaisser and A. Murshed, "Meta-Scheduling Techniques for Energy-Efficient, Robust and Adaptive Time-Triggered Systems," in Knowledge-Based Engineering and Innovation (KBEI), 2017 IEEE 4th International Conference on, Tehran, 2017.
• B. Sorkhpour, O. Roman, and Y. Bebawy, "Optimization of Frequency-Scaling in Time-Triggered Multi-Core Architectures using Scenario-Based Meta-Scheduling": in AmE 2019-Automotive meets Electronics; 10th GMM-Symposium VDE, 2019.
• B. Sorkhpour. "Scenario-based meta-scheduling for energy-efficient, robust and adaptive time-triggered multi-core architectures", University of Siegen, Doctoral thesis, July 2019.