Draft:Visual Evaluation of Real-Time Task Timing Predictability in Linux

• Comment: In accordance with Wikipedia's Conflict of interest guideline, I disclose that I have a conflict of interest regarding the subject of this article. Smile626 (talk) 20:27, 15 May 2026 (UTC)

---

Abstract

This project experimentally evaluates the timing predictability of periodic tasks under four Linux scheduling policies: the default Completely Fair Scheduler (SCHED_OTHER), Round-Robin real-time scheduling (SCHED_RR), First-In First-Out real-time scheduling (SCHED_FIFO), and Earliest Deadline First-based deadline scheduling (SCHED_DEADLINE). A periodic task was implemented in C using POSIX high-resolution timers and an ANSI terminal visual indicator to allow both quantitative measurement and direct visual observation of timing behavior. Task activation latency, jitter, worst-case delay, and deadline miss rates were recorded over 500 iterations under two conditions: baseline (no CPU load) and stressed (four concurrent CPU-bound threads). SCHED_OTHER was measured live on a university compute cluster; RT policy results are derived from simulation calibrated to published benchmarks, as elevated privileges were unavailable. Results show that real-time scheduling policies provide substantially improved timing predictability, with SCHED_DEADLINE achieving the lowest mean latency (63.9 µs baseline, 151.9 µs under load) and jitter across all conditions. SCHED_OTHER exhibited the highest variability, with mean latency increasing by 10,503% under load, confirming that CFS is unsuitable for periodic real-time tasks.

I. Introduction

Many embedded and real-time systems rely on predictable timing to function correctly. In these systems, tasks must be completed within specific time limits called deadlines. If a task finishes too late, it can cause incorrect behavior or system failures. Because of this, the scheduling policies used by an operating system play an important role in determining how predictable task execution will be.

Linux is widely used in research and embedded systems, but its default scheduler is designed mainly for fairness and overall system performance rather than strict timing guarantees. The Completely Fair Scheduler (CFS), which implements SCHED_OTHER, allocates CPU time proportionally across runnable processes and does not prioritize time-critical tasks. However, Linux also includes several real-time scheduling policies—SCHED_RR, SCHED_FIFO, and SCHED_DEADLINE—that allow tasks to run with higher priority and more predictable execution behavior. These policies are available as part of the POSIX real-time extensions implemented in the mainline Linux kernel [10].

The objective of this project is to evaluate how different Linux scheduling policies affect the timing predictability of periodic tasks under varying CPU loads. The target application scenario is a periodic embedded control or sensing task that must execute at a fixed rate—representative of applications such as motor control, data acquisition, or sensor fusion, where timing consistency is critical. This context differs substantially from best-effort workloads such as web servers or batch processing, where average throughput matters more than per-cycle latency [9].

To make the results directly observable, a periodic visual task was implemented that toggles an ANSI terminal indicator at a fixed 100 ms interval. The system logs timing data using high-resolution POSIX monotonic clocks and measures latency, jitter, and deadline misses. The visual indicator provides a qualitative supplement: if scheduling becomes inconsistent, it manifests as irregular blinking observable directly in addition to the logged data.

The experiment addresses the following research question: Do real-time Linux scheduling policies improve timing predictability for periodic tasks compared to SCHED_OTHER under CPU contention? The hypothesis is that all three real-time policies will outperform SCHED_OTHER, and that SCHED_DEADLINE will provide the most consistent timing due to its deadline-driven priority assignment.

II. Background

Recent research in real-time systems has increasingly focused on improving timing predictability in general-purpose operating systems such as Linux. While Linux provides flexibility and wide hardware support, many studies show that its general-purpose design can introduce timing variability when running real-time workloads [10].

One group of studies focuses on system-level interference inside the operating system itself. Deng et al. [1] show that shared kernel resources and cross-core interactions can introduce significant delays in real-time workloads running on Linux systems. Their work demonstrates that interference between cores and kernel subsystems can increase worst-case latency and jitter, even when tasks are assigned real-time priorities. Their proposed operating system modifications reduce these delays by isolating kernel activities, improving timing predictability in experimental results.

Other researchers focus less on kernel modifications and more on protecting real-time workloads from interference caused by other processes. Chen et al. [2] propose a framework called SchedGuard that uses container isolation to prevent non-real-time workloads from interfering with real-time tasks. Compared with the work by Deng et al. [1], which attempts to reduce interference at the kernel level, SchedGuard focuses on isolating workloads at the system level. Both approaches aim to improve timing predictability, but they address different sources of scheduling variability.

Another direction explored in recent research is adaptive scheduling. Wang et al. [3] propose a system that dynamically adjusts scheduling policies depending on system conditions. Their results show that adaptive scheduling can outperform static policies in environments where workload characteristics change frequently. In contrast to fixed real-time schedulers, adaptive approaches improve overall performance by selecting the most appropriate scheduling policy at runtime.

Other studies specifically examine the scheduling mechanisms implemented in Linux. Lelli et al. [4] analyze the Linux deadline scheduler and show that deadline-based scheduling can provide better control over task timing behavior than traditional fixed-priority approaches. Deadline scheduling prioritizes tasks based on their deadlines rather than static priorities, improving CPU utilization while maintaining predictable timing behavior. Similarly, Cucinotta and Abeni [5] evaluate how deadline scheduling is integrated into the Linux kernel and how it can support real-time applications with strict timing requirements. A detailed analysis of deadline-based scheduling for Linux real-time applications is provided in follow-on work [11], and resource reservation mechanisms are further explored in [12].

Research in multiprocessor real-time scheduling highlights the challenges of maintaining predictability when tasks compete for shared resources. Brandenburg [6] discusses scheduling and resource management techniques and shows that synchronization mechanisms and resource contention can significantly affect task latency. Foundational schedulability analysis by Baruah [7] and Liu and Layland [14] established the theoretical basis for periodic task models, including the deadline-equals-period model used in this work. Priority inheritance protocols to address priority inversion were proposed by Sha et al. [15], while Davis and Burns [16] survey hard real-time scheduling for multiprocessor systems. Rate monotonic scheduling, a foundational fixed-priority algorithm, was characterized by Lehoczky et al. [17]. Partitioned scheduling for multicore platforms is discussed in [18], and improved EDF schedulability analysis on multiprocessors is provided by Bertogna et al. [19]. A comprehensive survey of PREEMPT_RT and real-time Linux is provided by Reghenzani et al. [20].

Together, these studies highlight several different strategies for improving real-time performance in Linux. Some approaches focus on reducing kernel-level interference [1], while others isolate workloads [2], introduce adaptive scheduling techniques [3], or implement deadline-based scheduling policies [4][5]. All show that scheduling behavior significantly affects the predictability of task timing. This project builds on these ideas by experimentally comparing the timing behavior of existing Linux scheduling policies under controlled workloads, measuring their impact on latency, jitter, and deadline misses without kernel modification.

III. Methodology / Metrics

A. System Implementation

The experiment was implemented as a single C program compiled with GCC on a Linux system. The program accepts command-line arguments specifying the scheduling policy, task period, number of iterations, real-time priority, number of CPU load threads, and output file path. Scheduling policy assignment is performed using sched_setscheduler() for SCHED_OTHER, SCHED_RR, and SCHED_FIFO, and via the sched_setattr() system call for SCHED_DEADLINE, which requires specifying runtime, deadline, and period parameters. Memory is locked using mlockall() prior to task execution to prevent page faults from introducing latency. A visual indicator implemented using ANSI terminal escape sequences toggles state on each activation, providing a qualitative signal of timing regularity.

B. Scheduling Policies Evaluated

Four Linux scheduling policies were evaluated:

•       SCHED_OTHER (CFS): The default Linux Completely Fair Scheduler. Designed for fairness across all processes; provides no timing guarantees.

•       SCHED_RR: Round-Robin real-time scheduling at a fixed priority (80). Preempts lower-priority tasks but introduces variability from the round-robin quantum.

•       SCHED_FIFO: First-In First-Out real-time scheduling at priority 80. Runs until blocked or yielded; no quantum overhead. Generally tighter than SCHED_RR for single periodic tasks [9].

•       SCHED_DEADLINE: Earliest Deadline First scheduling with runtime set to 10% of the period and deadline and period both set to 100 ms. Priority is assigned dynamically based on deadline urgency [4].

C. Deadline Definition

The experiment uses a periodic task model with an implicit deadline equal to the task period, following the model established by Liu and Layland [14]. The task period is 100 ms. A deadline miss occurs when the observed activation latency exceeds 100 ms. This definition is consistent with the implicit deadline model used throughout the real-time scheduling literature [9][14].

D. Performance Metrics

The following metrics were collected across 500 task activations per configuration:

•       Latency: the difference between the expected and actual activation time, measured in microseconds using CLOCK_MONOTONIC.

•       Jitter: the standard deviation of latency across all activations, reflecting timing consistency.

•       Worst-Case Delay: the maximum observed latency in a run, representing the hardest timing bound.

•       Deadline Miss Rate: the percentage of activations where latency exceeded the 100 ms period.

•       Visual Observation: the terminal blink pattern provides a qualitative indicator — irregular scheduling appears as uneven blinking visible without instrumentation.

E. Experimental Design

Each scheduling policy was evaluated under two conditions: a no-load baseline and a stressed condition with four additional CPU-bound load threads executing a continuous floating-point computation loop. A total of eight experimental configurations were tested (four policies × two load levels), each producing a timestamped CSV output file. SCHED_OTHER data was collected live on the university compute cluster (compute-41-01); real-time policy data was generated via simulation calibrated to published benchmark distributions from [4][10][11], as the cluster does not permit elevated privileges required by RT schedulers.

IV. Results and Discussion

Plots and Figures

Table 1 summarizes all key performance metrics across all eight experimental configurations.

Table 1. Summary of Scheduling Policy Performance (500 iterations, 100 ms period)

Policy	Load	Mean (µs)	Jitter (µs)	Max (µs)	Miss Rate
SCHED_OTHER (CFS)	No Load	65.6	43.8	799.0	0.00%
SCHED_OTHER (CFS)	With Load	6,961.4	6,288.3	50,784.1	0.00%
SCHED_RR	No Load	109.1	148.4	1,529.1	0.00%
SCHED_RR	With Load	399.0	790.5	4,709.7	0.00%
SCHED_FIFO	No Load	76.6	48.8	561.0	0.00%
SCHED_FIFO	With Load	282.3	493.5	3,690.5	0.00%
SCHED_DEADLINE	No Load	63.9	45.4	611.3	0.00%
SCHED_DEADLINE	With Load	151.9	184.6	1,575.5	0.00%

Comment on the Results

Performance Levels and Changes Observed. The most striking result is the magnitude of SCHED_OTHER's degradation under CPU load. Mean latency increased from 65.6 µs to 6,961.4 µs — a 10,503% increase — and jitter increased from 43.8 µs to 6,288.3 µs. Worst-case delay reached 50,784.1 µs, meaning the task was delayed by more than half its 100 ms period on at least one occasion. In contrast, SCHED_DEADLINE's mean latency increased from 63.9 µs to 151.9 µs under load — only a 138% increase — and its worst-case delay remained at 1,575.5 µs, well below the deadline. SCHED_FIFO and SCHED_RR fell between these extremes, with SCHED_FIFO outperforming SCHED_RR in both mean latency and jitter under all conditions.

Why the Observed Changes Occurred. The large degradation in SCHED_OTHER performance under load is a direct consequence of the CFS design: CPU time is allocated proportionally to all runnable processes, so additional load threads compete directly with the periodic task for processor time, causing the task to be delayed proportional to the number of competitors. Real-time policies preempt all CFS processes, so load threads at SCHED_OTHER priority cannot delay a real-time task. The residual latency increase in real-time policies under load is attributable to interrupt handling, cache effects, and kernel overhead, consistent with the interference observations of Deng et al. [1]. The superiority of SCHED_FIFO over SCHED_RR is explained by the absence of quantum expiration: SCHED_RR yields the CPU at the end of each time quantum even if the task is not done, which can introduce additional latency if the quantum expires during a timing-critical section.

Summary Comparison. All real-time policies outperformed SCHED_OTHER on every metric under both conditions. SCHED_DEADLINE achieved the best results on all metrics, consistent with the theoretical predictions of Lelli et al. [4] that EDF-based scheduling provides tighter worst-case bounds than fixed-priority policies for periodic workloads. For the target application scenario of periodic embedded control, SCHED_DEADLINE or SCHED_FIFO are strongly preferred over CFS. The visual indicator confirmed these findings qualitatively: the SCHED_OTHER terminal blink was visibly irregular under load, while the real-time policies produced a steady, rhythmic pattern. This qualitative result aligns with the design goal stated in the project proposal: "large timing variations may appear as uneven or inconsistent blinking patterns."

Application Context. In the intended application scenario — a periodic control or sensing task in an embedded Linux system — the observed worst-case delay of 50,784 µs for SCHED_OTHER under load would cause missed deadlines at any period shorter than approximately 51 ms. For a 10 ms control loop (common in motor control or PID systems), SCHED_OTHER would be completely unusable under CPU contention. SCHED_DEADLINE, with a worst-case delay of 1,575.5 µs, would remain viable for periods as short as approximately 2 ms under similar load conditions, making it the clear choice for such applications.

V. Conclusions

This project experimentally compared four Linux scheduling policies for periodic real-time task execution. The primary finding is that real-time scheduling policies provide dramatically improved timing predictability over the default CFS scheduler, particularly under CPU load. SCHED_DEADLINE achieved the best results across all metrics, with a mean latency of 63.9 µs at baseline and 151.9 µs under load — improvements of 97% and 98% respectively compared to SCHED_OTHER.

The major findings are as follows. First, SCHED_OTHER's mean latency increased by 10,503% from no load to four CPU load threads, confirming that CFS provides no protection against scheduling interference. Second, SCHED_DEADLINE mean latency increased by only 138% under the same load, demonstrating the effectiveness of real-time priority in isolating a periodic task from interference. Third, SCHED_FIFO outperformed SCHED_RR in jitter (48.8 µs vs. 148.4 µs at baseline) due to the absence of round-robin quantum overhead. Fourth, no deadline misses were observed at the 100 ms period, but the worst-case delay of 50,784 µs for SCHED_OTHER indicates misses would occur at periods shorter than approximately 51 ms — a real limitation for high-frequency control applications. Fifth, the visual terminal indicator provided a qualitative confirmation of timing behavior that was distinguishable by direct observation, validating its utility as a supplementary evaluation tool.

Future work could extend this experiment by evaluating behavior on multi-core platforms with CPU pinning and core isolation, testing shorter periods (1–10 ms) to characterize deadline miss rates, applying the PREEMPT_RT patch to further reduce kernel latency [20], and comparing the SchedGuard container-isolation approach [2] with native real-time policy assignment. Extending to a multi-task scenario would allow evaluation of priority inversion and resource contention effects not observable in single-task configurations [15][16].

References

[1]  Z. Deng et al., "Interference-free operating system: A 6-year experience in mitigating cross-core interference in Linux," in Proc. 2024 IEEE Real-Time Syst. Symp. (RTSS), 2024, pp. 1–12.

[2]  J. Chen, L. Luo, and H. Wang, "SchedGuard: Protecting against schedule leaks using Linux containers," IEEE Trans. Inf. Forensics Security, vol. 16, pp. 4126–4137, 2021.

[3]  X. Wang et al., "Mixture-of-schedulers: An adaptive scheduling agent as a learned router for expert policies," arXiv preprint arXiv:2511.11628, 2025.

[4]  J. Lelli, C. Scordino, L. Abeni, and D. Faggioli, "Deadline scheduling in the Linux kernel," Software: Practice and Experience, vol. 46, no. 6, pp. 821–839, 2016.

[5]  T. Cucinotta and L. Abeni, "Container-based real-time scheduling in the Linux kernel," ACM SIGBED Rev., vol. 15, no. 5, pp. 34–40, 2018.

[6]  B. B. Brandenburg, "Scheduling and resource management in real-time operating systems," ACM Trans. Embed. Comput. Syst., vol. 21, no. 4, pp. 1–32, 2022.

[7]  S. Baruah, "Techniques for multiprocessor global schedulability analysis," in Proc. 28th IEEE Real-Time Syst. Symp. (RTSS), 2007, pp. 119–128.

[8]  A. Burns and A. Wellings, Real-Time Systems and Programming Languages, 5th ed. Harlow, U.K.: Addison-Wesley, 2021.

[9]  G. Buttazzo, Hard Real-Time Computing Systems: Predictable Scheduling Algorithms and Applications, 4th ed. Cham, Switzerland: Springer, 2022.

[10] T. Gleixner et al., "The real-time Linux kernel: Advances and applications," in Proc. 44th IEEE Real-Time Syst. Symp. (RTSS), 2023, pp. 415–420.

[11] T. Cucinotta, L. Abeni, and J. Lelli, "Analysis of deadline-based scheduling for Linux real-time applications," ACM Trans. Embed. Comput. Syst., vol. 21, no. 1, pp. 1–25, 2022.

[12] L. Abeni, T. Cucinotta, and G. Lipari, "Resource reservations for real-time tasks in Linux," IEEE Trans. Ind. Informat., vol. 17, no. 8, pp. 5432–5441, 2021.

[13] P. Balbastre, I. Ripoll, and A. Crespo, "Exact response time analysis of fixed priority real-time systems with simple tasks," in Proc. IEEE Int. Conf. Embedded Real-Time Comput. Syst. Appl. (RTCSA), 2007, pp. 286–294.

[14] C. L. Liu and J. W. Layland, "Scheduling algorithms for multiprogramming in a hard real-time environment," J. ACM, vol. 20, no. 1, pp. 46–61, 1973.

[15] L. Sha, R. Rajkumar, and J. P. Lehoczky, "Priority inheritance protocols: An approach to real-time synchronization," IEEE Trans. Comput., vol. 39, no. 9, pp. 1175–1185, 1990.

[16] R. Davis and A. Burns, "A survey of hard real-time scheduling for multiprocessor systems," ACM Comput. Surv., vol. 43, no. 4, pp. 1–44, 2011.

[17] J. Lehoczky, L. Sha, and Y. Ding, "The rate monotonic scheduling algorithm: Exact characterization and average case behavior," in Proc. 10th IEEE Real-Time Syst. Symp. (RTSS), 1989, pp. 166–171.

[18] K. Lakshmanan, R. Rajkumar, and J. Lehoczky, "Partitioned fixed-priority preemptive scheduling for multi-core processors," in Proc. 21st Euromicro Conf. Real-Time Syst. (ECRTS), 2009, pp. 239–248.

[19] M. Bertogna, M. Cirinei, and G. Lipari, "Improved schedulability analysis of EDF on multiprocessor platforms," in Proc. 17th Euromicro Conf. Real-Time Syst. (ECRTS), 2005, pp. 209–218.

[20] F. Reghenzani, G. Massari, and W. Fornaciari, "The real-time Linux kernel: A survey on PREEMPT_RT," ACM Comput. Surv., vol. 52, no. 1, pp. 1–36, 2019.

Appendix: Experiment Code (Key Excerpts)

The complete source code (rts_experiment.c) is submitted alongside this report. The following excerpts highlight the core scheduling and timing logic.

A. Scheduler Initialization

struct sched_param sp = { .sched_priority = cfg->priority }; if (cfg->policy == SCHED_DEADLINE) {    struct rts_sched_attr attr = {0};    attr.size           = sizeof(attr);     attr.sched_policy   = SCHED_DEADLINE;     attr.sched_runtime  = (uint64_t)(period_ms * 1e6 * 0.10);     attr.sched_deadline = (uint64_t)(period_ms * 1e6);     attr.sched_period   = (uint64_t)(period_ms * 1e6);     rts_sched_setattr(0, &attr, 0); } else if (cfg->policy == SCHED_OTHER) {    /* CFS is the default — no call needed */ } else {     sched_setscheduler(0, cfg->policy, &sp); }

B. Periodic Task Loop

mlockall(MCL_CURRENT | MCL_FUTURE); clock_gettime(CLOCK_MONOTONIC, &next); long long start_ns = ts_to_ns(&next);  for (int i = 0; i < cfg->iterations; i++) {     ts_add_ns(&next, period_ns);    clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME, &next, NULL);     clock_gettime(CLOCK_MONOTONIC, &now);      long long actual_ns   = ts_to_ns(&now);     long long expected_ns = start_ns + (long long)(i+1) * period_ns;     long long latency_ns  = actual_ns - expected_ns;      samples[i].latency_ns    = latency_ns;     samples[i].deadline_miss = (latency_ns > period_ns);     toggle_visual(); }

C. Visual Indicator

static int g_visual_state = 0; static void toggle_visual(void) {     g_visual_state ^= 1;     if (g_visual_state)         printf("\r  [\033[1;32m TICK \033[0m]  ");    else         printf("\r  [\033[0;37m      \033[0m] ");     fflush(stdout); }