NVIDIA is still tuning the Linux scheduler for its Vera CPU. On Aug. 31, NVIDIA engineer Andrea Righi posted a new patch series to the kernel mailing list that makes one hardware thread on each of Vera’s Olympus cores the preferred sibling. In NVIDIA’s testing, the change raised throughput and made runs more predictable, Phoronix reported.
Why Olympus cores are touchy
Olympus implements SMT with two symmetric processing elements, or PEs. When only one is active, it gets the whole core. When both are active, they share it. Righi wrote in the cover letter that Olympus is “particularly sensitive to brief sibling activations because returning from two-thread mode to single-thread mode after a sibling becomes idle is not immediate.” On the tested system, full single-thread performance returns only after the sibling has stayed idle for 10 Ki cycles.
An earlier Righi commit, merged for Linux 7.3, stopped the NOHZ idle load balancer from needlessly waking a busy PE’s sibling. On its own, that change lifted a GEMM benchmark on Vera from 6.2 to 9.2 TFLOP/s. Ordinary task placement could still pick either sibling, though. Repeatedly switching the active PE could keep cores stuck in two-thread mode even when the siblings had little useful overlap.
The new series uses SD_ASYM_PACKING to make PE0 the preferred sibling. It also teaches the fair scheduler’s idle-selection paths to honor that priority. The scheduler first chooses an idle core using its usual rules, then picks the highest-priority free sibling on that core. NVIDIA ran an 88-thread single-precision GEMM on the 88 physical cores of one NUMA node in a two-node Vera system. Throughput rose from about 9.4 TFLOP/s to about 10.1 TFLOP/s, and run-to-run results became more consistent because the work settled on PE0. The patches are now under review.
The earlier 2x cliff
This is the second round of Vera scheduler work. In March, Righi posted patches to fix a much bigger problem on the Vera Rubin platform. With SMT enabled, the firmware there reports small frequency variations as differences in CPU capacity, which turns on the kernel’s SD_ASYM_CPUCAPACITY logic. “Without these patches, performance can drop up to ~2x with CPU-intensive workloads, because the SD_ASYM_CPUCAPACITY idle selection policy does not account for busy SMT siblings,” he wrote. The fix teaches that policy to prefer fully idle cores. NVIDIA tried alternatives such as equalizing capacities in firmware or in the kernel, but found that adding SMT awareness worked better. Righi added that the change “also seems worthwhile in general, as other platforms in the future may enable SMT with asymmetric CPU topologies.”
NVIDIA is also shipping related fixes in its own Ubuntu-based kernel before they reach mainline. A Launchpad bug Righi filed in June describes concurrent wakeups that put CPU-heavy work on sibling threads while other physical cores sat idle. The fix landed in linux-nvidia-bos 7.0.0-2015.15 in late July. On a Vera Rubin system running an NVPL SGEMM workload limited to one thread per core, the patch produced a consistent 23% increase in mean throughput.
For teams evaluating Vera, the practical point is that kernel version matters. Until the remaining patches reach mainline, CPU-bound performance will depend on running NVIDIA’s kernel or a mainline release that includes these scheduler changes.
WebProNews is an iEntry Publication