Page MenuHomeMiraheze

Remove :rootjobs: periodically
Closed, ResolvedPublic

Description

The current Redis JobRunner code relies on Redis' expiry to handle rootjobs. While this isn't a big issue as memory isn't a problem for us - this requires a strong cluster of Redis servers to handle an increasing loads of O(N) Lua script calls.

If we remove expired :rootjobs: once the relevant SHA1 is no longer known to us, we will reduce the number of abandoned keys inside Redis, reduce the time complexity of our Lua scripts and improve the performance of our jobrunner processes.