Building a distributed job scheduler with Akka.NET

Aaron Stannard of Petabridge describes how to build a distributed scheduler for long-running jobs with Akka.NET — for cases where jobs run from minutes to hours and the cluster must scale without losing work already done. He explains why plain Akka.Cluster.Sharding doesn't fit here: when the cluster resizes, it moves jobs to new nodes and throws away work already completed. Instead, the system is built on several actors, where the main one, JobTracker, keeps the schedule and survives crashes. The key idea is to distribute jobs by nodes' actual load: the scheduler estimates a job's "size" and sends it to the least busy node, avoiding overloading individual machines. This approach wastes no computation and fits well for transcription, transcoding, ETL, and financial reporting.