What actually makes a batch service horizontally scalable?
Workers that own nothing. If a worker holds state or assigns its own work, adding a second one creates a coordination problem instead of capacity.
Three properties, in order of how often they are missed:
Work is claimed, not pushed. If a coordinator assigns work, the coordinator is the bottleneck and the worker count is baked into its config. If workers atomically claim the next unit, worker count becomes a runtime variable.
Units are idempotent. A worker that dies mid-unit must be safe to replace. That means a unit writes to its own output and can be re-run — which is also what makes the whole run restartable rather than all-or-nothing.
Claims expire. A lease with a timeout releases work from a silently dead worker without needing anything to notice the death.
Get those three and scaling is arithmetic. Miss any one and every added worker buys less than the last.