If you had a partition with 1 replica on a deleted device, 1 on a
zero-weight device, and 1 on a normal device, then rebalancing would
crash. This was because the ring builder was memoizing tiers_for_dev()
for all *available* devices (i.e. weight > 0 and not deleted), but
depended on it being present for all devices with partitions still on
them.
Since the builder moved the replica from the deleted device, it left
the one on the zero-weight device alone, so you had an unavailable
device with a partition replica still on it, triggering the crash.
Now we go ahead and memoize tiers_for_dev() for *all* devices, not
just available ones, thereby fixing the crash.
Change-Id: Ie0b58b65e0353732cf785ab772e95e699f3a5b5d
e7fb089f59