Does horovod support GPU-level elasticity?

[mozizhao]

Hi,

I understand that horovod can dynamically alter the number of worker nodes, which I think I should call it "node-level elasticity". I want to know if horovod can support "GPU-level" elasticity? For example, given a node with 2 GPUs, a elastic job can switch its worker number from 2 to 1 or 1 to 2. Besides, a elastic job can switch its worker number from 4 (across 2 hosts) to 1(1 worker in only 1 host). To sum up, I want to know if it is possible to achieve GPU-level granularity in horovod.

Thanks.