in pytorch mode, why register hook on tmp parameter

[tingshua-yts]

optimizer.py
in horovod pytorch DistributedOptimizer code, why register hook on tmp parameter not the real parameter.

        for param_group in self.param_groups:
            for p in param_group['params']:
                if p.requires_grad:
                    self._requires_update.add(p)
                    p_tmp = p.expand_as(p)
                   # why get  the p_tmp grad_acc, does just set p.register_hook() not work ?
                    grad_acc = p_tmp.grad_fn.next_functions[0][0]
                    grad_acc.register_hook(self._make_hook(p))
                    self._grad_accs.append(grad_acc)