When running trainer script of transformers with some changes , throwing error

[22Mukesh22]

4/07/2023 15:13:14 - INFO - main - Grouping texts into single entries
[INFO|trainer.py:568] 2023-04-07 15:13:16,718 >> Using cuda_amp half precision backend
/home/user/.local/lib/python3.8/site-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set no_deprecation_warning=True to disable this warning
warnings.warn(
Traceback (most recent call last):
File "run_clm.py", line 555, in
main()
File "run_clm.py", line 518, in main
train_result = trainer.train(resume_from_checkpoint=checkpoint)
File "/home/user/.local/lib/python3.8/site-packages/transformers/trainer.py", line 1572, in train
return inner_training_loop(
File "/home/user/.local/lib/python3.8/site-packages/transformers/trainer.py", line 1650, in inner_training_loop
self.create_optimizer_and_scheduler(num_training_steps=max_steps)
File "/home/user/.local/lib/python3.8/site-packages/transformers/trainer.py", line 1021, in create_optimizer_and_scheduler
self.create_optimizer()
File "/home/user/.local/lib/python3.8/site-packages/transformers/trainer.py", line 1085, in create_optimizer
hvd.broadcast_parameters(self.model.state_dict(), root_rank=0) #hvd_18
File "/usr/local/lib/python3.8/site-packages/horovod/torch/functions.py", line 54, in broadcast_parameters
handle = broadcast_async(p, root_rank, name)
File "/usr/local/lib/python3.8/site-packages/horovod/torch/mpi_ops.py", line 880, in broadcast_async_
return _broadcast_async(tensor, tensor, root_rank, name, process_set)
File "/usr/local/lib/python3.8/site-packages/horovod/torch/mpi_ops.py", line 777, in _broadcast_async
function = _check_function(_broadcast_function_factory, tensor)
File "/usr/local/lib/python3.8/site-packages/horovod/torch/mpi_ops.py", line 100, in _check_function
raise ValueError('Tensor type %s is not supported.' % tensor.type())