ray dataset not handled well post downloading deltas

[valiantljk]

https://github.com/ray-project/deltacat/blob/main/deltacat/compute/compactor/steps/hash_bucket.py#L121

if storage_type is StorageType.DISTRIBUTED, the returned tables is a distributed ray dataset.
Several changes are needed to deal with a distributed dataset:

• replace len() with ds.count(), as len() is not an available attribute of ray dataset
• figure out how to enumerate a ray dataset as 'Dataset' object is not iterable
• figure out how to avoid memory copy if ray dataset has to be converted into a pa.table, etc.