You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
mvashishtha opened this issue
Mar 1, 2023
· 0 comments
· May be fixed by #5724
Labels
new feature/request 💬Requests and pull requests for new featuresP2Minor bugs or low-priority feature requestsPerformance 🚀Performance related issues and pull requests.
You can provide a list of local files (but not a list of directories or s3 files) to read_parquet, as in #5698. For #5698 I will make a fix to read the files separately and concat the results. I attempted a more general solution but it required too much surgery on modin's read_parquet code, which assumes in many places that there's just one file or directory to read. One difficulty I had is that ParquetDirectory can take a list of files but not a list of directories, so we can't convert to a list of directories at the beginning.
The text was updated successfully, but these errors were encountered:
new feature/request 💬Requests and pull requests for new featuresP2Minor bugs or low-priority feature requestsPerformance 🚀Performance related issues and pull requests.
You can provide a list of local files (but not a list of directories or s3 files) to read_parquet, as in #5698. For #5698 I will make a fix to read the files separately and concat the results. I attempted a more general solution but it required too much surgery on modin's
read_parquet
code, which assumes in many places that there's just one file or directory to read. One difficulty I had is thatParquetDirectory
can take a list of files but not a list of directories, so we can't convert to a list of directories at the beginning.The text was updated successfully, but these errors were encountered: