Can't understand work of listObject method in Java MinIO client

[aarexer]

I am using Java Minio client:

io.minio:minio:8.5.7

I create a bucket, upload 1000 files and it's all good. Every file name creates from template: "my-objectname-" + number, where number - is just a counter.

But after that i need to get information about objects, i need to show user list of objects in bucket.

My first attempt was like this:

            Iterable<Result<Item>> results =
                    minioClient.listObjects(
                            ListObjectsArgs.builder()
                                    .bucket("my-bucketname")
                                    .maxKeys(5)
                                    .build()
                    );

            for (Result<Item> result : results) {
                Item item = result.get();
                System.out.println(item.lastModified() + "\t" + item.size() + "\t" + item.objectName());
            }

And as a result it prints me all files in bucket:

2023-12-28T07:28:39.268Z	850000	my-objectname
2024-01-06T13:57:50.470Z	850000	my-objectname-0
2024-01-06T13:57:50.496Z	0	my-objectname-1
2024-01-06T13:57:50.687Z	0	my-objectname-10
2024-01-06T13:57:52.481Z	0	my-objectname-100
2024-01-06T13:58:12.930Z	0	my-objectname-1000
2024-01-06T13:58:12.955Z	0	my-objectname-1001
2024-01-06T13:58:13.032Z	0	my-objectname-1002
2024-01-06T13:58:13.065Z	0	my-objectname-1003
2024-01-06T13:58:13.091Z	0	my-objectname-1004
2024-01-06T13:58:13.114Z	0	my-objectname-1005
......

After that i have questions:

1. Why maxKeys doesn't work as i think? Why it's iterate for all files in bucket?
2. How to get full information about file? For example, content-disposition header (how i can get headers?)? Do additional calls to minio like statObject?
3. Why files are in this order?
4. What is the best way (implementation) for my task? I need to show list of files with pagination or infinite scroll - doesn't matter, but i can't understand how to develop it without ugly code - cause i got iterator for all files in bucket. And Item doesn't store all information about file (headers: content type, content disposition)

Thanks for your help!

[klauspost]

1. I believe maxKeys sets the number of keys per request and then does multiple requests. @balamurugana can confirm.
2. If you need things like "content-disposition" you will need to Stat the object. This is not returned by listings. You can try includeUserMetadata(), but that is not an official S3 API, so it will only work on MinIO hosts.
3. This is Lexicographic (alphabetical) order, which is the order ListObjects returns results.
4. Correct - this is not trivial. The ListObjects API only has start-after, which makes pagination pretty much impossible. You will probably need to do the listing on your backend to be able to do pagination.

We cannot change the S3 API.

[aarexer]

Very BIG Thanks for your answer!

1. Yes, but it's not simple to understand (for me) - cause i can iterate for all files in bucket (as in my example).
2. Can't find any information that use includeUserMetadata() includes all headers, but it's good solution for me, we are using only MinIO hosts. Thanks!
3. Is java client has any methods (comparator?) to change this behavior?
4. Ok, we can use keyset - and nextToken would be the last object name in returned list? Yes?

And another question (i forgot it):
How can i do expiration policy for files? I want to store some temporary files with ttl, and some files without ttl (until the user deletes it). Is there any way to store it, but without dividing into different buckets?

[klauspost]

3. Is java client has any methods (comparator?) to change this behavior?

This is the order the server must return the results. There are no options for this in the S3 API.

4. Ok, we can use keyset - and nextToken would be the last object name in returned list? Yes?

No. Again, not part of the API.

How can i do expiration policy for files?

See: https://min.io/docs/minio/kubernetes/upstream/administration/object-management/create-lifecycle-management-expiration-rule.html - it is configured per bucket - but can also be set for prefixed within a bucket.

[aarexer]

Thanks you very much!

[balamurugana]

The meaning of maxKeys is as per S3 specification i.e. how many keys to be fetched by each iteration from the server. listObjects() method works recursively calling listObjects low level S3 calls to return all requested objects.

If you would need to limit listing, either you stop iterating as per your conditions or use low level S3 API.

[aarexer]

Thanks, if you can - see my additions for question above. Thanks you very much!