Information missing from libmediainfo outptut using MediaInfo_Open_Buffer

[disc-kuraudo]

mediainfo GUI and CLI show a lot more information in both Text and JSON output, but through library using MediaInfo_Open_Buffer it doesn't seem to get the same information.

For example Format Profile/Level/Settings and Writing Library is missing in the Video structure.

sample_code.c.txt
sample_file

mediainfo_output_json.txt
libmediainfo_buffer_output_json.txt

I resort to non-seekable files and feed them through MediaInfo_Open_Buffer_Continue since I'm reading them directly off memory-mapped discs, where files may be fragmented. Even setting the seekable option to 1 and feeding the sample file all at once seems to be cutting out said metadata.

Using Mediainfo_Open(mi, SAMPLE_FILE) shows the full information.
I'd have to resort to a workaround, writing files to tmp files on disk and nullifying the @ref field after. Possible but introducing a lot of unnecessary writes to disk, as FIFOs don't work with mediainfo, either.

libmediainfo: 24.01

[JeromeMartinez]

With your file, the parser tries to read the frame index at the end for knowing where is the first frame, and stops because it can not know where is the frame (which has the profile etc), so the behavior is expected when you don't provide the full file.

feeding the sample file all at once seems to be cutting out said metadata.

Here it should works, we should be able to "seek" internally, this is a bug, I let the ticket open due to that.

It should maybe also work with MI configured with "File_IsSeekable", "0" as we could bet that the frames are in sequential order (not sure if it is mandatory but in practice this is the case, I think), but not implemented (and not sure we will, depends on what says the specs about the order of frames).

In your case, an easy workaround is to simulate the seek feature on a buffer with the full file, something like that (tested C++ based on the example but easy to map to your style):

        #define FILE_SIZE 244789248
        #define SAMPLE_SIZE 64*1024
        int fd = open(SAMPLE_FILE, O_RDONLY);
        uint8_t* buf = (uint8_t*)malloc(FILE_SIZE);

        read(fd, buf, FILE_SIZE);

        MediaInfo MI;
        //MI.Option("File_IsSeekable", "0");
        MI.Option("Output", "JSON");
        uint64_t C;

        MI.Open_Buffer_Init(FILE_SIZE, 0);
        uint8_t* From_Buffer = buf;
        do
        {
            //Reading data somewhere, do what you want for this.
            size_t From_Buffer_Size = buf + FILE_SIZE - From_Buffer;
            if (From_Buffer_Size > SAMPLE_SIZE)
                From_Buffer_Size = SAMPLE_SIZE;

            //Sending the buffer to MediaInfo
            size_t Status = MI.Open_Buffer_Continue(From_Buffer, From_Buffer_Size);
            if (Status & 0x08) //Bit3=Finished
                break;
            From_Buffer += From_Buffer_Size;

            //Testing if there is a MediaInfo request to go elsewhere
            if (MI.Open_Buffer_Continue_GoTo_Get() != (MediaInfo_int64u)-1)
            {
                From_Buffer = buf + MI.Open_Buffer_Continue_GoTo_Get();             //Position the file
                MI.Open_Buffer_Init(FILE_SIZE, MI.Open_Buffer_Continue_GoTo_Get()); //Informing MediaInfo we have seek
            }
        } while (buf < buf + FILE_SIZE);

Note that Open_Buffer_Init always expects file size, not sample size.

I'm reading them directly off memory-mapped discs,

So you could seek directly off memory-mapped discs, it would be less stressing, not reading the whole file but only the requested chunks.

as FIFOs don't work with mediainfo, either

Pipes work but you need to provide "-" as the file name (we should autodetect... to be implemented at some point):

cat aAa_s_frags_collection_2003__counter_strike_-_quake_III_arena_-_unreal_tournament_2003_.avi | mediainfo - "--Output=General;%Format%"
AVI

But you'll never get what you are looking for due to the lack of seek and we don't bufferize the whole file.

[disc-kuraudo]

Thanks for the example, I remember having read some (old) documentation snippets about to how to use the library and was under the impression that the interface is not fully functional and worked with what I got, but this looks good and seems to work, seeking would actually not be an issue, would just have to implement this on my side.

So bottom line is currently I would have to resort to not pushing through the whole file (buffer) at once but to resort to feeding it in smaller chunks until the issue is fixed? 99.9% of the time the files are not fragmented but a few are, so performance wise I'd like to push the whole file through at once when fixed.

Also oops for putting this issue under the MediaInfo repo instead of MediaInfoLib.

[JeromeMartinez]

So bottom line is currently I would have to resort to not pushing through the whole file (buffer) at once but to resort to feeding it in smaller chunks until the issue is fixed?

Yes (+ seek request management).

so performance wise I'd like to push the whole file through at once when fixed

performance wise it is better to feed as it comes from the input, and loading 200+ MB is really not the best approach even if we think just to RAM bandwidth, read 16 MiB chunk rather than 64 KiB chunks if you prefer (IIRC limit is not the whole file else... The bug) but I don't recommend at all the whole file even if the issue is fixed.

Also oops for putting this issue under the MediaInfo repo instead of MediaInfoLib.

Oops, I missed it, fixed :-D.

[disc-kuraudo]

I have a small Blu-Ray CLIPINF file that doesn't seem to get parsed correctly as well with the workaround, even with SAMPLE_SIZE down to 2048 or 512 bytes:
00000.clpi.zip

Piping to mediainfo - doesn't work, either.

May be related and hopefully fixed in the same pass.

Edit: Also an .mpls file from Blu-Ray for good measure, same as above.
00000.mpls.zip
Note: mediainfo cli/gui provides additional information for if the associated .m2ts file if found under STREAM where expected, just to not be confused as to why this additional information would be missing if there is no such context.

Off-topic: HD-DVD (gone, but not forgotten) .MAP files containing metadata don't seem to be recognized as a format for mediainfo, would this be something I could feature request (as an Issue)?

[JeromeMartinez]

I have a small Blu-Ray CLIPINF file that doesn't seem to get parsed correctly as well with the workaround

Actually it was deactivated on purpose, but with a second thought 10 years later it may make sense to parse single Bluray files without the directory structure, and the current code supports that even if you don't have all, so changed.

In latest snapshots.

Note: mediainfo cli/gui provides additional information for if the associated .m2ts file if found under STREAM where expected

And this is not supported for the moment. It is planned to support Bluray ISO as we do now for DVD ISO but currently not yet done.

Off-topic: HD-DVD (gone, but not forgotten) .MAP files containing metadata don't seem to be recognized as a format for mediainfo, would this be something I could feature request (as an Issue)?

Separate ticket please.

[disc-kuraudo]

Thanks, will check it out soon.

Note: mediainfo cli/gui provides additional information for if the associated .m2ts file if found under STREAM where expected

And this is not supported for the moment. It is planned to support Bluray ISO as we do now for DVD ISO but currently not yet done.

I don't mind that, I just wanted to point it out so noone implementing the buffer/pipe fix would be wondering why some information would seemingly be missing over command line/GUI directly off a mounted disc.

[disc-kuraudo]

Can confirm it works now through the library: fuseboxfilms.sky.sharks.r2021.08.26.aacs.bd50.dkm
The .clpi and .mpls now provide a boatload of useful metadata about the encrypted .m2ts files.