TFLite Movenet multipose/lightning input error

[ArtanBerisha1]

Hi,
I am trying to load and run the Movenet multipose model, the loading of model works fine but when I try to run it gives this error:
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_input:0) with 3 bytes from a Java Buffer with 491520 bytes.

As I can understand (and also I looked the model graph in Netron app) the input of the model is type: uint8[1,1,1,3] , and this is what the error is also telling, that my converted image doesn't fit the input of the model. So my question is: is this intentionally or the model is not correct ?

I downloaded model from Tensorflow Hub -> https://tfhub.dev/google/lite-model/movenet/multipose/lightning/tflite/float16/1
And here (in the link above) also is specified that the model accepts 'A frame of video or an image, represented as an int32 tensor of dynamic shape: 1xHxWx3'. I handled the dynamic shape tensor as suggested in the link.

Thank you!

[sushreebarsa]

@ArtanBerisha1
In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here & let us know which TF version you're using ?Thanks!

[ArtanBerisha1]

Hi, @sushreebarsa
Here are TF versions:

    implementation 'org.tensorflow:tensorflow-lite:2.6.0'
    implementation 'org.tensorflow:tensorflow-lite-gpu:2.6.0'

I am replacing the model in Object Detection Example app(Android App) provided by tensorflow, here is the code snippet that I use to run the inference on Bitmap:

public List<Classifier.Recognition> recognizeImage(final Bitmap bitmap) {
    bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());

    imgData.rewind();
    for (int i = 0; i < inputSizeWidth; ++i) {
      for (int j = 0; j < inputSizeHeight; ++j) {
        int pixelValue = intValues[i * inputSizeHeight + j];
        if (isModelQuantized) {
          // Quantized model
          imgData.put((byte) ((pixelValue >> 16) & 0xFF));
          imgData.put((byte) ((pixelValue >> 8) & 0xFF));
          imgData.put((byte) (pixelValue & 0xFF));
        } else { // Float model
          imgData.putFloat(((pixelValue >> 16) & 0xFF));
          imgData.putFloat(((pixelValue >> 8) & 0xFF));
          imgData.putFloat((pixelValue & 0xFF));
        }
      }
    }
    modelOutput = new float[1][6][56];
    xCoordinates = new float[17];
    yCoordinates = new float[17];
    confidences = new float[17];

    Object[] inputArray = {imgData};
    Map<Integer, Object> outputMap = new HashMap<>();
    outputMap.put(0, modelOutput);

    tfLite.runForMultipleInputsOutputs(inputArray, outputMap);  // <- this is where I get the error
    
    final ArrayList<Recognition> recognitions = new ArrayList<>(NUM_DETECTIONS);
    for (int i = 0; i < 6; ++i) {
      for(int x = 0; x < 17; x++) {
        yCoordinates[x] = modelOutput[0][i][0 + x];
        xCoordinates[x] = modelOutput[0][i][1 + x];
        confidences[x] = modelOutput[0][i][2 + x];
      }
        recognitions.add(new Recognition(xCoordinates, yCoordinates, confidences));
    }
    return recognitions;
  }

EDIT:
The model loads successfully, but the error is when I try run the model on the frame.

[mohantym]

Hi @ArtanBerisha1 ! Can you check this thread to change your image width , length and input shape accordingly(For example length 500 and height 250 ,buffer size 4x500x250x1 = 500000 , it can copy 49,15,20 bytes from 50,00,00 bytes) ? Thanks!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[Raaed-Ilham]

Hello
Im working on a similar project and ran across the same issue
Could you please let me know if you found any solution to this

[sushreebarsa]

@Raaed-Ilham Could you please create a new issue after filling all the details so that we can track the same?
Thank you!

[Raaed-Ilham]

@Raaed-Ilham Could you please create a new issue after filling all the details so that we can track the same? Thank you!

Done! Please take a look at it if possible
Thanks a lot
#66564