Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TFLite Movenet multipose/lightning input error #53127

Closed
ArtanBerisha1 opened this issue Nov 19, 2021 · 8 comments
Closed

TFLite Movenet multipose/lightning input error #53127

ArtanBerisha1 opened this issue Nov 19, 2021 · 8 comments
Assignees
Labels
2.6.0 comp:lite TF Lite related issues stale This label marks the issue/pr stale - to be closed automatically if no activity stat:awaiting response Status - Awaiting response from author type:others issues not falling in bug, perfromance, support, build and install or feature

Comments

@ArtanBerisha1
Copy link

ArtanBerisha1 commented Nov 19, 2021

Hi,
I am trying to load and run the Movenet multipose model, the loading of model works fine but when I try to run it gives this error:
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_input:0) with 3 bytes from a Java Buffer with 491520 bytes.

As I can understand (and also I looked the model graph in Netron app) the input of the model is type: uint8[1,1,1,3] , and this is what the error is also telling, that my converted image doesn't fit the input of the model. So my question is: is this intentionally or the model is not correct ?

I downloaded model from Tensorflow Hub -> https://tfhub.dev/google/lite-model/movenet/multipose/lightning/tflite/float16/1
And here (in the link above) also is specified that the model accepts 'A frame of video or an image, represented as an int32 tensor of dynamic shape: 1xHxWx3'. I handled the dynamic shape tensor as suggested in the link.

Thank you!

@ArtanBerisha1 ArtanBerisha1 added the type:others issues not falling in bug, perfromance, support, build and install or feature label Nov 19, 2021
@sushreebarsa sushreebarsa added the comp:lite TF Lite related issues label Nov 19, 2021
@sushreebarsa
Copy link
Contributor

sushreebarsa commented Nov 22, 2021

@ArtanBerisha1
In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here & let us know which TF version you're using ?Thanks!

@sushreebarsa sushreebarsa added the stat:awaiting response Status - Awaiting response from author label Nov 22, 2021
@ArtanBerisha1
Copy link
Author

ArtanBerisha1 commented Nov 22, 2021

Hi, @sushreebarsa
Here are TF versions:

    implementation 'org.tensorflow:tensorflow-lite:2.6.0'
    implementation 'org.tensorflow:tensorflow-lite-gpu:2.6.0'

I am replacing the model in Object Detection Example app(Android App) provided by tensorflow, here is the code snippet that I use to run the inference on Bitmap:

public List<Classifier.Recognition> recognizeImage(final Bitmap bitmap) {
    bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());

    imgData.rewind();
    for (int i = 0; i < inputSizeWidth; ++i) {
      for (int j = 0; j < inputSizeHeight; ++j) {
        int pixelValue = intValues[i * inputSizeHeight + j];
        if (isModelQuantized) {
          // Quantized model
          imgData.put((byte) ((pixelValue >> 16) & 0xFF));
          imgData.put((byte) ((pixelValue >> 8) & 0xFF));
          imgData.put((byte) (pixelValue & 0xFF));
        } else { // Float model
          imgData.putFloat(((pixelValue >> 16) & 0xFF));
          imgData.putFloat(((pixelValue >> 8) & 0xFF));
          imgData.putFloat((pixelValue & 0xFF));
        }
      }
    }
    modelOutput = new float[1][6][56];
    xCoordinates = new float[17];
    yCoordinates = new float[17];
    confidences = new float[17];

    Object[] inputArray = {imgData};
    Map<Integer, Object> outputMap = new HashMap<>();
    outputMap.put(0, modelOutput);

    tfLite.runForMultipleInputsOutputs(inputArray, outputMap);  // <- this is where I get the error
    
    final ArrayList<Recognition> recognitions = new ArrayList<>(NUM_DETECTIONS);
    for (int i = 0; i < 6; ++i) {
      for(int x = 0; x < 17; x++) {
        yCoordinates[x] = modelOutput[0][i][0 + x];
        xCoordinates[x] = modelOutput[0][i][1 + x];
        confidences[x] = modelOutput[0][i][2 + x];
      }
        recognitions.add(new Recognition(xCoordinates, yCoordinates, confidences));
    }
    return recognitions;
  }

EDIT:
The model loads successfully, but the error is when I try run the model on the frame.

@sushreebarsa sushreebarsa added 2.6.0 and removed stat:awaiting response Status - Awaiting response from author labels Nov 22, 2021
@jvishnuvardhan jvishnuvardhan added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Dec 3, 2021
@mohantym mohantym self-assigned this Apr 26, 2022
@mohantym
Copy link
Contributor

mohantym commented May 10, 2022

Hi @ArtanBerisha1 ! Can you check this thread to change your image width , length and input shape accordingly(For example length 500 and height 250 ,buffer size 4x500x250x1 = 500000 , it can copy 49,15,20 bytes from 50,00,00 bytes) ? Thanks!

@mohantym mohantym added stat:awaiting response Status - Awaiting response from author and removed stat:awaiting tensorflower Status - Awaiting response from tensorflower labels May 10, 2022
@google-ml-butler
Copy link

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

@google-ml-butler google-ml-butler bot added the stale This label marks the issue/pr stale - to be closed automatically if no activity label May 17, 2022
@google-ml-butler
Copy link

Closing as stale. Please reopen if you'd like to work on this further.

@Raaed-Ilham
Copy link

Hello
Im working on a similar project and ran across the same issue
Could you please let me know if you found any solution to this

@sushreebarsa
Copy link
Contributor

@Raaed-Ilham Could you please create a new issue after filling all the details so that we can track the same?
Thank you!

@Raaed-Ilham
Copy link

@Raaed-Ilham Could you please create a new issue after filling all the details so that we can track the same? Thank you!

Done! Please take a look at it if possible
Thanks a lot
#66564

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.6.0 comp:lite TF Lite related issues stale This label marks the issue/pr stale - to be closed automatically if no activity stat:awaiting response Status - Awaiting response from author type:others issues not falling in bug, perfromance, support, build and install or feature
Projects
None yet
Development

No branches or pull requests

5 participants