Masks not aligned with image

[AaronLopes]

Went through the notebook to generate embeddings for dogs.jpg. However, when hovering over the image the mask is offset above the original image. I have tried this with various images and the mask is always offset or skewed in some way. I tried running through the demo on a clean repo and still ran into this issue. I suspect it has something to with the scaled down model being displayed (despite console.log telling me that image and maskImage are the same dimensions). Anyone have any pointers?

[deepakanand14]

@AaronLopes Were you able to solve this?

[VincentLimbach]

If you really didn't modify anything in the code than this shouldn't happen. However, I encountered something similar, so did you dynamically derive the input_size from the transformed image? If you look into the SAM Predictor class they take the last two shape entries after transform which is necessary to derive the correct upscaled masks

[mnmarinaccio]

My guess is something to do with mismatched image or mask shapes at some point. Can you verify the shape of the input image and the shape of the output mask from SAM to make sure they are the same? If they are, then it would be an issue with how you are outputting the images on a plot visually.

[pypae]

I had the same issue and was able to fix the demo by changing the orig_im_size in export_onnx_model.py to the dimensions of dogs.jpg.

I wasn't able to set the image size dynamically for an onnx exported model yet.