Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an example of Segment Anything Model [Inference] #814

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Add an example of Segment Anything Model [Inference] #814

wants to merge 12 commits into from

Conversation

cfgfung
Copy link
Contributor

@cfgfung cfgfung commented Mar 18, 2024

What does this PR do?

Add an example code of SegmentAnythingModel using Graph mode and BF16.

Original FP32:
n_iterations: 20
Total latency (ms): 10380.430936813354
Average latency (ms): 519.0215468406677

Enabled graph mode and BF16:
n_iterations: 20
Total latency (ms): 1639.5680904388428
Average latency (ms): 81.97840452194214

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@cfgfung cfgfung requested a review from regisss as a code owner March 18, 2024 18:09
@cfgfung cfgfung changed the title Raymond/segmentanythingmodel Add an example of Segment Anything Model Mar 18, 2024
@cfgfung cfgfung changed the title Add an example of Segment Anything Model Add an example of Segment Anything Model [Inference] Mar 18, 2024
@cfgfung
Copy link
Contributor Author

cfgfung commented Mar 21, 2024

@regisss
This is a validated model and this is an example script for users. No core code has been changed.

yeonsily
yeonsily reviewed Mar 21, 2024
View reviewed changes
examples/object-segementation/ClipSeg/run_example.py Outdated
model.to("hpu")

with torch.no_grad(), autocast:
for i in range(args.warmup):
Copy link
Collaborator

@yeonsily yeonsily Mar 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we change this warmup step similar to https://github.com/huggingface/optimum-habana/blob/main/optimum/habana/transformers/trainer.py#L881?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may not apply to this case. The code in the link is for 'training'. There are some variables dedicated for training.
For example: args.gradient_accumulation_steps , epoch * steps_in_epoch
These should not be included in this 'inference' example.

yeonsily
yeonsily reviewed Mar 21, 2024
View reviewed changes
examples/object-segementation/SegmentAnythingModel/run_example.py Outdated
model.to("hpu")

with torch.no_grad(), autocast:
for i in range(args.warmup):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replied above.

@yeonsily
Copy link
Collaborator

@cfgfung Can you please provide GPU benchmark and add ci tests?

This was referenced Mar 21, 2024
Open
Merged
@cfgfung
Copy link
Contributor Author

cfgfung commented Mar 25, 2024 •
edited
Loading

@cfgfung Can you please provide GPU benchmark and add ci tests?

@yeonsily

Sure. Is there any reference for the CI test? This is just a validated example. I checked another merged example (2f55de3) and that does not require CI.

GPU Benchmark:
A100-80GB , CUDA12.2, Driver Version: 535.54.03
With BF16 Autocast:
n_iterations: 20
Total latency (ms): 2248.983144760132
Average latency (ms): 112.44915723800659

@cfgfung cfgfung requested a review from yeonsily April 5, 2024 17:19
@cfgfung cfgfung force-pushed the raymond/segmentanythingmodel branch from 28a2549 to 144c499 Compare April 15, 2024 20:44
@jiminha
Copy link
Collaborator

jiminha commented Apr 18, 2024

@cfgfung Your code looks good. Can you just modify your test file name to something like test_image_segementation.py . Also, please make sure your test runs on CI test.

optimum-habana/Makefile

Line 45 in 8786b75

slow_tests_1x: test_installs

@cfgfung
Copy link
Contributor Author

cfgfung commented Apr 22, 2024

@cfgfung Your code looks good. Can you just modify your test file name to something like test_image_segementation.py . Also, please make sure your test runs on CI test.

optimum-habana/Makefile

Line 45 in 8786b75

slow_tests_1x: test_installs

Thanks for the review. I have renamed the test file and add few lines of codes to the makefile for the CI tests.

@jiminha
Copy link
Collaborator

jiminha commented Apr 23, 2024

Could you delete old file : test_modelenabling.py?

Also I ran your tests on the latest, and I'm getting this error on the last test. Do you see this passing on Docker 1.15.0?
tests/test_image_segmentation.py::GaudiSAMTester::test_inference_default PASSED
tests/test_image_segmentation.py::GaudiSAMTester::test_inference_hpu_graphs PASSED
tests/test_image_segmentation.py::GaudiSAMTester::test_no_latency_regression_bf16 FAILED

FAILED tests/test_image_segmentation.py::GaudiSAMTester::test_no_latency_regression_bf16 - AssertionError: 81.58655166625977 not greater than or equal to 93.97604942321777

jiminha
jiminha reviewed Apr 24, 2024
View reviewed changes
examples/object-segementation/SegmentAnythingModel/run_example.py Outdated

processor = SamProcessor.from_pretrained(args.model_name_or_path)
model = SamModel.from_pretrained(args.model_name_or_path)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead have this code specific to SAM model, can you use AutoProcessor, AutoModel and have this script generic?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, plz remove this specific SAM folder(we don't have specific model folder in all examples), and move the script under object-segmentation/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. These two are addressed in the new commit.

@cfgfung
Copy link
Contributor Author

cfgfung commented Apr 30, 2024

Could you delete old file : test_modelenabling.py?

Also I ran your tests on the latest, and I'm getting this error on the last test. Do you see this passing on Docker 1.15.0? tests/test_image_segmentation.py::GaudiSAMTester::test_inference_default PASSED tests/test_image_segmentation.py::GaudiSAMTester::test_inference_hpu_graphs PASSED tests/test_image_segmentation.py::GaudiSAMTester::test_no_latency_regression_bf16 FAILED

FAILED tests/test_image_segmentation.py::GaudiSAMTester::test_no_latency_regression_bf16 - AssertionError: 81.58655166625977 not greater than or equal to 93.97604942321777

Removed the file.
Somehow it passed on my side. I fixed the logic of that test. It should be able to pass now.

@jiminha
Copy link
Collaborator

jiminha commented May 21, 2024

@cfgfung can you merge your change to the latest baseline?

@cfgfung cfgfung force-pushed the raymond/segmentanythingmodel branch from 455d728 to 5f7b15d Compare May 29, 2024 21:03
@cfgfung
Copy link
Contributor Author

cfgfung commented May 29, 2024

@cfgfung can you merge your change to the latest baseline?

Hi,
I have applied a git rebase. It should be aligned with the latest baseline.

@libinta
Copy link
Collaborator

libinta commented Jun 25, 2024

@cfgfung can you rebase and provide the test result on latest?

@cfgfung
Aligned the file architecture by moving the files under object-segmen…
5cd8f2b 
…tation.

Used Automodel and related processor to replace model-specific API.
Improved the testing logic.
@cfgfung cfgfung force-pushed the raymond/segmentanythingmodel branch from ea58674 to b398475 Compare June 25, 2024 17:40
@cfgfung
Copy link
Contributor Author

cfgfung commented Jun 25, 2024

@cfgfung can you rebase and provide the test result on latest?

Hi,

I have applied rebase and ran the tests.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jiminha jiminha jiminha left review comments

@regisss regisss Awaiting requested review from regisss regisss is a code owner

@yeonsily yeonsily Awaiting requested review from yeonsily

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@cfgfung @yeonsily @jiminha @libinta