Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an example of FastViT model (Infernece) #826

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

cfgfung
Copy link
Contributor

@cfgfung cfgfung commented Mar 21, 2024

What does this PR do?

Added the example code of the validated FastViT.

Original performance:
n_iterations: 20
Total latency (ms): 451.754093170166
Average latency (ms): 22.5877046585083

BF16 + Graph mode:
n_iterations: 20
Total latency (ms): 47.97697067260742
Average latency (ms): 2.398848533630371

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@cfgfung cfgfung requested a review from regisss as a code owner March 21, 2024 17:56
@yeonsily
Copy link
Collaborator

@cfgfung Is this the same change with #814?

@cfgfung
Copy link
Contributor Author

cfgfung commented Mar 25, 2024

@cfgfung Is this the same change with #814?

@yeonsily This is another example. Look similar but not the same. Removed unrelated files. Thanks for the catch.

@cfgfung
Copy link
Contributor Author

cfgfung commented Mar 25, 2024

Add GPU Benchmark result:

A100 - 80GB , CUDA 12.2 , NVIDIA-SMI 535.54.03
With BF16 Autocast:
n_iterations: 20
Total latency (ms): 138.54217529296875
Average latency (ms): 6.9271087646484375

@cfgfung cfgfung requested a review from yeonsily March 29, 2024 16:06
@cfgfung
Copy link
Contributor Author

cfgfung commented May 15, 2024

Hi,

Some updates here:

  • Tested with 'make style' to align the code styles.
  • Added and updated related readmes.
  • Intentionally not adding test cases as this is a simple example
  • This is a TIMM model and does not belong to Transformer module. The current existing codes in run_image_classification.py are not compatible with TIMM model. Therefore I separate this into another file. run_timm_example.py.

image

@libinta
Copy link
Collaborator

libinta commented Jun 25, 2024

@cfgfung can you rebase and provide the latest test result. thanks

@cfgfung
Copy link
Contributor Author

cfgfung commented Jun 25, 2024

@cfgfung can you rebase and provide the latest test result. thanks

Hi,
I have applied rebase on this branch and added test cases.

ViT_TestCases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants