DPT 3.1 models are now available in the Transformers library

[NielsRogge]

Hi folks!

DPT 3.1 (and the older DPT 3.0) models are now available for use in the Transformers library! These include:

• DPT with a BEiT backbone: https://huggingface.co/models?search=dpt%20beit. Best performance, but also the most heavy models.
• DPT with a Swinv2 backbone: https://huggingface.co/models?search=dpt%20swinv2. Great performance, better for speed trade-offs.
• the older DPT 3.0 model is available here: https://huggingface.co/Intel/dpt-large

Inference can be done in a few lines of code (using the pipeline API):

from transformers import pipeline

pipe = pipeline(task="depth-estimation", model="Intel/dpt-beit-base-384")
result = pipe("http://images.cocodataset.org/val2017/000000039769.jpg")
result["depth"]

The documentation of DPT can be found here: https://huggingface.co/docs/transformers/model_doc/dpt.

Example notebooks can be found here: https://github.com/NielsRogge/Transformers-Tutorials/tree/master/DPT.

[FabianSchuetze]

Thanks for updating the DPT models in transformers.

Can you explain what the corresponding scratch network would be in hf/Transformers?

Edit: I saw this file: https://github.com/huggingface/transformers/blob/main/src/transformers/models/dpt/convert_dpt_beit_to_hf.py which explains the relationship between the models.