Skip to content

Hybrid Spectral Denoising Transformer with Guided Attention (ICCV 2023)

License

Notifications You must be signed in to change notification settings

Zeqiang-Lai/HSDT

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HSDT

PWC PWC

Official PyTorch Implementation of Hybrid Spectral Denoising Transformer with Guided Attention. ICCV 2023

Zeqiang Lai, Chenggang Yan, Ying Fu.

🌟 Hightlights

  • Superior hybrid spectral denoising transformer (HSDT), powered by
    • a novel 3D guided spectral self-attention (GSSA),
    • 3D spectral-spatial seperable convolution (S3Conv), and
    • self-modulated feed-forward network (SM-FFN).
  • Super fast convergence
    • 1 epoch to reach 39.5 PSNR on ICVL Gaussian 50.
    • 3 epochs surpasses QRNN3D trained with 30 epochs.
  • Super lightweight
    • HSDT-S achieves comparable performance against the SOTA with only 0.13M parameters.
    • HSDT-M outperforms the SOTA by a large margin with only 0.52M parameters.

🤗 See Also

  • MAN : Another superior HSI denoising network based on RNN.
  • DPHSIR : Plug-and-play ADMM that utilize HSI denoiser for unified HSI restoration without any training.
  • HSIR : Out-of-box HSI denoising training, testing, and visualization framework.

Usage

Download the pretrained model at Github Release.

  • Training, testing, and visualize results with HSIR.
python -m hsirun.test -a hsdt.hsdt -r ckpt/man_gaussian.pth -t icvl_512_30 icvl_512_50
python -m hsirun.train -a hsdt.hsdt -s schedule.gaussian
python -m hsirun.train -a hsdt.hsdt -s schedule.complex -r checkpoints/hsdts.hsdt/model_latest.pth
python -m hsiboard.app --logdir results
  • Using our model.
import torch
from hsdt import hsdt

net = hsdt()
x = torch.randn(4,1,31,64,64)
y = net(x)
  • Using our components.
import torch
from hsdt import (
    S3Conv
)

x = torch.randn(4,16,31,64,64)
block = S3Conv(16, 16, 3, 1, 1)
out = block(x) # [4,16,31,64,64]

Tips for training

  • use xavier_normal_ weight initialization.

Performance

Gaussian denoising
Complex denoising
Real/CAVE denoising
Comparsion with other methods

Citation

@inproceedings{lai2023hsdt,
  author = {Lai, Zeqiang and Chenggang, Yan and Fu, Ying},
  title = {Hybrid Spectral Denoising Transformer with Guided Attention},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  year = {2023},
}

About

Hybrid Spectral Denoising Transformer with Guided Attention (ICCV 2023)

Resources

License

Stars

Watchers

Forks

Packages

No packages published