Mamba-in-Computer-Vision

Mamba-in-Vision

A paper list of some recent Mamba-based CV works. If you find some ignored papers, please open issues or pull requests.

**Last updated: 2024/05/15

Mamba

• (arXiv 2023.12) Mamba: Linear-Time Sequence Modeling with Selective State Spaces, [Paper], [Code]

Survey

• (arXiv 2024.04) Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges, [Paper], [Project]
• (arXiv 2024.04) A Survey on Visual Mamba, [Paper]
• (arXiv 2024.04) State Space Model for New-Generation Network Alternative to Transformers: A Survey, [Paper], [Project]
• (arXiv 2024.05) A Survey on Vision Mamba: Models, Applications and Challenges, [Paper], [Project]
• (arXiv 2024.05) Vision Mamba: A Comprehensive Survey and Taxonomy, [Paper], [Project]

Recent Papers

Action

• (arXiv 2024.03) HARMamba: Efficient Wearable Sensor Human Activity Recognition Based on Bidirectional Selective SSM, [Paper]
• (arXiv 2024.04) Simba: Mamba augmented U-ShiftGCN for Skeletal Action Recognition in Videos, [Paper]

Adversarial Attack

• (arXiv 2024.03) Understanding Robustness of Visual State Space Models for Image Classification, [Paper]

Anomaly Detection

• (arXiv 2024.04) MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection, [Paper], [Code]

Autonomous Driving

• (arXiv 2024.05) DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving, [Paper]

Classification (Backbone)

• (arXiv 2024.01) Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model, [Paper], [Code]
• (arXiv 2024.01) VMamba: Visual State Space Model, [Paper], [Code]
• (arXiv 2024.02) Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining, [Paper], [Code]
• (arXiv 2024.02) Res-VMamba: Fine-Grained Food Category Visual Classification Using Selective State Space Models with Deep Residual Learning, [Paper],[Code]
• (arXiv 2024.02) Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data, [Paper]
• (arXiv 2024.03) LocalMamba: Visual State Space Model with Windowed Selective Scan, [Paper], [Code]
• (arXiv 2024.03) EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba, [Paper], [Code]
• (arXiv 2024.03) On the low-shot transferability of [V]-Mamba, [Paper]
• (arXiv 2024.03) SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series, [Paper], [Code]
• (arXiv 2024.03) PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition, [Paper],[Code]
• (arXiv 2024.03) MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selection, [Paper],[Code]

Crowd Counting

• (arXiv 2024.05) VMambaCC: A Visual State Space Model for Crowd Counting, [Paper]

Deblurring

• (arXiv 2024.03) Aggregating Local and Global Features via Selective State Spaces Model for Efficient Image Deblurring, [Paper],[Code]

Dehazing

• (arXiv 2024.02) U-shaped Vision Mamba for Single Image Dehazing, [Paper],[Code]

Deraining

• (arXiv 2024.04) FreqMamba: Viewing Mamba from a Frequency Perspective for Image Deraining, [Paper]

Detection

• (arXiv 2024.03) MiM-ISTD: Mamba-in-Mamba for Efficient Infrared Small Target Detection, [Paper],[Code]
• (arXiv 2024.04) Fusion-Mamba for Cross-modality Object Detection, [Paper]
• (arXiv 2024.04) CFMW: Cross-modality Fusion Mamba for Multispectral Object Detection under Adverse Weather Conditions, [Paper],[Code]
• (arXiv 2024.05) SOAR: Advancements in Small Body Object Detection for Aerial Imagery Using State Space Models and Programmable Gradients, [Paper],[Code]

Diffusion

• (arXiv 2024.03) ZigMa: Zigzag Mamba Diffusion Model, [Paper],[Code]

Domain

• (arXiv 2024.04) DGMamba: Domain Generalization via Generalized State Space Model, [Paper],[Code]

Enhancement

• (arXiv 2024.04) MambaUIE&SR: Unraveling the Ocean's Secrets with Only 2.8 FLOPs, [Paper],[Code]
• (arXiv 2024.05) Retinexmamba: Retinex-based Mamba for Low-light Image Enhancement, [Paper],[Code]
• (arXiv 2024.05) WaterMamba: Visual State Space Model for Underwater Image Enhancement, [Paper]

Event Cameras

• (arXiv 2024.02) State Space Models for Event Cameras, [Paper]
• (arXiv 2024.04) MambaPupil: Bidirectional Selective Recurrent model for Event-based Eye tracking, [Paper]

Face

• (arXiv 2024.05) FER-YOLO-Mamba: Facial Expression Detection and Classification Based on Selective State Space, [Paper],[Code]

Fusion

• (arXiv 2024.04) FusionMamba: Efficient Image Fusion with State Space Model, [Paper]
• (arXiv 2024.04) MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion, [Paper]
• (arXiv 2024.04) FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba, [Paper]
• (arXiv 2024.04) A Novel State Space Model with Local Enhancement and State Sharing for Image Fusion, [Paper]

Gesture

• (arXiv 2024.03) MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models, [Paper]

Graph

• (arXiv 2024.01) Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces, [Paper],[Code]
• (arXiv 2024.02) Graph Mamba: Towards Learning on Graphs with State Space Models, [Paper],[Code]

Hyperpsectral

• (arXiv 2024.04) HSIMamba: Hyperpsectral Imaging Efficient Feature Learning with Bidirectional State Space for Classification, [Paper]
• (arXiv 2024.04) SpectralMamba: Efficient Mamba for Hyperspectral Image Classification, [Paper],[Code]
• (arXiv 2024.04) HSIDMamba: Exploring Bidirectional State-Space Models for Hyperspectral Denoising, [Paper],[Code]
• (arXiv 2024.05) Spectral-Spatial Mamba for Hyperspectral Image Classification, [Paper]
• (arXiv 2024.05) S2Mamba: A Spatial-spectral State Space Model for Hyperspectral Image Classification, [Paper],[Code]
• (arXiv 2024.05) SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising, [Paper],[Code]

LLM

• (arXiv 2024.03) DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models, [Paper],[Code]

Medical

• (arXiv 2024.01) U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation, [Paper], [Code]
• (arXiv 2024.01) SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation, [Paper], [Code]
• (arXiv 2024.01) Vivim: a Video Vision Mamba for Medical Video Object Segmentation, [Paper], [Code]
• (arXiv 2024.01) MambaMorph: a Mamba-based Backbone with Contrastive Feature Learning for Deformable MR-CT Registration, [Paper], [Code]
• (arXiv 2024.02) VM-UNet: Vision Mamba UNet for Medical Image Segmentation, [Paper],[Code]
• (arXiv 2024.02) nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Model,[Paper],[Code]
• (arXiv 2024.02) FD-Vision Mamba for Endoscopic Exposure Correction, [Paper]
• (arXiv 2024.02) Semi-Mamba-UNet: Pixel-Level Contrastive Cross-Supervised Visual Mamba-based UNet for Semi-Supervised Medical Image Segmentation, [Paper],[Code]
• (arXiv 2024.02) Marrying Perona Malik Diffusion with Mamba for Efficient Pediatric Echocardiographic Left Ventricular Segmentation,[[Paper]
• (arXiv 2024.02) Weak-Mamba-UNet: Visual Mamba Makes CNN and ViT Work Better for Scribble-based Medical Image Segmentation,[Paper],[Code]
• (arXiv 2024.03) MedMamba: Vision Mamba for Medical Image Classification,[Paper],[Code]
• (arXiv 2024.03) MambaMIR: An Arbitrary-Masked Mamba for Joint Medical Image Reconstruction and Uncertainty Estimation，[Paper],[Code]
• (arXiv 2024.03) MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models,[Paper]
• (arXiv 2024.03) LightM-UNet: Mamba Assists in Lightweight UNet for Medical Image Segmentation,[Paper],[Code]
• (arXiv 2024.03) MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology,[Paper],[Code]
• (arXiv 2024.03) VM-UNET-V2 Rethinking Vision Mamba UNet for Medical Image Segmentation,[Paper],[Code]
• (arXiv 2024.03) MD-Dose: A Diffusion Model based on the Mamba for Radiotherapy Dose Prediction,[Paper],[Code]
• (arXiv 2024.03) Large Window-based Mamba UNet for Medical Image Segmentation: Beyond Convolution and Self-attention,[Paper],[Code]
• (arXiv 2024.03) ProMamba: Prompt-Mamba for polyp segmentation,[Paper],[Code]
• (arXiv 2024.03) H-vmunet: High-order Vision Mamba UNet for Medical Image Segmentation,[Paper],[Code]
• (arXiv 2024.03) Rotate to Scan: UNet-like Mamba with Triplet SSM Module for Medical Image Segmentation,[Paper]
• (arXiv 2024.03) Integrating Mamba Sequence Model and Hierarchical Upsampling Network for Accurate Semantic Segmentation of Multiple Sclerosis Legion,[Paper]
• (arXiv 2024.03) UltraLight VM-UNet: Parallel Vision Mamba Significantly Reduces Parameters for Skin Lesion Segmentation,[Paper],[Code]
• (arXiv 2024.04) T-Mamba: Frequency-Enhanced Gated Long-Range Dependency for Tooth 3D CBCT Segmentation,[Paper],[Code]
• (arXiv 2024.04) ViM-UNet: Vision Mamba for Biomedical Segmentation,[Paper],[Code]
• (arXiv 2024.04) SurvMamba: State Space Model with Multi-grained Multi-modal Interaction for Survival Prediction,[Paper]
• (arXiv 2024.04) Vim4Path: Self-Supervised Vision Mamba for Histopathology Images,[Paper],[Code]
• (arXiv 2024.04) Sparse Reconstruction of Optical Doppler Tomography Based on State Space Model,[Paper]
• (arXiv 2024.05) AC-MAMBASEG: An adaptive convolution and Mamba-based architecture for enhanced skin lesion segmentation,[Paper],[Code]
• (arXiv 2024.05) HC-Mamba: Vision MAMBA with Hybrid Convolutional Techniques for Medical Image Segmentation,[Paper]
• (arXiv 2024.05) VM-DDPM: Vision Mamba Diffusion for Medical Image Synthesis,[Paper]

Mesh

• (arXiv 2024.05) HandSSCA: 3D Hand Mesh Reconstruction with State Space Channel Attention from RGB images,[Paper]

Mixture of Experts

• (arXiv 2024.01) MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts, [Paper]
• (arXiv 2024.01) BlackMamba: Mixture of Experts for State-Space Models, [Paper], [Code]

Motion

• (arXiv 2024.03) Motion Mamba: Efficient and Long Sequence Motion Generation with Hierarchical and Bidirectional Selective SSM, [Paper], [Code]
• (arXiv 2024.04) Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion, [Paper]
• (arXiv 2024.04) HumMUSS: Human Motion Understanding using State Space Models, [Paper]
• (arXiv 2024.04) MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model, [Paper], [Code]
• (arXiv 2024.05) SMCD: High Realism Motion Style Transfer via Mamba-based Diffusion, [Paper]

Multimodal

• (arXiv 2024.03) VL-Mamba: Exploring State Space Models for Multimodal Learning，[Paper],[Code]

OOD

• (arXiv 2024.05) CLIP-Mamba: CLIP Pretrained Mamba Models with OOD and Hessian Evaluation，[Paper],[Code]

OCR

• (arXiv 2024.01) LOCOST: State-Space Models for Long Document Abstractive Summarization, [Paper],[Code]

Point Cloud

• (arXiv 2024.02) PointMamba: A Simple State Space Model for Point Cloud Analysis, [Paper],[Code]
• (arXiv 2024.02) Point Could Mamba: Point Cloud Learning via State Space Model, [Paper],[Code]
• (arXiv 2024.03) Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategy, [Paper],[Code]
• (arXiv 2024.04) 3DMambaComplete: Exploring Structured State Space Model for Point Cloud Completion, [Paper]
• (arXiv 2024.04) Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model, [Paper]

Recognition

• (arXiv 2024.05) MemoryMamba: Memory-Augmented State Space Model for Defect Recognition, [Paper]
• (arXiv 2024.05) OverlapMamba: Novel Shift State Space Model for LiDAR-based Place Recognition, [Paper]

Reconstruction

• (arXiv 2024.03) Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction, [Paper]
• (arXiv 2024.05) GMSR:Gradient-Guided Mamba for Spectral Reconstruction from RGB Images, [Paper],[Code]

Referring

• (arXiv 2024.03) ReMamber: Referring Image Segmentation with Mamba Twister, [Paper]

Registration

• (arXiv 2024.04) VMambaMorph: a Visual Mamba-based Framework with Cross-Scan Module for Deformable 3D Image Registration, [Paper],[Code]

Remote Sensing

• (arXiv 2024.03) RSMamba: Remote Sensing Image Classification with State Space Model, [Paper],[Code]
• (arXiv 2024.04) RS-Mamba for Large Remote Sensing Image Dense Prediction, [Paper],[Code]
• (arXiv 2024.04) RS3Mamba: Visual State Space Model for Remote Sensing Images Semantic Segmentation, [Paper],[Code]
• (arXiv 2024.04) Samba: Semantic Segmentation of Remotely Sensed Images with State Space Model, [Paper],[Code]
• (arXiv 2024.04) ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model, [Paper],[Code]
• (arXiv 2024.05) RSCaMa: Remote Sensing Image Change Captioning with State Space Model, [Paper],[Code]
• (arXiv 2024.05) Frequency-Assisted Mamba for Remote Sensing Image Super-Resolution, [Paper]
• (arXiv 2024.05) Rethinking Scanning Strategies with Vision Mamba in Semantic Segmentation of Remote Sensing Imagery: An Experimental Study, [Paper]

Restoration

• (arXiv 2024.02) A Simple Baseline for Image Restoration with State-Space Model, [Paper],[Code]
• (arXiv 2024.03) VmambaIR: Visual State Space Model for Image Restoration, [Paper],[Code]
• (arXiv 2024.03) Serpent: Scalable and Efficient Image Restoration via Multi-scale Structured State Space Models, [Paper]

Semantic Segmentation

• (arXiv 2024.04) Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation, [Paper],[Code]

Spatiotemporal Forecasting

• (arXiv 2024.03) VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting, [Paper],[Code]

State Space Model (SSM)

• (NeurIPS 2020) HiPPO: Recurrent Memory with Optimal Polynomial Projections, [Paper],[Code]
• (ICLR 2022) Efficiently Modeling Long Sequences with Structured State Spaces, [Paper],[Code]
• (ICLR 2023) Hungry Hungry Hippos: Toward Language Modeling with State Space Models, [Paper],[Code]
• (arXiv 2024.01) MambaByte: Token-free Selective State Space Model, [Paper],[Code]
• (arXiv 2024.02) Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks, [Paper]
• (arXiv 2024.02) Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling, [Paper],[Code]
• (arXiv 2024.05) MambaOut: Do We Really Need Mamba for Vision, [Paper],[Code]

Style Transfer

• (arXiv 2024.05) StyleMamba : State Space Model for Efficient Text-driven Image Style Transfer, [Paper]

Super-Resolution

• (arXiv 2024.05) DVMSR: Distillated Vision Mamba for Efficient Super-Resolution, [Paper],[Code]

Tracking

• (arXiv 2024.05) Mamba-FETrack: Frame-Event Tracking via State Space Model, [Paper],[Code]

Video

• (arXiv 2024.03) VideoMamba: State Space Model for Efficient Video Understanding, [Paper],[Code]
• (arXiv 2024.03) Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding, [Paper],[Code]
• (arXiv 2024.03) SSM Meets Video Diffusion Models: Efficient Video Generation with Structured State Spaces, [Paper],[Code]
• (arXiv 2024.04) SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Grounding, [Paper]
• (arXiv 2024.05) Matten: Video Generation with Mamba-Attention, [Paper]

Other

• (arXiv 2024.02) Pan-Mamba: Effective pan-sharpening with State Space Model, [Paper],[Code]
• (arXiv 2024.04) InsectMamba: Insect Pest Classification with State Space Model, [Paper]

Contact & Feedback

If you have any suggestions about this project, feel free to contact me.

• [e-mail: yzhangcst[at]gmail.com]