Noisy-Correspondence Learning Summary (Updating)

A new research direction of label noise learning. Noisy correspondence learning aims to eliminate the negative impact of the mismatched pairs (e.g., false positives/negatives) instead of annotation errors in several tasks.

We mark works contributed by ourselves with ⭐.

This repository now is maintained by Mouxing Yang, Yijie Lin, and Yang Qin. We hope more AI-workers join us and thank all contributors!

Tasks

Image-Text Matching/Retrieval	Vision-Language Pre-training
Re-identification	Video-Text Learning
Image Captioning	Image Contrastive Learning
Graph Matching	Visual-Audio Learning
Machine Reading Comprehension	Dense Retrieval
Multi-View Clustering

Image-Text Matching/Retrieval

2024

• [2024 IJCV] ⭐Learning with Noisy Correspondence
  Zhenyu Huang, Peng Hu, Guocheng Niu, Xinyan Xiao, Jiancheng Lv, Xi Peng
  [paper]

• [2024 TOIS] Breaking Through the Noisy Correspondence: A Robust Model for Image-Text Matching
  Haitao Shi, Meng Liu, Xiaoxuan Mu, Xuemeng Song, Yupeng Hu, Liqiang Nie
  [paper]

• [2024 ICASSP] NAC: Mitigating Noisy Correspondence in Cross-Modal Matching Via Neighbor Auxiliary Corrector
  Yuqing Li, Haoming Huang, Jian Xu, Shao-Lun Huang
  [paper]

• [2024 CVPR] Robust Noisy Correspondence Learning with Equivariant Similarity Consistency
  Yuchen Yang, Likai Wang, Erkun Yang, Cheng Deng
  [paper]

• [2024 CVPR] Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning
  Zihua Zhao, Mengxi Chen, Tianjie Dai, Jiangchao Yao, Bo han, Ya Zhang, Yanfeng Wang
  [paper] [code]

• [2024 CVPR] Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval
  Haochen Han, Qinghua Zheng, Guang Dai, Minnan Luo, Jingdong Wang
  [paper] [code]

• [2024 CVPR] Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning
  Zihua Zhao, Mengxi Chen, Tianjie Dai, Jiangchao Yao, Bo Han, Ya Zhang, Yanfeng Wang
  [paper]

• [2024 CVPR] Robust Noisy Correspondence Learning with Equivariant Similarity Consistency
  Yuchen Yang, Erkun Yang, Likai Wang, Cheng Deng
  [paper]

• [2024 Arxiv] REPAIR: Rank Correlation and Noisy Pair Half-replacing with Memory for Noisy Correspondence
  Ruochen Zheng, Jiahao Hong, Changxin Gao, Nong Sang
  [paper]

• [2024 TIP] ⭐Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining
  Xinran Ma, Mouxing Yang, Yunfan Li, Peng Hu, Jiancheng Lv, Xi Peng
  [paper] [code]

• [2024 AAAI] Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation
  Zhuohang Dang, Minnan Luo, Chengyou Jia, Guang Dai, Xiaojun Chang, Jingdong Wang
  [paper]

• [2024 AAAI] Negative Pre-aware for Noisy Cross-modal Matching
  Xu Zhang, Hao Li, Mang Ye
  [paper] [code]

2023

• [2023 NeurIPS] ⭐Cross-modal Active Complementary Learning with Self-refining Correspondence
  Yang Qin and Yuan Sun and Dezhong Peng and Joey Tianyi Zhou and Xi Peng and Peng Hu
  [paper] [code]

• [2023 TPAMI] ⭐Cross-Modal Retrieval with Partially Mismatched Pairs
  Peng Hu, Zhenyu Huang, Dezhong Peng, Xu Wang, Xi Peng
  [paper] [code]

• [2023 TMM] Integrating Language Guidance Into Image-Text Matching for Correcting False Negatives
  Zheng Li, Caili Guo, IEEE, Zerun Feng, Jenq-Neng Hwang, Zhongtian Du
  [paper]

• [2023 TMM] Learning From Noisy Correspondence With Tri-Partition for Cross-Modal Matching
  Feng, Zerun and Zeng, Zhimin and Guo, Caili and Li, Zheng and Hu, Lin
  [paper]

• [2023 CVPR] BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency
  Shuo Yang, Zhapan XU, Kai Wang, Yang You, Hongxun Yao, Tongliang Liu, Min Xu
  [paper] [code]

• [2023 CVPR] MSCN: Noisy Correspondence Learning with Meta Similarity Correction
  Han, Haochen and Miao, Kaiyao and Zheng, Qinghua and Luo, Minnan
  [paper] [code]

2022

• [2022 ACMMM] ⭐Deep Evidential Learning with Noisy Correspondence for Cross-Modal Retrieval
  Qin, Yang and Peng, Dezhong and Peng, Xi and Wang, Xu and Hu, Peng
  [paper] [code]

2021

• [2021 NeurIPS Oral] ⭐Learning with Noisy Correspondence for Cross-modal Matching
  Huang, Zhenyu and Niu, Guocheng and Liu, Xiao and Ding, Wenbiao and Xiao, Xinyan and Wu, Hua and Peng, Xi
  [paper] [code]

Vision-Language Pre-training

• [2023 AAAI] NLIP: Noise-Robust Language-Image Pre-training
  Runhui Huang, Yanxin Long, Jianhua Han, Hang Xu, Xiwen Liang, Chunjing Xu, Xiaodan Liang
  [paper]

• [2022 CVPR] Robust Cross-Modal Representation Learning with Progressive Self-Distillation Andonian, Alex and Chen, Shixing and Hamid, Raffay
  [paper]

• [2022 ICML] Blip: Bootstrapping Language-image Pre-training for Unified Vision-language Understanding and Generation
  Li, Junnan and Li, Dongxu and Xiong, Caiming and Hoi, Steven*
  [paper] [code]

• [2021 NeurIPS Spotlight] Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
  Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, Steven Chu Hong Hoi
  [paper] [code]

Re-identification

• [2024 CVPR] ⭐Noisy-Correspondence Learning for Text-to-Image Person Re-identification
  Qin, Yang and Chen, Yingke and Peng, Dezhong and Peng, Xi and Zhou, Joey Tianyi and Hu, Peng
  [paper] [code]

• [2024 IJCV] ⭐Robust Object Re-identification with Coupled Noisy Labels
  Mouxing Yang, Zhenyu Huang, Xi Peng
  [paper] [code]

• [2024 EAAI] Modality Blur and Batch Alignment Learning for Twin Noisy Labels-based Visible–infrared Person Re-identification
  Song Wu, Shihao Shan, Guoqiang Xiao, Michael S. Lew, Xinbo Gao
  [paper] [code]

• [2022 CVPR] ⭐Learning With Twin Noisy Labels for Visible-Infrared Person Re-Identification
  Mouxing Yang, Zhenyu Huang, Peng Hu, Taihao Li, Jiancheng Lv, Xi Peng
  [paper] [code]

Video-Text Learning

• [2024 ICLR Oral] ⭐Multi-granularity Correspondence Learning from Long-term Noisy Videos
  Yijie Lin, Jie Zhang, Zhenyu Huang, Jia Liu, Zujie Wen, Xi Peng
  [paper] [code]

• [2024 Arxiv] A Strong Baseline for Temporal Video-Text Alignment
  Li, Zeqian and Chen, Qirui and Han, Tengda and Zhang, Ya and Wang, Yanfeng and Xie, Weidi
  [paper]

• [2023 TMM] Robust Video-Text Retrieval Via Noisy Pair Calibration
  Zhang, Huaiwen and Yang, Yang and Qi, Fan and Qian, Shengsheng and Xu, Changsheng
  [paper]

• [2021 ICCV] Crossclr: Cross-modal Contrastive Learning for Multi-modal Video Representations
  Zolfaghari, Mohammadreza and Zhu, Yi and Gehler, Peter and Brox, Thomas
  [paper]

• [2022 CVPR Oral] Temporal Alignment Networks for Long-term Video
  Han, Tengda and Xie, Weidi and Zisserman, Andrew
  [paper] [code]

• [2021 EMNLP] Videoclip: Contrastive Pre-training for Zero-shot Video-text Understanding Xu, Hu and Ghosh, Gargi and Huang, Po-Yao and Okhonko, Dmytro and Aghajanyan, Armen and Metze, Florian and Zettlemoyer, Luke and Feichtenhofer, Christoph
  [paper] [code]

Image Captioning

• [2024 AAAI] Noise-Aware Image Captioning with Progressively Exploring Mismatched Words Zhongtian Fu, Kefei Song, Luping Zhou, Yang Yang
  [paper] [code]

• [2022 CVPR] Noise-aware Learning from Web-crawled Image-Text Data for Image Captioning
  Wooyoung Kang, Jonghwan Mun, Sungjun Lee, Byungseok Roh
  [paper] [code]

Image Contrastive Learning

• [2022 CVPR] Robust contrastive learning against noisy views
  Ching-Yao Chuang, R Devon Hjelm, Xin Wang, Vibhav Vineet, Neel Joshi, Antonio Torralba, Stefanie Jegelka, Yale Song
  [paper] [code]

Graph Matching

• [2024 TIP] ⭐Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining
  Xinran Ma, Mouxing Yang, Yunfan Li, Peng Hu, Jiancheng Lv, Xi Peng
  [paper] [code]
• [2023 ICCV] ⭐Graph Matching with Noisy Correspondence
  Lin, Yijie and Yang, Mouxing and Yu, Jun and Hu, Peng and Zhang, Changqing and Peng, Xi
  [paper] [code]

Visual-Audio Learning

• [2024 TMM] Noise-Tolerant Learning for Audio-Visual Action Recognition
  Haochen Han, Qinghua Zheng, Minnan Luo, Kaiyao Miao, Feng Tian and Yan Chen
  [paper]

Machine Reading Comprehension

• [2023 AAAI] ⭐Robust domain adaptation for machine reading comprehension
  Jiang, Liang and Huang, Zhenyu and Liu, Jia and Wen, Zujie and Peng, Xi
  [paper]

Dense Retrieval

• [2023 EMNLP Findings] Noisy Pair Corrector for Dense Retrieval
  Hang Zhang, Yeyun Gong, Xingwei He, Dayiheng Liu, Daya Guo, Jiancheng Lv, Jian Guo
  [paper]

Multi-View Clustering

• [2024 TKDE] ⭐Robust Multi-View Clustering with Noisy Correspondence
  Yuan Sun, Yang Qin, Yongxiang Li, Dezhong Peng, Xi Peng, Peng Hu

• [2024 AAAI] ⭐Decoupled Contrastive Multi-view Clustering with High-order Random Walks
  Yiding Lu, Yijie Lin, Mouxing Yang, Dezhong Peng, Peng Hu, Xi Peng
  [paper] [code]

• [2022 TPAMI] ⭐Robust Multi-View Clustering With Incomplete Information
  Mouxing Yang, Yunfan Li, Peng Hu, Jinfeng Bai, Jiancheng Lv, Xi Peng
  [paper] [code]

• [2021 CVPR] ⭐Partially View-aligned Representation Learning with Noise-robust Contrastive Loss
  Mouxing Yang, Yunfan Li, Zhenyu Huang, Zitao Liu, Peng Hu, Xi Peng
  [paper] [code]

• [2020 NeurIPS Oral] ⭐Partially View-aligned Clustering
  Zhenyu Huang, Peng Hu, Joey Tianyi Zhou, Jiancheng Lv, and Xi Peng
  [paper] [code]