Skip to content

This repository summarizes the material gathered for the tutorial on learning disentangled representations in the imaging domain, and serves as a roadmap for the disentanglement aficionados.

Notifications You must be signed in to change notification settings

vios-s/disentanglement_tutorial

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

49 Commits
 
 
 
 

Repository files navigation

applications

Visual summary of disentangled representation learning applications in medical imaging. Red connections indicate vector-based disentanglement, while blue connections indicate tensor/vector-based one (CSD). The visual examples are taken from the papers and repositories of the applications that are reported in the Tables below.

Applications of Disentanglement

Exemplar methods that exploit disentanglement to improve challenging tasks in computer vision and medical image analysis.

NOTE: The scope of this tutorial is not to survey every existing application for the to domains. If you feel that we should include your research as part of this tutorial we would be happy to do so. Please send an email to any of the contributor and help us create a place where everyone can understand and leverage disentanglement!

Computer Vision

For each application we denote the type of disentanglement as either "Vector" or content-style "C-S".

Task Model Abbrev. Paper (link to PDF) Repository Framework Type Original Implementation
I2I translation MUNIT pdf repo PyTorch C-S
Face Attribute Transfer ELEGANT pdf repo PyTorch Vector
Pose Estimation n/a pdf repo TensorFlow C-S
Point cloud Generation SetVAE pdf repo PyTorch Vector

Medical Image Analysis

For each application we denote the type of disentanglement as either "Vector" or content-style "C-S".

Task Model Abbrev. Paper (link to PDF) Repository Framework Type Original Implementation
Single-modal Segmentation SDNet pdf repo Keras C-S
Brain Synthesis n/a pdf repo Keras C-S
Causal Image Synthesis n/a pdf repo PyTorch C-S
Classification MIDNET pdf repo Tensorflow Vector

Note that the SDNet model has also been implemented in PyTorch and has been integrated into the GaNDLF framework.

Metrics

Measuring disentanglement in single vector latent variables:

Property Paper (link to PDF) Repository
Modularity pdf
pdf
pdf
repo
repo
repo
Compactness / Completeness pdf
pdf
pdf
pdf
repo
repo
repo
repo
Linear separability pdf repo
Explicitness pdf
pdf
repo
repo
Consistency & Restrictiveness pdf repo
Robustness pdf repo

Measuring disentanglement between two latent variables of the same or different dimensionality:

Metric Paper (link to PDF Repository
Distance Correlation & Information over Bias pdf repo
Kernel-target alignment pdf repo
Hilbert-Schmidt independence criterion pdf repo

Citation

@article{liu2022learning,
 title={Learning disentangled representations in the imaging domain},
 author={Liu, Xiao and Sanchez, Pedro and Thermos, Spyridon and O’Neil, Alison Q and Tsaftaris, Sotirios A},
 journal={Medical Image Analysis},
 pages={102516},
 year={2022},
 publisher={Elsevier}
}

About

This repository summarizes the material gathered for the tutorial on learning disentangled representations in the imaging domain, and serves as a roadmap for the disentanglement aficionados.

Topics

Resources

Stars

Watchers

Forks