TomoGAN

Paper: arXiv:1902.07582; arXiv:1910.04081 Relevant presentation

An image quality enhancement model based on generative adversarial networks originally developed for synchrotron X-Ray tomography. It has also been used for other case like streaming tomography, the joint ptycho-tomography problem. We also developed a workflow to deploy TomoGAN on Coral edge TPU (arXiv:1911.05878) which can process an image with 1k x 1k pixels in 500ms.

I (Zhengchun Liu), also implemented the generator model based on pure Intel DNNL (MKL-DNN) and NVIDIA cudnn seperately for inference, using C++, so that you can easily integrate it in your C++ project. I will open source them soon.

To give it a try:

• download sample dataset and the ImageNet pretrained vgg model from Here

• install dependencies, Tensorflow(preferably 1.13)

• run with python ./main-gan.py -gpu 0 -expName test -xtrain dataset/noisy4train.h5 -ytrain dataset/clean4train.h5 -xtest dataset/noisy4test.h5 -ytest dataset/clean4test.h5

Note: with the sample dataset (128 images for training) provided here, you may not reproduce our results because we used a much larger dataset that has 2048 images. You may download the X-ray projection from TomoBank and make a reconstruction using all the projections as ground truth and a subsampled projections (every 32) as (low dose) noisy input. The dataset we provided, both norml dose (all projections) and low-dose (1/32 subsampled), were reconstructed using SIRT algorithms with 100 iterastions.

Citation

If you use this code for your research, please cite our paper(s):

@inproceedings{liu2019deep,
    title={Deep Learning Accelerated Light Source Experiments},
    author={Zhengchun Liu and Tekin Bicer and Rajkumar Kettimuthu and Ian Foster},
    year={2019},
    booktitle={2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)}
}

@misc{liu2019tomogan,
    title={TomoGAN: Low-Dose X-Ray Tomography with Generative Adversarial Networks},
    author={Zhengchun Liu and Tekin Bicer and Rajkumar Kettimuthu and Doga Gursoy and Francesco De Carlo and Ian Foster},
    year={2019},
    eprint={1902.07582},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

Example

As an example / teaser, a pair of noisy input and its denoised output are shown as follows,

An example input (a low-dose X-ray tomographic image ):

The corresponding output (i.e., denoised using TomoGAN):

Video clips

PhotoAcoustic Data

To train and test the model with photoacoustic data, you can use the following link: Photoacoustic data for tomoGAN model the file should be extracted into ./dataset/ folder which contains cropped and full-frame data. cropped folder includes images and also npy format of data where each frame is cropped from top (to exclude the nosiy top) and is split into left (index 0) and right (index 1).