The brain magnetic resonance images can present the anatomical information noninvasively for more accurate diagnosis and exploration of brain. However, constrained by the physical properties, contraindication and cost, it is challenging to acquire adequate resolution medical image clinically. The super-resolution technology, that provides an economic and efficient approach for reconstructing the high-resolution images from the low-resolution counterparts, has attracted lots of attention and make some success. 

However, the existing deep learning SR models attempt to stack convolutional neural networks (CNN) as deep as possible to fit the sophisticated mapping from low-resolution (LR) images to high-resolution (HR) images. According to the frequency principle (F-Principle), deep neural networks tend to capture the low-frequency (LF) components first and then capture the high-frequency (HF) ones slowly. Therefore, purely stacking CNN layers would limit the sensitivity of models for the anatomical details and finally lead to oversmoothed SR results. 

To deal with the challenges of the super-resolution of brain MR images, a research team led by Prof. WANG Shuqiang from the Shenzhen Institute of Advanced Technology (SIAT) of Chinese Academy of Sciences proposed a fine perceptive generative adversarial network (FP-GANs) (Fig.1).  

The result was published in the IEEE Transaction of Neural Network and Learning Systems. 

Fig.1 Architecture of FP-GANs. (Image by SIAT) 

Benefiting from the divide-and-conquer manner which divides the MR images into LF and HF components with discrete wavelet transformation and super-resolves the sub-band images independently with four GANs, the sensitivity of the proposed model to the fine anatomical structures of brain is enhance. Besides, this strategy simplifies the SR task for one single GAN and hence accelerates and stabilizes the training procedure. Furthermore, a novel attention that can adaptively trade off the weight among the sub-band images is proposed to enhance the anatomical textures. 

Comprehensive experiments on the MultiRes_7T and ADNI datasets demonstrated that the proposed model achieves finer structure recovery and outperforms the competing methods quantitatively and qualitatively (Fig.2). Moreover, FP-GANs further show the value by applying the SR results in classification tasks (Fig. 3). 

Fig.2 Visual comparison of detail recovery performance on the MultiRes_7T dataset. (Image by SIAT) 

Fig.3 Comparison of auto-diagnosis performance with low-resolution MR images and SR MR images on ADNI dataset. (Image by SIAT) 

Media Contact:
ZHANG Xiaomin
Email:xm.zhang@siat.ac.cn