Frontiers in Neuroscience (Sep 2022)

Self-supervised learning for modal transfer of brain imaging

  • Dapeng Cheng,
  • Dapeng Cheng,
  • Chao Chen,
  • Mao Yanyan,
  • Mao Yanyan,
  • Panlu You,
  • Xingdan Huang,
  • Jiale Gai,
  • Feng Zhao,
  • Feng Zhao,
  • Ning Mao

DOI
https://doi.org/10.3389/fnins.2022.920981
Journal volume & issue
Vol. 16

Abstract

Read online

Today's brain imaging modality migration techniques are transformed from one modality data in one domain to another. In the specific clinical diagnosis, multiple modal data can be obtained in the same scanning field, and it is more beneficial to synthesize missing modal data by using the diversity characteristics of multiple modal data. Therefore, we introduce a self-supervised learning cycle-consistent generative adversarial network (BSL-GAN) for brain imaging modality transfer. The framework constructs multi-branch input, which enables the framework to learn the diversity characteristics of multimodal data. In addition, their supervision information is mined from large-scale unsupervised data by establishing auxiliary tasks, and the network is trained by constructing supervision information, which not only ensures the similarity between the input and output of modal images, but can also learn valuable representations for downstream tasks.

Keywords