npj Computational Materials (Sep 2022)

Microstructure segmentation with deep learning encoders pre-trained on a large microscopy dataset

  • Joshua Stuckner,
  • Bryan Harder,
  • Timothy M. Smith

DOI
https://doi.org/10.1038/s41524-022-00878-5
Journal volume & issue
Vol. 8, no. 1
pp. 1 – 12

Abstract

Read online

Abstract This study examined the improvement of microscopy segmentation intersection over union accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures were trained on over 100,000 labeled microscopy images from 54 material classes. These pre-trained encoders were then embedded into multiple segmentation architectures including UNet and DeepLabV3+ to evaluate segmentation performance on created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2% reduction in relative intersection over union error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.