TEM image restoration from fast image streams.

Microscopy imaging experiments generate vast amounts of data, and there is a high demand for smart acquisition and analysis methods. This is especially true for transmission electron microscopy (TEM) where terabytes of data are produced if imaging a full sample at high resolution, and analysis can t...

Full description

Bibliographic Details
Main Authors: Håkan Wieslander, Carolina Wählby, Ida-Maria Sintorn
Format: Article
Language:English
Published: Public Library of Science (PLoS) 2021-01-01
Series:PLoS ONE
Online Access:https://doi.org/10.1371/journal.pone.0246336
id doaj-bc5b4ad1bcb24234a492a72989955021
record_format Article
spelling doaj-bc5b4ad1bcb24234a492a729899550212021-07-29T04:33:11ZengPublic Library of Science (PLoS)PLoS ONE1932-62032021-01-01162e024633610.1371/journal.pone.0246336TEM image restoration from fast image streams.Håkan WieslanderCarolina WählbyIda-Maria SintornMicroscopy imaging experiments generate vast amounts of data, and there is a high demand for smart acquisition and analysis methods. This is especially true for transmission electron microscopy (TEM) where terabytes of data are produced if imaging a full sample at high resolution, and analysis can take several hours. One way to tackle this issue is to collect a continuous stream of low resolution images whilst moving the sample under the microscope, and thereafter use this data to find the parts of the sample deemed most valuable for high-resolution imaging. However, such image streams are degraded by both motion blur and noise. Building on deep learning based approaches developed for deblurring videos of natural scenes we explore the opportunities and limitations of deblurring and denoising images captured from a fast image stream collected by a TEM microscope. We start from existing neural network architectures and make adjustments of convolution blocks and loss functions to better fit TEM data. We present deblurring results on two real datasets of images of kidney tissue and a calibration grid. Both datasets consist of low quality images from a fast image stream captured by moving the sample under the microscope, and the corresponding high quality images of the same region, captured after stopping the movement at each position to let all motion settle. We also explore the generalizability and overfitting on real and synthetically generated data. The quality of the restored images, evaluated both quantitatively and visually, show that using deep learning for image restoration of TEM live image streams has great potential but also comes with some limitations.https://doi.org/10.1371/journal.pone.0246336
collection DOAJ
language English
format Article
sources DOAJ
author Håkan Wieslander
Carolina Wählby
Ida-Maria Sintorn
spellingShingle Håkan Wieslander
Carolina Wählby
Ida-Maria Sintorn
TEM image restoration from fast image streams.
PLoS ONE
author_facet Håkan Wieslander
Carolina Wählby
Ida-Maria Sintorn
author_sort Håkan Wieslander
title TEM image restoration from fast image streams.
title_short TEM image restoration from fast image streams.
title_full TEM image restoration from fast image streams.
title_fullStr TEM image restoration from fast image streams.
title_full_unstemmed TEM image restoration from fast image streams.
title_sort tem image restoration from fast image streams.
publisher Public Library of Science (PLoS)
series PLoS ONE
issn 1932-6203
publishDate 2021-01-01
description Microscopy imaging experiments generate vast amounts of data, and there is a high demand for smart acquisition and analysis methods. This is especially true for transmission electron microscopy (TEM) where terabytes of data are produced if imaging a full sample at high resolution, and analysis can take several hours. One way to tackle this issue is to collect a continuous stream of low resolution images whilst moving the sample under the microscope, and thereafter use this data to find the parts of the sample deemed most valuable for high-resolution imaging. However, such image streams are degraded by both motion blur and noise. Building on deep learning based approaches developed for deblurring videos of natural scenes we explore the opportunities and limitations of deblurring and denoising images captured from a fast image stream collected by a TEM microscope. We start from existing neural network architectures and make adjustments of convolution blocks and loss functions to better fit TEM data. We present deblurring results on two real datasets of images of kidney tissue and a calibration grid. Both datasets consist of low quality images from a fast image stream captured by moving the sample under the microscope, and the corresponding high quality images of the same region, captured after stopping the movement at each position to let all motion settle. We also explore the generalizability and overfitting on real and synthetically generated data. The quality of the restored images, evaluated both quantitatively and visually, show that using deep learning for image restoration of TEM live image streams has great potential but also comes with some limitations.
url https://doi.org/10.1371/journal.pone.0246336
work_keys_str_mv AT hakanwieslander temimagerestorationfromfastimagestreams
AT carolinawahlby temimagerestorationfromfastimagestreams
AT idamariasintorn temimagerestorationfromfastimagestreams
_version_ 1721259432511275008