Data Discovery in Images Using a Median-Absolute-Deviation Based Filter

Abstract

The utility of a recently developed, novel, computational digital image curation tool called SOCKS (Statistical Outlier Curation Kernel Software) is demonstrated in this paper. Raw image-data of the James Webb Space Telescope (JWST) is used for the tests. Data is decontaminated through an iterative, parametrically tuned sequence of operations involving the conditional flagging of outliers, followed by the curation of the flagged pixels. Every iteration produces a decontaminated image and an outlier image, till the sequence converges asymptotically producing a final decontaminated image that represents the ground-truth as accurately as possible. The intermediate outlier images are shown to contain a wealth of information about the original image. An iteration-by-iteration approach of deriving insight out of the intermediate outlier images is described in detail. Traditional filtering methods, in contrast, discard outliers causing irreversible loss of information contained therein.