Xiaoming Liu
Skip Abstract Section
Abstract
Infrared imaging is currently the most effective clinical method to evaluate the morphology of the meibomian glands (MGs) in patients. As an important indicator for monitoring the development of MG dysfunction, it is necessary to accurately measure gland-drop and gland morphology. Although there are existing methods for automatic segmentation of MGs using deep learning frameworks, they require fully annotated ground-truth labels for training, which is time-consuming and laborious. In this article, we proposed a new scribble-supervised deep learning framework for segmenting the MGs, which only requires easily attainable scribble annotations for training. To cope with the shortage of supervision and regularize the network, a transformation consistent strategy is incorporated, which requires the prediction to follow the same transformation if a transform is performed on an input image of the network. The proposed segmentation method consists of two stages. In the first stage, a U-Net network is used to obtain the meibomian region segmentation map. In the second stage, we concentrate on segmenting glands in the meibomian region. We utilize the gradient prior information of the original image at the decoder part of the segmentation network, which can coarsely locate the target contour. We automatically generate reliable labels using the exponential moving average of the predictions during training and filter out the unreliable pseudo-label by uncertainty threshold. Experimental results on a local MG dataset and two other public medical image datasets demonstrate the effectiveness of the proposed segmentation framework.
- [1] . 1996. Meibomian gland dysfunction. 40, 5 (1996), 343–367.Google Scholar
- [2] . 2005. Comparison of ultrahigh- and standard-resolution optical coherence tomography for imaging macular pathology. Ophthalmology 111, 11 (2005), 2033–2043.Google ScholarCross Ref
- [3] . 2021. Global prevalence of meibomian gland dysfunction: A systematic review and meta-analysis. Ocular Immunology and Inflammation 29, 1 (2021), 66–75.Google ScholarCross Ref
- [4] . 2011. The international workshop on meibomian gland dysfunction: Report of the subcommittee on anatomy, physiology, and pathophysiology of the meibomian gland. Investigative Ophthalmology & Visual Science 52, 4 (2011), 1938–78.Google ScholarCross Ref
- [5] 2022. Weakly supervised segmentation of COVID19 infection with scribble annotation on CT images. Pattern Recognition 122 (2022), 108341.Google ScholarDigital Library
- [6] . 2021. An evolvable adversarial network with gradient penalty for COVID-19 infection segmentation. Applied Soft Computing 113 (2021), 107947.Google ScholarDigital Library
- [7] . 2015. Safety and efficacy of lacrimal drainage system plugs for dry eye syndrome a report by the American academy of ophthalmology. Ophthalmology 122, 8 (2015), 1681–1687.Google ScholarCross Ref
- [8] . 2019. Automated layer segmentation of retinal optical coherence tomography images using a deep feature enhanced structured random forests classifier. IEEE Journal of Biomedical and Health Informatics 23, 4 (2019), 1404–1416.Google ScholarCross Ref
- [9] . 2008. Noncontact Infrared meibography to document age-related changes of the meibomian glands in a normal population. Ophthalmology 115, 5 (2008), 911–915.Google ScholarCross Ref
- [10] . 2019. Comparison of meibomian gland dropout using two infrared imaging devices. Contact Lens and Anterior Eye 42, 3 (2019), 311–317.Google ScholarCross Ref
- [11] . 2003. Meibomian gland dysfunction: A clinical scheme for description, diagnosis, classification, and grading. Ocular Surface 1, 3 (2003), 107–126.Google ScholarCross Ref
- [12] . 2012. Detection of meibomian glands and classification of meibography images. Journal of Biomedical Optics 17, 8 (2012), 086008.Google ScholarCross Ref
- [13] . 2014. Objective image analysis of the meibomian gland area. British Journal of Ophthalmology 98, 6 (2014), 746–755.Google ScholarCross Ref
- [14] . 2021. Automatic fluid segmentation in retinal optical coherence tomography images using attention based deep learning. Neurocomputing 452 (2021), 576–591.Google ScholarCross Ref
- [15] . 2020. Multimodal MR image synthesis using gradient prior and adversarial learning. IEEE Journal of Selected Topics in Signal Processing 14, 6 (2020), 1–1.Google ScholarCross Ref
- [16] . 2020. Deep learning segmentation and quantification of Meibomian glands. Biomedical Signal Processing and Control 57 (2020), 101776.Google ScholarCross Ref
- [17] . 2020. Scribble2Label: Scribble-supervised cell segmentation via self-generating pseudo-labels with consistency. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 14–23.Google ScholarDigital Library
- [18] . 2020. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20).Google ScholarCross Ref
- [19] A. Tarvainen and H. Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17).Google Scholar
- [20] D. T. Nguyen, C. K. Mummadi, T. Ngo, T. Nguyen, L. Beggel, and T. Brox. 2020. SELF: Learning to Filter Noisy Labels with Self-Ensembling. In 2020 International Conference on Learning Representations (ICLR).Google Scholar
- [21] . 2015. U-Net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 234–241.Google ScholarCross Ref
- [22] . 2021. Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Transactions on Neural Networks and Learning Systems 32, 2 (2021), 523–534.Google ScholarCross Ref
- [23] . 2017. Mass classification of benign and malignant with a new twin support vector machine joint l2,1-norm. International Journal of Machine Learning & Cybernetics 10, 5 (2017), 1–17.Google Scholar
- [24] . 2017. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 12 (2017), 2481–2495.Google ScholarCross Ref
- [25] . 2021. MDC-Net: A new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information. Computers in Biology and Medicine 135 (2021), 104543.Google ScholarDigital Library
- [26] X. Liu, J. Cao, S. Wang, Y. Zhang, and M. Wang. 2021. Confidence-guided topology-preserving layer segmentation for optical coherence tomography images with focus-column module. IEEE Transactions on Instrumentation and Measurement 70 (2021), 5005612.Google Scholar
- [27] . 2021. A densely connected network based on U-Net for medical image segmentation. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 3 (2021), 1–14.Google ScholarDigital Library
- [28] . 2019. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis 54 (2019), 30–44.Google ScholarCross Ref
- [29] . 2021. Residual refinement network with attribute guidance for precise saliency detection. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 3 (2021), 1–19.Google ScholarDigital Library
- [30] . 2019. A deep learning approach for meibomian gland atrophy evaluation in meibography images. Translational Vision Science Technology 8, 6 (2019), 37.Google ScholarCross Ref
- [31] . 2017. Pyramid scene parsing network. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 2881–2890.Google ScholarCross Ref
- [32] . 2021. Image based analysis of meibomian gland dysfunction using conditional generative adversarial neural network. BMJ Open Ophthalmology 6, 1 (2021), e000436.Google ScholarCross Ref
- [33] . 2017. ReLayNet: Retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomedical Optics Express 8, 8 (2017), 3627.Google ScholarCross Ref
- [34] . 2018. DRUNET: A dilated-residual U-Net deep learning network to segment optic nerve head tissues in optical coherence tomography images. Biomedical Optics Express 9, 7 (2018), 3244–3265.Google ScholarCross Ref
- [35] . 2015. From image-level to pixel-level labeling with convolutional networks. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15).Google ScholarCross Ref
- [36] . 2016. Built-in foreground/ background prior for weakly-supervised semantic segmentation. In European Conference on Computer Vision. Springer, 413–432.Google ScholarCross Ref
- [37] . 2021. Railroad is not a train: Saliency as pseudo-pixel supervision for weakly supervised semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5495–5505.Google ScholarCross Ref
- [38] . 2016. Simple does It: Weakly supervised instance and semantic segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).Google Scholar
- [39] . 2021. Semi-supervised semantic segmentation with cross pseudo supervision. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2613–2622.Google ScholarCross Ref
- [40] . 2016. What's the point: Semantic segmentation with point supervision. In European Conference on Computer Vision.Google ScholarCross Ref
- [41] . 2018. Where are the blobs: Counting by localization with point supervision. In European Conference on Computer Vision (ECCV’18). 547–562.Google ScholarCross Ref
- [42] . 2016. ScribbleSup: Scribble-supervised convolutional networks for semantic segmentation. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).Google Scholar
- [43] . 2019. Boundary perception guidance: A scribble-supervised semantic segmentation approach. In 28th International Joint Conference on Artificial Intelligence (IJCAI’19).Google ScholarCross Ref
- [44] . 2019. Scribbles for metric learning in histological image segmentation. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’19). IEEE, 1026–1030.Google ScholarCross Ref
- [45] . 2020. Self-supervised nuclei segmentation in histopathological images using attention. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 393–402.Google ScholarDigital Library
- [46] . 2019. Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 605–613.Google ScholarDigital Library
- [47] . 2016. ITK-SNAP: An interactive tool for semi-automatic segmentation of multi-modality biomedical images. In Engineering in Medicine & Biology Society.Google Scholar
- [48] . 1998. Evaluation of subjective assessments and objective diagnostic tests for diagnosing tear-film disorders known to cause ocular irritation. Cornea, 17, 1 (1998), 38–56.Google ScholarCross Ref
- [49] . 2005. An assessment of grading scales for meibography images. Cornea 24, 4 (2005), 382–388.Google ScholarCross Ref
- [50] . 2012. Comparison of subjective grading and objective assessment in meibography. Contact Lens & Anterior Eye: The Journal of the British Contact Lens Association 36, 1 (2012), 22–27.Google ScholarCross Ref
- [51] . 2012. Annotated high-throughput microscopy image sets for validation. Nature Methods 9, 7 (2012), 637–637.Google ScholarCross Ref
- [52] . 2020. Latest evidences on meibomian gland dysfunction diagnosis and management. The Ocular Surface 18, 4 (2020), 871–892.Google ScholarCross Ref
- [53] D. Kingma and J. Ba, 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR'15).Google Scholar
- [54] . 2018. On regularized losses for weakly-supervised CNN segmentation. In European Conference on Computer Vision (ECCV’18). 507–522.Google ScholarCross Ref
- [55] . 2020. Weakly-supervised salient object detection via scribble annotations. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20).Google ScholarCross Ref
- [56] . 2017. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Transactions on Medical Imaging 36, 7 (2017), 1550–1560.Google ScholarCross Ref
Index Terms
- Scribble-Supervised Meibomian Glands Segmentation in Infrared Images
Recommendations
Weakly Supervised Segmentation of COVID19 Infection with Scribble Annotation on CT Images
Highlights- We propose a weakly supervised COVID-19 infection segmentation method with scribble supervision. To the best of our knowledge, it is the first work adopting ...
AbstractSegmentation of infections from CT scans is important for accurate diagnosis and follow-up in tackling the COVID-19. Although the convolutional neural network has great potential to automate the segmentation task, most existing deep ...
Scribble-Based Hierarchical Weakly Supervised Learning for Brain Tumor Segmentation
Medical Image Computing and Computer Assisted Intervention – MICCAI 2019AbstractThe recent state-of-the-art deep learning methods have significantly improved brain tumor segmentation. However, fully supervised training requires a large amount of manually labeled masks, which is highly time-consuming and needs domain ...
Comments