Skip to main content
SpringerLink
Log in

Detail-semantic guide network based on spatial attention for surface defect detection with fewer samples

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Surface defect detection is an important part of the process of product quality control in the industry. Automatic detection of surface defects based on machine learning is an up-and-coming research field, and there have been many successful cases. Deep learning has become the most suitable detection method for this task. Most algorithms require a large number of defect samples to achieve good results. However, defect samples in actual production are very limited. Although some unsupervised or semi-supervised methods can reduce training costs, their accuracy is difficult to guarantee, so they are difficult to be applied in industrial inspection. In this paper, we propose a detail-semantic guide network (DSGNet), which can achieve better result with fewer training samples. It is a two-stage neural network framework. In the first stage, we design a new semantic branch based on the modified residual shrinkage network and the proposed joined atrous spatial pyramid pooling (JASPP) module. This is the first time that residual shrinkage network is applied to defect detection and achieves good results. Also, we design a clear and efficient detail branch based on dense connection network. Specially, we propose a new detail-semantic guide module (DSGM), which can better integrate the feature information of the two branches. In the training phase, we propose a weight mask based on defect area to improve the ability of extracting small defects. We did experiments on four datasets and our method achieved excellent detection results even with only a small number of training samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Code Availability

Not applicable, The code will be available on github in future.

References

  1. Kwak J, Lee KB, Jang J, Chang KS, Kim CO (2019) Automatic inspection of salt-and-pepper defects in oled panels using image processing and control chart techniques. J Intell Manuf 30(3):1047–1055. https://doi.org/10.1007/s10845-017-1304-8

    Article  Google Scholar 

  2. Qiu Y, Tang L, Li B, Niu S, Niu T (2020) Uneven illumination surface defects inspection based on saliency detection and intrinsic image decomposition. IEEE Access 8:190663–190676. https://doi.org/10.1109/access.2020.3032108

    Article  Google Scholar 

  3. Liu L et al (2019) Planar eddy current sensor array with null-offset. IEEE Sens J 19(12):4647–4651. https://doi.org/10.1109/jsen.2019.2901351

    Article  Google Scholar 

  4. Yuan F, Yu Y, Wang W, Xue K, Tian G (2021) Pulsed eddy current array design and electromagnetic imaging for defects detection in metallic materials. Nondestruct Test Evaluation 37 (1):81–99. https://doi.org/10.1080/10589759.2021.1909012

    Article  Google Scholar 

  5. Dua G, Arora V, Mulaveesala R (2021) Defect detection capabilities of pulse compression based infrared non-destructive testing and evaluation. IEEE Sens J 21(6):7940–7947. https://doi.org/10.1109/jsen.2020.3046320

    Article  Google Scholar 

  6. Liu J, Zhang Z, Lin Z, Chen H, Yin W (2021) Characterization method of surface crack based on laser thermography. IEEE Access 9:76395–76402. https://doi.org/10.1109/access.2021.3081435

    Article  Google Scholar 

  7. Wu Q, Ding K, Huang B (2018) Approach for fault prognosis using recurrent neural network. J Intell Manuf 31(7):1621–1633. https://doi.org/10.1007/s10845-018-1428-5

    Article  Google Scholar 

  8. Badmos O, Kopp A, Bernthaler T, Schneider G (2019) Image-based defect detection in lithium-ion battery electrode using convolutional neural networks. J Intell Manuf 31(4):885–897. https://doi.org/10.1007/s10845-019-01484-x

    Article  Google Scholar 

  9. Çelik A, Küçükmanisa A, Sümer A, Çelebi AT, Urhan O (2020) A real-time defective pixel detection system for lcds using deep learning based object detectors. J Intell Manuf, https://doi.org/10.1007/s10845-020-01704-9

  10. Wang Y, Zhou J, Zheng L, Gogu C (2020) An end-to-end fault diagnostics method based on convolutional neural network for rotating machinery with multiple case studies. J Intell Manuf, https://doi.org/10.1007/s10845-020-01671-1

  11. Djavadifar A, Graham-Knight JB, Körber M, Lasserre P, Najjaran H (2021) Automated visual detection of geometrical defects in composite manufacturing processes using deep convolutional neural networks. J Intell Manuf, https://doi.org/10.1007/s10845-021-01776-1

  12. Tabernik D, Šela S, Skvarč J, Skočaj D (2019) Segmentation-based deep-learning approach for surface-defect detection. J Intell Manuf 31(3):759–776. https://doi.org/10.1007/s10845-019-01476-x

    Article  Google Scholar 

  13. Liu R et al (2020) Real-time defect detection network for polarizer based on deep learning. J Intell Manuf 31(8):1813–1823. https://doi.org/10.1007/s10845-020-01536-7

    Article  Google Scholar 

  14. Cui L et al (2021) Sddnet: a fast and accurate network for surface defect detection. IEEE Trans Instrum Meas 70:1–13. https://doi.org/10.1109/tim.2021.3056744

    Article  Google Scholar 

  15. Niu T, Li B, Li W, Qiu Y, Niu S (2022) Positive-sample-based surface defect detection using memory-augmented adversarial autoencoders. IEEE/ASME Trans Mechatron 27(1):46–57. https://doi.org/10.1109/tmech.2021.3058147

    Article  Google Scholar 

  16. Jiang L, Wang Y, Tang Z, Miao Y, Chen S (2021) Casting defect detection in x-ray images using convolutional neural networks and attention-guided data augmentation. Measurement 170. https://doi.org/10.1016/j.measurement.2020.108736

  17. Qiu L, Wu X, Yu Z (2019) A high-efficiency fully convolutional networks for pixel-wise surface defect detection. IEEE Access 7:15884–15893. https://doi.org/10.1109/access.2019.2894420

    Article  Google Scholar 

  18. Liu G, Yang N, Guo L (2020) An attention-based network for textured surface anomaly detection. Appl Sci 10(18). https://doi.org/10.3390/app10186215

  19. Zhao M, Zhong S, Fu X, Tang B, Pecht M (2020) Deep residual shrinkage networks for fault diagnosis. IEEE Trans Industr Inform 16(7):4681–4690. https://doi.org/10.1109/tii.2019.2943898

    Article  Google Scholar 

  20. Weimer D, Scholz-Reiter B, Shpitalni M (2016) Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Annals 65(1):417–420. https://doi.org/10.1016/j.cirp.2016.04.072

    Article  Google Scholar 

  21. Božič J, Tabernik D, Skočaj D (2021) Mixed supervision for surface-defect detection: from weakly to fully supervised learning. Computers in Industry 129. https://doi.org/10.1016/j.compind.2021.103459https://doi.org/10.1016/j.compind.2021.103459

  22. Huang Y, Qiu C, Yuan K (2018) Surface defect saliency of magnetic tile. The Visual Computer 36(1):85–96. https://doi.org/10.1007/s00371-018-1588-5

    Article  MathSciNet  Google Scholar 

  23. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation. Lect Notes Comput Sci 9351:234–241

    Article  Google Scholar 

  24. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2020) Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867. https://doi.org/10.1109/TMI.2019.2959609. https://www.ncbi.nlm.nih.gov/pubmed/31841402

    Article  Google Scholar 

  25. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  26. Lin G, Milan A, Shen C, Reid I (2017) Refinenet: Multi-path refinement networks for high-resolution semantic segmentation, pp 5168–5177

  27. Yu CQ et al (2018) BiSeNet: bilateral segmentation network for real-time semantic segmentation. Lect Notes Comput Sci 11217:334–349

    Article  Google Scholar 

  28. Peng C, Tian T, Chen C, Guo X, Ma J (2021) Bilateral attention decoder: A lightweight decoder for real-time semantic segmentation. Neural Netw 137:188–199. https://doi.org/10.1016/j.neunet.2021.01.021

    Article  Google Scholar 

  29. Yang MY, Kumaar S, Lyu Y, Nex F (2021) Real-time semantic segmentation with context aggregation network. ISPRS J Photogramm Remote Sens 178:124–134. https://doi.org/10.1016/j.isprsjprs.2021.06.006

    Article  Google Scholar 

  30. Cai J et al (2021) Real-time semantic segmentation of remote sensing images based on bilateral attention refined network. IEEE Access 9:28349–28360. https://doi.org/10.1109/access.2021.3058571

    Article  Google Scholar 

  31. Hao R, Lu B, Cheng Y, Li X, Huang B (2020) A steel surface defect inspection approach towards smart industrial monitoring. J Intell Manuf, https://doi.org/10.1007/s10845-020-01670-2

  32. He Y, Song K, Meng Q, Yan Y (2020) An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Trans Instrum Meas 69(4):1493–1504. https://doi.org/10.1109/tim.2019.2915404

    Article  Google Scholar 

  33. Masci J et al (2012) Steel defect classification with max-pooling convolutional neural networks. Int Jt Conf Neural Netw (IJCNN), <GotoISI>://WOS:000309341300106 IEEE

  34. Faghih-Roohi S et al (2016) Deep Convolutional Neural Networks for Detection of Rail Surface Defects. Int Jt Conf Neural Netw (IJCNN), pp 2584–2589 IEEE

  35. Lin H, Li B, Wang XG, Shu YF, Niu SL (2019) Automated defect inspection of led chip using deep convolutional neural network. J Intell Manuf 30(6):2525–2534. https://doi.org/10.1007/s10845-018-1415-x

    Article  Google Scholar 

  36. Li Y, Yang J (2021) Meta-learning baselines and database for few-shot classification in agriculture. Comput Electron Agric, vol 182, https://doi.org/10.1016/j.compag.2021.106055

  37. Li Y, Yang J (2020) Few-shot cotton pest recognition and terminal realization. Comput Electron Agric, vol 169, https://doi.org/10.1016/j.compag.2020.105240

  38. Li Y, Chao X (2021) Semi-supervised few-shot learning approach for plant diseases recognition. Plant Methods 17(1):68. https://doi.org/10.1186/s13007-021-00770-1. https://www.ncbi.nlm.nih.gov/pubmed/34176505

    Article  Google Scholar 

  39. Racki D, Tomazevic D, Skocaj D (2018) A compact convolutional neural network for textured surface anomaly detection, pp 1331–1339

  40. He Z, Liu Q (2020) Deep regression neural network for industrial surface defect detection. IEEE Access 8:35583–35591. https://doi.org/10.1109/access.2020.2975030

    Article  Google Scholar 

  41. Tao X et al (2020) Weak scratch detection of optical components using attention fusion network. Int Conf Autom Sci Eng, pp 855–862 IEEE

  42. Su BY et al (2021) Deep learning-based solar-cell manufacturing defect detection with complementary attention network. Ieee Trans Ind Inform 17(6):4084–4095. https://doi.org/10.1109/tii.2020.3008021

    Article  Google Scholar 

  43. Sun J, Wang P, Luo YK, Li W (2019) Surface defects detection based on adaptive multiscale image collection and convolutional neural networks. Ieee Trans Instrum Meas 68(12):4787–4797. https://doi.org/10.1109/tim.2019.2899478

    Article  Google Scholar 

  44. Zhu YS et al (2021) A lightweight multiscale attention semantic segmentation algorithm for detecting laser welding defects on safety vent of power battery. Ieee Access 9:39245–39254. https://doi.org/10.1109/access.2021.3064180

    Article  Google Scholar 

  45. Schlegl T, Seebock P, Waldstein SM, Langs G, Schmidt-Erfurth U (2019) F-anogan: fast unsupervised anomaly detection with generative adversarial networks. Med Image Anal 54:30–44. https://doi.org/10.1016/j.media.2019.01.010

    Article  Google Scholar 

  46. Bergmann P, Fauser M, Sattlegger D, Steger C, Ieee (2020) Uninformed students: student-teacher anomaly detection with discriminative latent embeddings. Conf Comput Vis Pattern Recognit, pp 4182–4191, IEEE

  47. Wang T, Chen Y, Qiao MN, Snoussi H (2018) A fast and robust convolutional neural network-based defect detection model in product quality control. Int J Adv Manuf 94(9-12):3465–3471. https://doi.org/10.1007/s00170-017-0882-0

    Article  Google Scholar 

  48. He K, Gkioxari G, Dollar P, Girshick R (2020) Mask r-cnn. IEEE Trans Pattern Anal Mach Intell 42(2):386–397. https://doi.org/10.1109/TPAMI.2018.2844175. https://www.ncbi.nlm.nih.gov/pubmed/29994331

    Article  Google Scholar 

  49. Wu X, Qiu L, Gu X, Long Z (2021) Deep learning-based generic automatic surface defect inspection (asdi) with pixelwise segmentation. IEEE Trans Instrum Meas 70:1–10. https://doi.org/10.1109/tim.2020.3026801

    Article  Google Scholar 

  50. Wang Z, Peng Y, Li D, Guo Y, Zhang B (2021) Mmnet: A multi-scale deep learning network for the left ventricular segmentation of cardiac mri images. Appl Intell, https://doi.org/10.1007/s10489-021-02720-9

  51. Wu Y, Jiang J, Huang Z, Tian Y (2021) Fpanet: feature pyramid aggregation network for real-time semantic segmentation. Appl Intell, https://doi.org/10.1007/s10489-021-02603-z

  52. Dong H et al (2020) Pga-net: Pyramid feature fusion and global context attention network for automated surface defect detection. IEEE Trans Ind Inform 16(12):7448–7458. https://doi.org/10.1109/tii.2019.2958826

    Article  Google Scholar 

  53. Hu X, Jing L, Sehar U (2021) Joint pyramid attention network for real-time semantic segmentation of urban scenes. Appl Intell, pp 1–15

  54. Dong XH, Taylor CJ, Cootes TF (2020) Defect detection and classification by training a generic convolutional neural network encoder. Ieee Trans Signal Process 68:6055–6069. https://doi.org/10.1109/tsp.2020.3031188

    Article  MathSciNet  MATH  Google Scholar 

  55. Li Y, Li J (2021) An end-to-end defect detection method for mobile phone light guide plate via multitask learning. IEEE Trans Instrum Meas 70:1–13. https://doi.org/10.1109/tim.2021.3053987

    Article  Google Scholar 

Download references

Funding

This work was supported by the Natural Science Foundation of China under Grant 51875113.

Author information

Authors and Affiliations

  1. College of Mechanical and Electrical Engineering, Harbin Engineering University, Nantong Street, Harbin, 150000, Heilongjiang, China

    Yihan Meng, He Xu, Zhen Ma, Jiaqiang Zhou & Daquan Hui

Authors
  1. Yihan Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. He Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiaqiang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daquan Hui
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yihan Meng: methodology, investigation, code, data analysis, writing-original draft; He Xu: supervision; Zhen Ma: review and editing; Jiaqiang Zhou, Daquan Hui: writing and editing.

Corresponding author

Correspondence to He Xu.

Ethics declarations

Conflict of Interests

The authors declare that they have no competing interests.

Additional information

Availability of data and materials

All data generated or analyzed during this study are included in this paper.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, Y., Xu, H., Ma, Z. et al. Detail-semantic guide network based on spatial attention for surface defect detection with fewer samples. Appl Intell 53, 7022–7040 (2023). https://doi.org/10.1007/s10489-022-03671-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03671-5

Keywords

Search

Navigation