Skip to content
2000
  1. Home
  2. A-Z Publications
  3. Recent Patents on Engineering
  4. Fast Track Articles
  5. Fast Track Article
image of High Resolution Medical Image Inpainting Based on Super Resolution
file format pdf download PDF

Abstract

Introduction

Image inpainting techniques and patents have made great progress in recent years. However, with higher image resolution, the large amount of computation and memory requirements cause great difficulties for training.

Method

We propose a medical image inpainting model that utilizes super-resolution techniques to enhance the resolution of images during the repair process. Moreover, to maximize the utilization of semantic information and enhance network performance, we incorporate multi-scale dense residual blocks for feature extraction from the image.

Results

Meanwhile, we utilize structural similarity as a loss function to encourage the network to synthesize more meaningful structural textures. Experimental results demonstrate that HRMII can effectively reduce the computation and memory occupation of the high-resolution image inpainting model, and obtain satisfactory inpainting results.

Conclusion

Additionally, ablation studies have substantiated the efficacy of diverse modules within the network.

© 2025 The Author(s). Published by Bentham Science Publishers
This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Loading

Article metrics loading...

/content/journals/eng/10.2174/0118722121297704240917145311
2025-01-09
2025-05-12

  • From This Site

    /content/journals/eng/10.2174/0118722121297704240917145311
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
The full text of this item is not currently available.

References

  1. Hou J. Xu J. Jiang L. Du S. Feng R. Zhang Y. Shan F. Xue X. Periphery-aware COVID-19 diagnosis with contrastive representation enhancement. Pattern Recognit. 2021 118 108005 10.1016/j.patcog.2021.108005 33972808
    [Google Scholar]
  2. Zheng Y. Zheng Y. Suehiro D. Uchida S. Top-rank convolutional neural network and its application to medical image-based diagnosis. Pattern Recognit. 2021 120 108138 10.1016/j.patcog.2021.108138
    [Google Scholar]
  3. Grünwedel E. Gloger C. Panakos A. Modular inpainting method. US-20220156894-A1 2024
  4. Hitchcock Y. J. Impact of integrated high-resolution FDG 18PET/CT into radiotherapy planning and post-treatment assessment for locoregional advanced squamous cell carcinoma of the head and neck. Fuel Energy Abst. 2010 78 3 10.1016/j.ijrobp.2010.07.1016
    [Google Scholar]
  5. Liu H. Xu J. Wu Y. Guo Q. Ibragimov B. Xing L. Learning deconvolutional deep neural network for high resolution medical image reconstruction. Inf. Sci. 2018 468 142 154 10.1016/j.ins.2018.08.022
    [Google Scholar]
  6. Zhao W. Li D. Niu K. Qin W. Peng H. Niu T. Robust beam hardening artifacts reduction for computed tomography using spectrum modeling. IEEE Trans. Comput. Imaging 2019 5 2 333 342 10.1109/TCI.2018.2884479
    [Google Scholar]
  7. Peng C. Li B. Liang P. Zheng J. Zhang Y. Qiu B. Chen D.Z. A cross-domain metal trace restoring network for reducing X-Ray CT metal artifacts. IEEE Trans. Med. Imaging 2020 39 12 3831 3842 10.1109/TMI.2020.3005432 32746126
    [Google Scholar]
  8. Jam J. Kendrick C. Walker K. Drouard V. Hsu J.G.S. Yap M.H. A comprehensive review of past and present image inpainting methods. Comput. Vis. Image Underst. 2021 203 103147 10.1016/j.cviu.2020.103147
    [Google Scholar]
  9. Elharrouss O. Almaadeed N. Al-Maadeed S. Akbari Y. Image inpainting: A review. Neural Process. Lett. 2020 51 2 2007 2028 10.1007/s11063‑019‑10163‑0
    [Google Scholar]
  10. Bertalmio M. Sapiro G. Caselles V. Ballester C. Image inpainting. Proceedings of the 27th annual conference on Computer graphics and interactive techniques 2000 417 424
    [Google Scholar]
  11. Barnes C. Shechtman E. Finkelstein A. Goldman D.B. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transac. Graphics 2009 28 2 1 11 10.1145/1576246.1531330
    [Google Scholar]
  12. Iizuka S. Simo-Serra E. Ishikawa H. Globally and locally consistent image completion. ACM Trans. Graph. 2017 36 4 1 14 10.1145/3072959.3073659
    [Google Scholar]
  13. Wang N. Ma S. Li J. Zhang Y. Zhang L. Multistage attention network for image inpainting. Pattern Recognit. 2020 106 107448 10.1016/j.patcog.2020.107448
    [Google Scholar]
  14. Yi Z. Tang Q. Azizi S. Jang D. Xu Z. Contextual Residual Aggregation for Ultra High- Resolution Image Inpainting 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 13-19 June 2020 Seattle, WA, USA 2020 10.1109/CVPR42600.2020.00753
    [Google Scholar]
  15. Liu G. Reda F.A. Shih K.J. Wang T-C. Tao A. Catanzaro B. Image Inpainting for Irregular Holes Using Partial Convolutions. Computer Vision – ECCV 2018. Ferrari V. Hebert M. Sminchisescu C. Weiss Y. Cham Springer International Publishing 2018 89 105 10.1007/978‑3‑030‑01252‑6_6
    [Google Scholar]
  16. Wang Q. Chen Y. Zhang N. Gu Y. Medical image inpainting with edge and structure priors. Measurement 2021 185 110027 10.1016/j.measurement.2021.110027
    [Google Scholar]
  17. Canny J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 1986 PAMI-8 6 679 698 10.1109/TPAMI.1986.4767851 21869365
    [Google Scholar]
  18. Yu F. Koltun V. Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122 2016
    [Google Scholar]
  19. Miyato T. Kataoka T. Koyama M. Yoshida Y. Spectral normalization for generative adversarial networks. arXiv:1802.05957 2018
    [Google Scholar]
  20. Jaeger S. Candemir S. Antani S. Wáng Y-X.J. Lu P-X. Thoma G. Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant. Imaging Med. Surg. 2014 4 6 475 477 10.3978/j.issn.2223‑4292.2014.11.20 25525580
    [Google Scholar]
  21. Halabi S.S. Prevedello L.M. Kalpathy-Cramer J. Mamonov A.B. Bilbily A. Cicero M. Pan I. Pereira L.A. Sousa R.T. Abdala N. Kitamura F.C. Thodberg H.H. Chen L. Shih G. Andriole K. Kohli M.D. Erickson B.J. Flanders A.E. The rSNA pediatric bone age machine learning challenge. Radiology 2019 290 2 498 503 10.1148/radiol.2018180736 30480490
    [Google Scholar]
  22. Zhao J. Zhang Y. He X. Xie P. Covid-ct-dataset: A ct scan dataset about covid-19. arXiv:2003.13865 2020
    [Google Scholar]
  23. Ballester C. Bertalmio M. Caselles V. Sapiro G. Verdera J. Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 2001 10 8 1200 1211 10.1109/83.935036 18255537
    [Google Scholar]
  24. Vlašánek P. Fuzzy image inpainting aimed to medical images. Seman. Sch. 2018 10.24132/CSRN.2018.2801.7
    [Google Scholar]
  25. Criminisi A. Pérez P. Toyama K. Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 2004 13 9 1200 1212 10.1109/TIP.2004.833105 15449582
    [Google Scholar]
  26. Drori I. Cohen-Or D. Yeshurun H. Fragment-based image completion. Seman. Sch. 2003 10.1145/1201775.882267
    [Google Scholar]
  27. Yu J. Lin Z. Yang J. Shen X. Lu X. Huang T.S. Generative image inpainting with contextual attention. arXiv:1801.07892 2018 10.1109/CVPR.2018.00577
    [Google Scholar]
  28. Nazeri K. Ng E. Joseph T. Qureshi F. Ebrahimi M. Edgeconnect: Structure guided image inpainting using edge prediction. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops 27-28 October 2019 Seoul, Korea (South) 2019 10.1109/ICCVW.2019.00408
    [Google Scholar]
  29. Pathak D. Krähenbühl P. Donahue J. Darrell T. Efros A.A. Context Encoders: Feature learning by inpainting. arXiv:1604.07379 2016 10.1109/CVPR.2016.278
    [Google Scholar]
  30. Armanious K. Mecky Y. Gatidis S. Yang B. Adversarial inpainting of medical image modalities. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 12-17 May 2019 Brighton, UK 2019 10.1109/ICASSP.2019.8682677
    [Google Scholar]
  31. Armanious K. Kumar V. Abdulatif S. Hepp T. Gatidis S. Yang B. IpA-MedGAN: Inpainting of arbitrary regions in medical imaging. 2020 IEEE International Conference on Image Processing (ICIP) 25-28 October 2020 Abu Dhabi, United Arab Emirates 2020 10.1109/ICIP40778.2020.9191207
    [Google Scholar]
  32. Yu W. Du J. Liu R. Li Y. Zhu Y. Interactive image inpainting using semantic guidance. arXiv:2201.10753 2022 10.1109/ICPR56361.2022.9956171
    [Google Scholar]
  33. Wang Y. Li D. Xu C. Yang M. Improve deep image inpainting by emphasizing the complexity of missing regions. arXiv:2202.06266 2022
    [Google Scholar]
  34. Huang Y. Zheng F. Wang D. Jiang J. Wang X. Shao L. Super-resolution and inpainting with degraded and upgraded generative adversarial networks. 2020 Available From: https://www.ijcai.org/proceedings/2020/90
  35. Cao L. Yang T. Wang Y. Yan B. Guo Y. Generator pyramid for high-resolution image inpainting. Complex Intell. Syst. 2023 9 6 6297 6306 10.1007/s40747‑023‑01080‑w
    [Google Scholar]
  36. Zeng Y. Lin Z. Yang J. Zhang J. Shechtman E. Lu H. High-resolution image inpainting with iterative confidence feedback and guided upsampling. arXiv:2005.11742 2020 10.1007/978‑3‑030‑58529‑7_1
    [Google Scholar]
  37. Simonyan K. Zisserman A. Very deep convolutional networks for large-scale image recognition arXiv:1409.1556 2015
    [Google Scholar]
  38. He K. Zhang X. Ren S. Sun J. Identity mappings in deep residual networks. arXiv:1603.05027 2016 10.1007/978‑3‑319‑46493‑0_38
    [Google Scholar]
  39. Zeiler M.D. Krishnan D. Taylor G.W. Fergus R. Deconvolutional networks. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition 13-18 June 2010 San Francisco, CA, USA 2010 10.1109/CVPR.2010.5539957
    [Google Scholar]
  40. Tong T. Li G. Liu X. Gao Q. Image super-resolution using dense skip connections. 2017 IEEE International Conference on Computer Vision (ICCV) 22-29 October 2017 Venice, Italy 2017 10.1109/ICCV.2017.514
    [Google Scholar]
  41. He K. Zhang X. Ren S. Sun J. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 27-30 June 2016 Las Vegas, NV, USA 2016 770 778 10.1109/CVPR.2016.90
    [Google Scholar]
  42. Wang Z. Bovik A.C. Sheikh H.R. Simoncelli E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004 13 4 600 612 10.1109/TIP.2003.819861 15376593
    [Google Scholar]
  43. Zeng Y. Fu J. Chao H. Guo B. Learning pyramid-context encoder network for high-quality image inpainting. Seman. Sch. 2019 10.1109/CVPR.2019.00158
    [Google Scholar]
  44. Johnson J. Alahi A. Fei-Fei L. Perceptual losses for real-time style transfer and super- resolution. arXiv:1603.08155 2016 10.1007/978‑3‑319‑46475‑6_43
    [Google Scholar]
  45. Gatys L.A. Ecker A.S. Bethge M. Image style transfer using convolutional neural networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 27-30 June 2016 Las Vegas, NV, USA 2016 10.1109/CVPR.2016.265
    [Google Scholar]
  46. Isola P. Zhu J.Y. Zhou T. Efros A.A. Image-to-image translation with conditional adversarial networks arXiv:1611.07004 2017 10.1109/CVPR.2017.632
    [Google Scholar]
  47. Kingma D.P. Ba J.L. Adam: A method for stochastic optimization. arXiv:1412.6980 2015
    [Google Scholar]
  48. Yu J. Lin Z. Yang J. Shen X. Lu X. Huang T. Free-form image inpainting with gated convolution. arXiv:1806.03589 2019 10.1109/ICCV.2019.00457
    [Google Scholar]
  49. Heusel M. Ramsauer H. Unterthiner T. Nessler B. Hochreiter S. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Advances Neural Inform. Process. Syst. 2019 8 1 9 10.18034/ajase.v8i1.9
    [Google Scholar]
  50. Salimans T. Goodfellow I. Zaremba W. Cheung V. Radford A. Chen X. Improved techniques for training GANs. arXiv:1606.03498 2016
    [Google Scholar]
  51. Pérez P. Gangnet M. Blake A. Poisson image editing. ACM Trans. Graph. 2003 22 3 313 318 10.1145/882262.882269
    [Google Scholar]
  52. Yeganeh Y. Farshad A. Navab N. Anatomy-aware masking for inpainting in medical imaging. Shape in Medical Imaging Berlin, Heidelberg Springer Link 2023 10.1007/978‑3‑031‑46914‑5_3
    [Google Scholar]
  53. Bansal N. Sridhar S. HEXA-GAN: Skin lesion image inpainting via hexagonal sampling based generative adversarial network. Biomed. Signal Process. Control 2024 89 105603 10.1016/j.bspc.2023.105603
    [Google Scholar]
  54. Yuan Y. Dai S. MII-DFNet: A Medical image inpainting framework based on deep fusion network. 2023 4th International Conference on Information Science, Parallel and Distributed Systems (ISPDS) 14-16 July 2023 Guangzhou, China 2023 10.1109/ISPDS58840.2023.10235664
    [Google Scholar]
/content/journals/eng/10.2174/0118722121297704240917145311
Loading
High Resolution Medical Image Inpainting Based on Super Resolution
Bentham Science Publishers ; https://doi.org/10.2174/0118722121297704240917145311
/content/journals/eng/10.2174/0118722121297704240917145311
/content/journals/eng/10.2174/0118722121297704240917145311
Loading

Data & Media loading...


  • Article Type:     Research Article
Keywords: super resolution ; High-resolution medical image inpainting ; Multi-scale dense residual blocks ; Structural similarity

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test