Skip to content
2000
  1. Home
  2. A-Z Publications
  3. Current Medical Imaging
  4. Fast Track Articles
  5. Fast Track Article
image of Enhanced Pneumonia Detection in Chest X-Rays Using Hybrid Convolutional and Vision Transformer Networks
file format pdf download PDF

Abstract

Objective:

The objective of this research is to enhance pneumonia detection in chest X-rays by leveraging a novel hybrid deep learning model that combines Convolutional Neural Networks (CNNs) with modified Swin Transformer blocks. This study aims to significantly improve diagnostic accuracy, reduce misclassifications, and provide a robust, deployable solution for underdeveloped regions where access to conventional diagnostics and treatment is limited.

Methods:

The study developed a hybrid model architecture integrating CNNs with modified Swin Transformer blocks to work seamlessly within the same model. The CNN layers perform initial feature extraction, capturing local patterns within the images. At the same time, the modified Swin Transformer blocks handle long-range dependencies and global context through window-based self-attention mechanisms. Preprocessing steps included resizing images to 224x224 pixels and applying Contrast Limited Adaptive Histogram Equalization (CLAHE) to enhance image features. Data augmentation techniques, such as horizontal flipping, rotation, and zooming, were utilized to prevent overfitting and ensure model robustness. Hyperparameter optimization was conducted using Optuna, employing Bayesian optimization (Tree-structured Parzen Estimator) to fine-tune key parameters of both the CNN and Swin Transformer components, ensuring optimal model performance.

Results:

The proposed hybrid model was trained and validated on a dataset provided by the Guangzhou Women and Children’s Medical Center. The model achieved an overall accuracy of 98.72% and a loss of 0.064 on an unseen dataset, significantly outperforming a baseline CNN model. Detailed performance metrics indicated a precision of 0.9738 for the normal class and 1.0000 for the pneumonia class, with an overall F1-score of 0.9872. The hybrid model consistently outperformed the CNN model across all performance metrics, demonstrating higher accuracy, precision, recall, and F1-score. Confusion matrices revealed high sensitivity and specificity with minimal misclassifications.

Conclusion:

The proposed hybrid CNN-ViT model, which integrates modified Swin Transformer blocks within the CNN architecture, provides a significant advancement in pneumonia detection by effectively capturing both local and global features within chest X-ray images. The modifications to the Swin Transformer blocks enable them to work seamlessly with the CNN layers, enhancing the model’s ability to understand complex visual patterns and dependencies. This results in superior classification performance. The lightweight design of the model eliminates the need for extensive hardware, facilitating easy deployment in resource-constrained settings. This innovative approach not only improves pneumonia diagnosis but also has the potential to enhance patient outcomes and support healthcare providers in underdeveloped regions. Future research will focus on further refining the model architecture, incorporating more advanced image processing techniques, and exploring explainable AI methods to provide deeper insights into the model's decision-making process.

© 2025 The Author(s). Published by Bentham Science Publisher.
This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Loading

Article metrics loading...

/content/journals/cmir/10.2174/0115734056326685250101113959
2025-01-09
2025-01-18

  • From This Site

    /content/journals/cmir/10.2174/0115734056326685250101113959
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

/deliver/fulltext/cmir/10.2174/0115734056326685250101113959/e15734056326685.html?itemId=/content/journals/cmir/10.2174/0115734056326685250101113959&mimeType=html&fmt=ahah

References

  1. Pneumonia in children. 2023 Available from: https://www.who.int/news-room/fact-sheets/detail/pneumonia
  2. Campbell H. el Arifeen S. Hazir T. O’Kelly J. Bryce J. Rudan I. Qazi S.A. Measuring coverage in MNCH: Challenges in monitoring the proportion of young children with pneumonia who receive antibiotic treatment. PLoS Med. 2013 10 5 e1001421 10.1371/journal.pmed.1001421 23667338
    [Google Scholar]
  3. Bio-remediation. 2023 Available from: https://www.researchgate.net/publication/327932092_Bio-remediation?channel=doi&linkId=5badf0f645851574f7ec1452&showFulltext=true
  4. Wootton D. Feldman C. The diagnosis of pneumonia requires a chest radiograph (x-ray)—yes, no or sometimes? Pneumonia. 2014 Available from: https://pneumonia.biomedcentral.com/articles/10.15172/pneu.2014.5/464
  5. Prendki V. Scheffler M. Huttner B. Garin N. Herrmann F. Janssens J.P. Marti C. Carballo S. Roux X. Serratrice C. Serratrice J. Agoritsas T. Becker C.D. Kaiser L. Rosset-Zufferey S. Soulier V. Perrier A. Reny J.L. Montet X. Stirnemann J. Low-dose computed tomography for the diagnosis of pneumonia in elderly patients: A prospective, interventional cohort study. Eur. Respir. J. 2018 51 5 1702375 10.1183/13993003.02375‑2017 29650558
    [Google Scholar]
  6. Pearce M.S. Salotti J.A. Little M.P. McHugh K. Lee C. Kim K.P. Howe N.L. Ronckers C.M. Rajaraman P. Craft A.W. Parker L. Berrington de González A. Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: A retrospective cohort study. Lancet 2012 380 9840 499 505 10.1016/S0140‑6736(12)60815‑0 22681860
    [Google Scholar]
  7. Resnick P. Varian H.R. Recommender systems. Commun. ACM 1997 40 3 56 58 10.1145/245108.245121
    [Google Scholar]
  8. Esteva A. Kuprel B. Novoa R.A. Ko J. Swetter S.M. Blau H.M. FGF23 and Fetuin-A interaction in the liver and in the circulation. Int. J. Biol. Sci. 2017 14 586 598 10.1038/nature21056
    [Google Scholar]
  9. Litjens G. Kooi T. Bejnordi B.E. Setio A.A.A. Ciompi F. Ghafoorian M. van der Laak J.A.W.M. van Ginneken B. Sánchez C.I. A survey on deep learning in medical image analysis. Med. Image Anal. 2017 42 60 88 10.1016/j.media.2017.07.005 28778026
    [Google Scholar]
  10. Shen D. Wu G. Suk H.I. Deep Learning in Medical Image Analysis Il. Annu. Rev. Biomed. Eng. 2017 19 1 221 248 10.1146/annurev‑bioeng‑071516‑044442 28301734
    [Google Scholar]
  11. Cires¸ancires¸an D.C. Giusti A. Gambardella L.M. Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv 2013 16 Pt 2 411 418
    [Google Scholar]
  12. Esteva A. Kuprel B. Novoa R.A. Ko J. Swetter S.M. Blau H.M. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017 542 7639 115 118 10.1038/nature21056
    [Google Scholar]
  13. He K. Zhang X. Ren S. Sun J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv:1502.01852 2015 10.1109/ICCV.2015.123
    [Google Scholar]
  14. Huang L. Qin J. Zhou Y. Zhu F. Liu L. Shao L. Normalization techniques in training DNNs: Methodology, analysis and application. arXiv:2009.12836 2020
    [Google Scholar]
  15. Ioffe S. Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML'15: Proceedings of the 32nd International Conference on International Conference on Machine Learning Volume 37, 2015 Pages 448 - 456
    [Google Scholar]
  16. Buades A. Coll B. Morel J.M. A non-local algorithm for image denoising. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) San Diego, CA, USA, 20-25 June 2005, pp. 60-65 10.1109/CVPR.2005.38
    [Google Scholar]
  17. Dabov K. Foi A. Katkovnik V. Egiazarian K. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 2007 16 8 2080 2095 10.1109/TIP.2007.901238 17688213
    [Google Scholar]
  18. Ulyanov D. Vedaldi A. Lempitsky V. Instance normalization: The missing ingredient for fast stylization. arXiv:1607.08022 2016 http://arxiv.org/abs/1607.08022
    [Google Scholar]
  19. Shin H.C. Roth H.R. Gao M. Lu L. Xu Z. Nogues I. Yao J. Mollura D. Summers R.M. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 2016 35 5 1285 1298 10.1109/TMI.2016.2528162 26886976
    [Google Scholar]
  20. Vaswani A. Brain G. Shazeer N. Parmar N. Uszkoreit J. Jones L. Attention is All you Need. Adv. Neural Inf. Process. Syst. 2017 30
    [Google Scholar]
  21. Chen LC Zhu Y Papandreou G Schroff F Adam H Encoder-decoder with atrous separable convolution for semantic image segmentation. arXiv:1802.02611 2018 10.1007/978‑3‑030‑01234‑2_49
    [Google Scholar]
  22. Bobowicz M. Rygusik M. Buler J. Buler R. Ferlin M. Kwasigroch A. Szurowska E. Grochowski M. Attention-based deep learning system for classification of breast lesions—multimodal, weakly supervised approach. Cancers 2023 15 10 2704 10.3390/cancers15102704 37345041
    [Google Scholar]
  23. Oktay O. Schlemper J. Le Folgoc L. Lee M. Heinrich M. Misawa K. Attention U-Net: Learning where to look for the pancreas. arXiv:1804.03999 2018 https://arxiv.org/abs/1804.03999v3
    [Google Scholar]
  24. Fu J. Liu J. Tian H. Li Y. Bao Y. Fang Z. Dual Attention Network for Scene Segmentation. 2019 https://arxiv.org/abs/1809.02983v4 10.1109/CVPR.2019.00326
    [Google Scholar]
  25. Stephen O Sain M Maduh UJ Jeong DU An efficient deep learning approach to pneumonia classification in healthcare. J Healthc Eng 2019 2019 4180949 4180949
    [Google Scholar]
  26. Yue H. Yu Q. Liu C. Huang Y. Jiang Z. Shao C. Zhang H. Ma B. Wang Y. Xie G. Zhang H. Li X. Kang N. Meng X. Huang S. Xu D. Lei J. Huang H. Yang J. Ji J. Pan H. Zou S. Ju S. Qi X. Machine learning-based CT radiomics method for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: a multicenter study. Ann. Transl. Med. 2020 8 14 859 859 10.21037/atm‑20‑3026 32793703
    [Google Scholar]
  27. Liang G. Zheng L. A transfer learning method with deep residual network for pediatric pneumonia diagnosis. Comput. Methods Programs Biomed. 2020 187 104964 10.1016/j.cmpb.2019.06.023 31262537
    [Google Scholar]
  28. Yadav S.S. Jadhav S.M. Deep convolutional neural network based medical image classification for disease diagnosis. J. Big Data 2019 6 1 113 10.1186/s40537‑019‑0276‑2
    [Google Scholar]
  29. Bhatt H. Shah M. A Convolutional Neural Network ensemble model for Pneumonia Detection using chest X-ray images. Healthc. Anal. 2023 3 100176 10.1016/j.health.2023.100176
    [Google Scholar]
  30. E ERDEM. Detection of pneumonia with a novel CNN-based approach. Sakarya Univ. J. Comput. Inf. Sci. 2021 4 1 26 34
    [Google Scholar]
  31. Saul C.J. Urey D.Y. Taktakoglu C.D. Early diagnosis of pneumonia with deep learning. arXiv:1904.00937 2019 https://arxiv.org/abs/1904.00937v1
    [Google Scholar]
  32. Zhang J. Xie Y. Pang G. Liao Z. Verjans J. Li W. Sun Z. He J. Li Y. Shen C. Xia Y. Viral pneumonia screening on chest X-Rays using confidence-aware anomaly detection. IEEE Trans. Med. Imaging 2021 40 3 879 890 10.1109/TMI.2020.3040950 33245693
    [Google Scholar]
  33. Chouhan V. Singh S.K. Khamparia A. Gupta D. Tiwari P. Moreira C. A novel transfer learning based approach for pneumonia detection in chest X-ray images. Appl. Sci 2020 10 2 559
    [Google Scholar]
  34. Rajpurkar P. Irvin J. Zhu K. Yang B. Mehta H. Duan T. CheXNet: Radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv:1711.05225 2017
    [Google Scholar]
  35. Jain R. Nagrath P. Kataria G. Sirish Kaushik V. Jude Hemanth D. Pneumonia detection in chest X-ray images using convolutional neural networks and transfer learning. Measurement 2020 165 108046 10.1016/j.measurement.2020.108046
    [Google Scholar]
  36. El Asnaoui K. Chawki Y. Idri A. Automated methods for detection and classification pneumonia based on X-Ray images using deep learning. arXiv:2003.14363 2021 10.1007/978‑3‑030‑74575‑2_14
    [Google Scholar]
  37. Mittal A. Kumar D. Mittal M. Saba T. Abunadi I. Rehman A. Detecting pneumonia using convolutions and dynamic capsule routing for chest X-ray images. Sensors 2020 20 4 1068
    [Google Scholar]
  38. Ayan E. Ünver H.M. Diagnosis of pneumonia from chest X-ray images using deep learning. 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT) Istanbul, Turkey, 24-26 April 2019, pp. 1-5 10.1109/EBBT.2019.8741582
    [Google Scholar]
  39. Kundu R. Das R. Geem Z.W. Han G.T. Sarkar R. Pneumonia detection in chest X-ray images using an ensemble of deep learning models. PLoS One 2021 16 9 e0256630 10.1371/journal.pone.0256630 34492046
    [Google Scholar]
  40. O’Quinn W. Haddad R.J. Moore D.L. Pneumonia radiograph diagnosis utilizing deep learning network. 2019 IEEE 2nd International Conference on Electronic Information and Communication Technology (ICEICT) Harbin, China, 20-22 January 2019, pp. 763-767 10.1109/ICEICT.2019.8846438
    [Google Scholar]
  41. Talo M. 2019
  42. Varshni D. Thakral K. Agarwal L. Nijhawan R. Mittal A. Pneumonia Detection Using CNN based Feature Extraction. 2019 10.1109/ICECCT.2019.8869364
    [Google Scholar]
  43. Li L. Xu M. Liu H. Li Y. Wang X. Jiang L. Wang Z. Fan X. Wang N. A Large-Scale Database and a CNN Model for Attention-Based Glaucoma Detection. IEEE Trans. Med. Imaging 2020 39 2 413 424 10.1109/TMI.2019.2927226 31283476
    [Google Scholar]
  44. Fu J. Li W. Du J. Huang Y. A multiscale residual pyramid attention network for medical image fusion. Biomed. Signal Process. Control 2021 66 102488 10.1016/j.bspc.2021.102488
    [Google Scholar]
  45. Gu R. Wang G. Song T. Huang R. Aertsen M. Deprest J. Ourselin S. Vercauteren T. Zhang S. CA-Net: Comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE Trans. Med. Imaging 2021 40 2 699 711 10.1109/TMI.2020.3035253 33136540
    [Google Scholar]
  46. Guo X. Yuan Y. Triple ANet: Adaptive Abnormal-aware Attention Network for WCE Image Classification Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, 22nd International Conference Shenzhen, China, October 13–17, 2019, Proceedings, Part I (pp.293-301)
    [Google Scholar]
  47. Shamshad F. Khan S. Zamir S.W. Khan M.H. Hayat M. Khan F.S. Fu H. Transformers in medical imaging: A survey. Med. Image Anal. 2023 88 102802 10.1016/j.media.2023.102802 37315483
    [Google Scholar]
  48. Khan S. Naseer M. Hayat M. Zamir S.W. Khan F.S. Shah M. Transformers in vision: A survey. ACM Comput. Surv. 2022 54 10
    [Google Scholar]
  49. Dosovitskiy A. Beyer L. Kolesnikov A. Weissenborn D. Zhai X. Unterthiner T. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929 2020
    [Google Scholar]
  50. Singh S. Kumar M. Kumar A. Verma B.K. Abhishek K. Selvarajan S. Efficient pneumonia detection using vision transformers on chest X-rays. Scientific Reports 2024 14 2487
    [Google Scholar]
  51. Usman M. Zia T. Tariq A. Analyzing transfer learning of vision transformers for interpreting chest radiography. J. Digit. Imaging 2022 35 6 1445 1462 10.1007/s10278‑022‑00666‑z 35819537
    [Google Scholar]
  52. Kermany D. Zhang K. Goldbaum M. Large dataset of labeled optical coherence tomography (oct) and chest x-ray images. Mendeley Data 2018 10.17632
    [Google Scholar]
  53. Shorten C. Khoshgoftaar T.M. A survey on image data augmentation for deep learning. J. Big Data 2019 6 1 60 10.1186/s40537‑019‑0197‑0
    [Google Scholar]
  54. Zhu X. Lyu S. Wang X. Zhao Q. TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) Montreal, BC, Canada, 11-17 October 2021, pp. 2778-2788
    [Google Scholar]
  55. Liu Z. Lin Y. Cao Y. Hu H. Wei Y. Zhang Z. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv:2103.14030 2021 https://arxiv.org/abs/2103.14030v2
    [Google Scholar]
  56. Darici M.B. Dokur Z. Olmez T. Pneumonia detection and classification using deep learning on chest x-ray images. Int. J. Intell. Syst. Appl. Eng. 2020 8 4 177 183 10.18201/ijisae.2020466310
    [Google Scholar]
  57. Hashmi MF Katiyar S Hashmi AW Keskar AG Pneumonia detection in chest X-ray images using compound scaled deep learning model. J. Control Meas. Electron. Comput. Commun. 2021 62 3-4 397 406
    [Google Scholar]
  58. Luján-García J.E. Yáñez-Márquez C. Villuendas-Rey Y. Camacho-Nieto O. A transfer learning method for pneumonia classification and visualization. Appl. Sci 2020 10 8 2908 10.3390/app10082908
    [Google Scholar]
/content/journals/cmir/10.2174/0115734056326685250101113959
Loading
Enhanced Pneumonia Detection in Chest X-Rays Using Hybrid Convolutional and Vision Transformer Networks
Bentham Science Publishers , e15734056326685; https://doi.org/10.2174/0115734056326685250101113959
/content/journals/cmir/10.2174/0115734056326685250101113959
/content/journals/cmir/10.2174/0115734056326685250101113959
Loading

Data & Media loading...


  • Article Type:     Research Article
Keywords: Vision transformers ; Pneumonia detection ; CLAHE ; Chest X-ray imaging ; Swin transformer ; Convolutional neural networks ; Deep learning ; Medical image processing ; Computer-aided diagnosis ; Hyperparameter optimization

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test