Skip to content
2000
  1. Home
  2. A-Z Publications
  3. Current Medical Imaging
  4. Volume 20, Issue 1
  5. Article
Volume 20, Issue 1
  • ISSN: 1573-4056
  • E-ISSN: 1875-6603
file format pdf download PDF
side by side viewer icon HTML

Abstract

Background

Patients with malignant tumors often develop bone metastases. SPECT bone scintigraphy is an effective tool for surveying bone metastases due to its high sensitivity, low-cost equipment, and radiopharmaceutical. However, the low spatial resolution of SPECT scans significantly hinders manual analysis by nuclear medicine physicians. Deep learning, a promising technique for automated image analysis, can extract hierarchal patterns from images without human intervention.

Objective

To enhance the performance of deep learning-based segmentation models, we integrate textual data from diagnostic reports with SPECT bone scans, aiming to develop an automated analysis method that outperforms purely unimodal data-driven segmentation models.

Methods

We propose a dual-path segmentation framework to extract features from bone scan images and diagnostic reports separately. In the first path, an encoder-decoder network is employed to learn hierarchical representations of features from SPECT bone scan images. In the second path, the Chinese version of the MacBERT model is utilized to develop a text encoder for extracting features from diagnostic reports. The extracted textual features are then fused with image features during the decoding stage in the first path, enhancing the overall segmentation performance.

Results

Experimental evaluation conducted on real-world clinical data demonstrated the superior performance of the proposed segmentation model. Our model achieved a 0.0209 increase in the DSC (Dice Similarity Coefficient) score compared to the well-known U-Net model.

Conclusions

The proposed multimodal data-driven method effectively identifies and isolates metastasis lesions in SPECT bone scans, outperforming existing classical deep learning models. This study demonstrates the value of incorporating textual data in the deep learning-based segmentation of low-resolution SPECT bone scans.

© 2024 The Author(s). Published by Bentham Science.
This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Loading

Article metrics loading...

/content/journals/cmir/10.2174/0115734056324977240911073841
2024-01-01
2025-06-20

  • From This Site

    /content/journals/cmir/10.2174/0115734056324977240911073841
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
The full text of this item is not currently available.

References

  1. HiragaT. Bone metastasis: Interaction between cancer cells and bone microenvironment.J. Oral Biosci./ JAOB, Jpn. Assoc. Oral Biol.2019612959810.1016/j.job.2019.02.00231109867
     [Google Scholar]
  2. FerreiraA.R. Abrunhosa-BranquinhoA. JorgeM. CostaL. Vaz-LuísL. Bone metastases.International Manual of Oncology Practice: (iMOP)-Principles of Medical Oncology.ChamSpringer201586788910.1007/978‑3‑319‑21683‑6_40
     [Google Scholar]
  3. JunqiW. ShuoG. PET in the evaluation of bone metastases.Int. J. Radiat. Med. Nuclear Med.2006308790
     [Google Scholar]
  4. RagerO. NkoulouR. ExquisN. GaribottoV. Tabouret-ViaudC. ZaidiH. AmzalagG. Lee-FelkerS.A. ZilliT. RatibO. Whole-body SPECT/CT versus planar bone scan with targeted SPECT/CT for metastatic workup.BioMed Res. Int.201720171810.1155/2017/703940628812019
     [Google Scholar]
  5. CollierB.D.Jr HellmanR.S. KrasnowA.Z. Bone spect.Semin. Nucl. Med.198717324726610.1016/S0001‑2998(87)80037‑53303342
     [Google Scholar]
  6. BombardieriE. AktolunC. BaumR.P. Bishof-DelaloyeA. BuscombeJ. ChatalJ.F. MaffioliL. MoncayoR. MortelmansL. ReskeS.N. Bone scintigraphy: Procedure guidelines for tumour imaging.Eur. J. Nucl. Med. Mol. Imaging20033012110.1007/s00259‑003‑1347‑214989222
     [Google Scholar]
  7. LinQ. ManZ. CaoY. DengT. HanC. CaoC. ZhangL. ZengS. GaoR. WangW. JiJ. HuangX. Classifying functional nuclear images with convolutional neural networks: A survey.IET Image Process.202014143300331310.1049/iet‑ipr.2019.1690
     [Google Scholar]
  8. NathanM. GnanasegaranG. AdamsonK. FogelmanI. Bone scintigraphy: Patterns, variants, limitations and artefacts.Radionuclide and Hybrid Bone ImagingBerlin, HeidelbergSpringerLink201210.1007/978‑3‑642‑02400‑9_15
     [Google Scholar]
  9. LitjensG. KooiT. BejnordiB.E. SetioA.A.A. CiompiF. GhafoorianM. van der LaakJ.A.W.M. van GinnekenB. SánchezC.I. A survey on deep learning in medical image analysis.Med. Image Anal.201742608810.1016/j.media.2017.07.00528778026
     [Google Scholar]
  10. DhillonA. VermaG.K. Convolutional neural network: A review of models, methodologies and applications to object detection.Prog. Artif. Intell.2020928511210.1007/s13748‑019‑00203‑0
     [Google Scholar]
  11. YaoG. LeiT. ZhongJ. A review of Convolutional-Neural-Network-based action recognition.Pattern Recognit. Lett.2019118142210.1016/j.patrec.2018.05.018
     [Google Scholar]
  12. GoceriE. Evaluation of denoising techniques to remove speckle and Gaussian noise from dermoscopy images.Comput. Biol. Med.202315210647410.1016/j.compbiomed.2022.10647436563540
     [Google Scholar]
  13. KaoY.S. HuangC.P. TsaiW.W. YangJ. A systematic review for using deep learning in bone scan classification.Clin. Transl. Imaging202311327128310.1007/s40336‑023‑00539‑7
     [Google Scholar]
  14. ParanavithanaI.R. StirlingD. RosM. FieldM. Systematic review of tumor segmentation strategies for bone metastases.Cancers (Basel)2023156175010.3390/cancers1506175036980636
     [Google Scholar]
  15. PiY. ZhaoZ. XiangY. LiY. CaiH. YiZ. Automated diagnosis of bone metastasis based on multi-view bone scans using attention-augmented deep neural networks.Med. Image Anal.20206510178410.1016/j.media.2020.10178432763793
     [Google Scholar]
  16. LinQ. LiT. CaoC. CaoY. ManZ. WangH. Deep learning based automated diagnosis of bone metastases with SPECT thoracic bone images.Sci. Rep.2021111422310.1038/s41598‑021‑83083‑633608560
     [Google Scholar]
  17. WangY. LinQ. ZhaoS. ZengX. ZhengB. CaoY. ManZ. Automated diagnosis of bone metastasis by classifying bone scintigrams using a self-defined deep learning model.Curr. Med. Imag.20242024210410810.2174/0115734056281578231212104108
     [Google Scholar]
  18. LinQ. CaoC. LiT. CaoY. ManZ. WangH. Multiclass classification of whole‐body scintigraphic images using a self‐defined convolutional neural network with attention modules.Med. Phys.202148105782579310.1002/mp.1519634455613
     [Google Scholar]
  19. LiT. LinQ. GuoY. ZhaoS. ZengX. ManZ. CaoY. HuY. Automated detection of skeletal metastasis of lung cancer with bone scans using convolutional nuclear network.Phys. Med. Biol.202267101500410.1088/1361‑6560/ac456534933282
     [Google Scholar]
  20. GuoY. LinQ. ZhaoS. LiT. CaoY. ManZ. ZengX. Automated detection of lung cancer-caused metastasis by classifying scintigraphic images using convolutional neural network with residual connection and hybrid attention mechanism.Insights Imaging20221312410.1186/s13244‑022‑01162‑235138479
     [Google Scholar]
  21. CaoY. LiuL. ChenX. ManZ. LinQ. ZengX. HuangX. Segmentation of lung cancer-caused metastatic lesions in bone scan images using self-defined model with deep supervision.Biomed. Signal Process. Control20237910406810.1016/j.bspc.2022.104068
     [Google Scholar]
  22. LinQ. GaoR. LuoM. WangH. CaoY. ManZ. WangR. Semi-supervised segmentation of metastasis lesions in bone scan images.Front. Mol. Biosci.2022995672010.3389/fmolb.2022.95672036387284
     [Google Scholar]
  23. ChenY.Y. YuP.N. LaiY.C. HsiehT.C. ChengD.C. Bone metastases lesion segmentation on breast cancer bone scan images with negative sample training.Diagnostics (Basel)20231319304210.3390/diagnostics1319304237835785
     [Google Scholar]
  24. AfnouchM. GaddourO. HentatiY. BougourziF. AbidM. AlouaniI. Taleb AhmedA. BM-Seg: A new bone metastases segmentation dataset and ensemble of CNN-based segmentation approach.Expert Syst. Appl.202322812037610.1016/j.eswa.2023.120376
     [Google Scholar]
  25. LinQ. ChenX. LiuL. CaoY. ManZ. ZengX. HuangX. Detecting multiple lesions of lung cancer-caused metastasis with bone scans using a self-defined object detection model based on SSD framework.Phys. Med. Biol.2022672222500910.1088/1361‑6560/ac944d36137545
     [Google Scholar]
  26. RadfordA. KimJ.W. HallacyC. RameshA. GohG. AgarwalS. SutskeverI. Learning transferable visual models from natural language supervision.Proc. Mach. Learn. Res.20211398748763
     [Google Scholar]
  27. RonnebergerO. FischerP. BroxT. U-Net: Convolutional networks for biomedical image segmentation.arXiv:1505045972015
     [Google Scholar]
  28. LüddeckeT. EckerA. Image segmentation using text and image prompts.2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)18-24 June 2022New Orleans, LA, USA202210.1109/CVPR52688.2022.00695
     [Google Scholar]
  29. XuM. ZhangZ. WeiF. HuH. BaiX. Side adapter network for open-vocabulary semantic segmentation.arXiv:230212242202310.1109/CVPR52729.2023.00288
     [Google Scholar]
  30. ZhangY. JiangH. MiuraY. ManningC.D. LanglotzC.P. Contrastive learning of medical visual representations from paired images and text.arXiv:2010007472022
     [Google Scholar]
  31. HuangS.C. ShenL. LungrenM.P. YeungS. Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition.2021 IEEE/CVF International Conference on Computer Vision (ICCV)10-17 October 2021Montreal, QC, Canada202110.1109/ICCV48922.2021.00391
     [Google Scholar]
  32. WangZ. WuZ. AgarwalD. SunJ. Medclip: Contrastive learning from unpaired medical images and text.Find. Assoc. Computat. Linguist.202220223876388710.18653/v1/2022.emnlp‑main.256
     [Google Scholar]
  33. TomarN.K. JhaD. BagciU. AliS. TGANet: Text-guided attention for improved polyp segmentation.arXiv:2205042802022
     [Google Scholar]
  34. CuiY. CheW. LiuT. QinB. WangS. HuG. Revisiting pre-trained models for Chinese natural language processing.Find. Assoc. Computat. Linguist..2020202065766810.18653/v1/2020.findings‑emnlp.58
     [Google Scholar]
  35. LinJ. ZhangY. HouW. QinQ. GalskyM. OhW. TsaoC.K. Evolving patterns of metastasis in renal cell carcinoma: Do we need to perform routine bone imaging?J. Kidney Cancer VHL202184131910.15586/jkcvhl.v8i4.20234722126
     [Google Scholar]
  36. PerrosP. Kendall-TaylorP. Thyroid-associated ophthalmopathy: Pathogenesis and clinical management.Baillieres Clin. Endocrinol. Metab.19959111513510.1016/S0950‑351X(95)80867‑17726793
     [Google Scholar]
  37. TsuyaA. KurataT. TamuraK. FukuokaM. Skeletal metastases in non-small cell lung cancer: A retrospective study.Lung Cancer200757222923210.1016/j.lungcan.2007.03.01317451841
     [Google Scholar]
  38. WooS. ParkJ. LeeY. KweonS. Cbam: Convolutional block attention module.arXiv:180706521201810.1007/978‑3‑030‑01234‑2_1
     [Google Scholar]
  39. LongJ. ShelhamerE. DarrellT. Fully convolutional networks for semantic segmentation.arXiv:14114038201510.1109/CVPR.2015.7298965
     [Google Scholar]
  40. BadrinarayananV. KendallA. CipollaR. Segnet: A deep convolutional encoder-decoder architecture for image segmentation.IEEE Trans. Pattern Anal. Mach. Intell.201739122481249510.1109/TPAMI.2016.264461528060704
     [Google Scholar]
  41. OktayO. SchlemperJ. FolgocL. LeeM. HeinrichM. MisawaK. RueckertD. Attention u-net: Learning where to look for the pancreas.arXiv:1804039992018
     [Google Scholar]
  42. ZhouZ. Rahman SiddiqueeM.M. TajbakhshN. LiangJ. UNet++: A nested u-net architecture for medical image segmentation.Lect. Notes Comput. Sci.20181104531110.1007/978‑3‑030‑00889‑5_132613207
     [Google Scholar]
  43. HuangH. LinL. TongR. HuH. ZhangQ. IwamotoY. WuJ. Unet 3+: A full-scale connected unet for medical image segmentation.arXiv:200408790202010.1109/ICASSP40776.2020.9053405
     [Google Scholar]
  44. GoceriE. Polyp segmentation using a hybrid vision transformer and a hybrid loss function.J. Imag. Inform. Med.202437285186310.1007/s10278‑023‑00954‑238343250
     [Google Scholar]
  45. GoceriE. Nuclei segmentation using attention aware and adversarial networks.Neurocomputing202457912744510.1016/j.neucom.2024.127445
     [Google Scholar]
/content/journals/cmir/10.2174/0115734056324977240911073841
Loading
Multimodal Data-driven Segmentation of Bone Metastasis Lesions in SPECT Bone Scans using Deep Learning
Curr. Med. Imaging 20, e15734056324977 (2024); https://doi.org/10.2174/0115734056324977240911073841
/content/journals/cmir/10.2174/0115734056324977240911073841
/content/journals/cmir/10.2174/0115734056324977240911073841
Loading

Data & Media loading...


  • Article Type:     Research Article
Keyword(s): Bone metastasis; Bone scan; Deep learning; Encoder-decoder network; Lesion segmentation; Multimodal data

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test