Skip to content
2000
  1. Home
  2. A-Z Publications
  3. Recent Advances in Computer Science and Communications
  4. Volume 18, Issue 1
  5. Article
Volume 18, Issue 1
  • ISSN: 2666-2558
  • E-ISSN: 2666-2566

Abstract

Aim

Scientific, technical, and educational research domains all heavily rely on handwritten mathematical expressions. The extensive use of online handwritten mathematical expression recognition is a consequence of the availability of strong computational touch-screen appliances, such as the recent development of deep neural networks as superior sequence recognition models.

Background

Further investigation and enhancement of these technologies are vital to tackle the contemporary obstacles presented by the widespread adoption of remote learning and work arrangements as a result of the global health crisis.

Objective

Handwritten document processing has gained more attention in the last ten years due to notable developments in deep neural network-based computer vision models and sequence recognition, as well as the widespread proliferation of touch and pen-enabled smartphones and tablets. It comes naturally to people to write by hand in daily interactions.

Method

In this article, authors implemented Hand written expressions using RNN-based encoder for the CROHME dataset. Later, the proposed model was validated using CNN-based encoder and end-to-end encoder decoder techniques. The proposed model is also validated on other datasets.

Results

The RNN-based encoder model yields 82.78%, while the CNN-based encoder model and end-to-end encoder-decoder technique results in 81.38% and 80.73%, respectively.

Conclusion

1.6% accuracy improvement was attained over CNN-based encoder while 2.4% accuracy improvement over end-to-end encoder-decoder. CROHME dataset 2019 version results in better accuracy than other datasets.

© 2025 Bentham Science Publishers
Loading

Article metrics loading...

/content/journals/rascs/10.2174/0126662558288845240521075304
2024-05-29
2025-01-15

  • From This Site

    /content/journals/rascs/10.2174/0126662558288845240521075304
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. BabliM. RinconJ.A. OnaindiaE. CarrascosaC. JulianV. Deliberative context-aware ambient intelligence system for assisted living homes Hum.-Cent. Comput. Inf. Sci.20211119
     [Google Scholar]
  2. ZhelezniakovD. ZaytsevV. RadyvonenkoO. Online handwritten mathematical expression recognition and applications: A survey.IEEE Access20219383523837310.1109/ACCESS.2021.3063413
     [Google Scholar]
  3. KhowajaS.A. YahyaB.N. LeeS.L. CAPHAR: context-aware personalized human activity recognition using associative learning in smart environments.Human-centric Computing and Information Sciences20201013510.1186/s13673‑020‑00240‑y
     [Google Scholar]
  4. ChanK.F. YeungD.Y. Mathematical expression recognition: A survey.Int. J. Doc. Anal. Recognit.20003131510.1007/PL00013549
     [Google Scholar]
  5. ChanC. Stroke extraction for offline handwritten mathematical expression recognition.IEEE Access20208615656157510.1109/ACCESS.2020.2984627
     [Google Scholar]
  6. ZhangT. New architectures for handwritten mathematical expressions recognition.Image Processing.Nantes, FranceUniversité de Nantes201710.1515/9783110524116
     [Google Scholar]
  7. MillerE.G. ViolaP.A. Ambiguity and constraint in mathematical expression recognition.Am. Assoc. Artif. Intell1998784791
     [Google Scholar]
  8. KosmalaA. RigollG. "On-line handwritten formula recognition using statistical methods", Fourteenth Int. Conf. Pattern Recognit.19981306130810.1109/ICPR.1998.711941
     [Google Scholar]
  9. ÁlvaroF. SánchezJ.A. BenedíJ.M. An integrated grammarbased approach for mathematical expression recognition.Pattern Recognit.20165113514710.1016/j.patcog.2015.09.013
     [Google Scholar]
  10. NaikS.A. MetkewarP.S. MapariS.A. "Recognition of ambiguous mathematical characters within mathematical expressionsProceedings of the 2017 International Conference on Electrical Computer and Communication TechnologiesCoimbatore, India20171410.1109/ICECCT.2017.8117840
     [Google Scholar]
  11. BhardwajC. JainS. SoodM. Deep learning based diabetic retinopathy prediction of colored fundus images with parameter tuning.Advances in Computational Intelligence Techniques. Algorithms for Intelligent Systems. JainS. SoodM. PaulS. SingaporeSpringer2020536210.1007/978‑981‑15‑2620‑6_4
     [Google Scholar]
  12. BhardwajC. JainS. SoodM. Deep learning–based diabetic retinopathy severity grading system employing quadrant ensemble model.J. Digit. Imaging202134244045710.1007/s10278‑021‑00418‑5 33686525
     [Google Scholar]
  13. ÁlvaroF. SánchezJ.A. BenedíJ.M. "Offline features for classifying handwritten math symbols with recurrent neural networks", Proceedings of the 2014 22nd International Conference on Pattern RecognitionStockholm, Sweden20142944294910.1109/ICPR.2014.507
     [Google Scholar]
  14. AwalA.M. MouchèreH. GaudinC.V. Towards handwritten mathematical expression recognitionProceedings of the 10th International Conference on Document Analysis and Recognition200910461050
     [Google Scholar]
  15. AwalA.M. MouchèreH. GaudinC.V. The problem of handwritten mathematical expression recognition evaluationProceedings of the 12th International Conference on Frontiers in Handwriting Recognition2010646651a10.1109/ICFHR.2010.106
     [Google Scholar]
  16. AwalA.M. MouchèreH. GaudinC.V. Improving online handwritten mathematical expressions recognition with contextual modelingProceedings of the 12th International Conference on Frontiers in Handwriting Recognition2010427432b10.1109/ICFHR.2010.73
     [Google Scholar]
  17. AwalA.M. MouchèreH. GaudinC.V. A global learning approach for an online handwritten mathematical expression recognition system.Pattern Recognit. Lett.201235068710.1016/j.patrec.2012.10.024
     [Google Scholar]
  18. AguilarF.J. GaudinC.V. MouchèreH. MedjkouneS. HirataN. Mathematical symbol hypothesis recognition with rejection option14th International Conference on Frontiers in Handwriting Recognition20149 a
     [Google Scholar]
  19. AguilarF.J. GaudinC.V. MouchèreH. MedjkouneS. HirataN. "Integration of shape context and neural networks for symbol recognition", Semaine du Document Numérique et de la Recherched’Information2014 (SDNRI) vol. 9, 2014p. 44 b
     [Google Scholar]
  20. MouchèreH. GaudinC.V. ZanibbiR. GarainU. "ICFHR 2016 CROHME: Competition on recognition of online handwritten mathematical expressions", International Conference on Frontiers in Handwriting Recognition (ICFHR)Shenzhen, China2016 hal- 01374346
     [Google Scholar]
  21. MouchèreH. ZanibbiR. GarainU. Viard-GaudinC. Advancing the state of the art for handwritten math recognition: the CROHME competitions, 2011–2014.Int. J. Doc. Anal. Recognit.201619217318910.1007/s10032‑016‑0263‑5
     [Google Scholar]
  22. Frank Julca-AguilarRecognition of Online Handwritten Mathematical Expressions using Contextual InformationComputer Vision and Pattern Recognition [cs.CV].. Université de Nantes;Université Bretagne Loire; Universidade de São Paulo,2016
     [Google Scholar]
  23. QuiniouS. MouchèreH. SaldarriagaS.P. GaudinC.P. "HAMEX – a Handwritten and Audio Dataset of Mathematical Expressions", International Conference on Document Analysis and Recognition201110.1109/ICDAR.2011.97
     [Google Scholar]
  24. LeA.D. NakagawaM. A system for recognizing online handwritten mathematical expressions by using improved structural analysis.Int. J. Doc. Anal. Recognit.201619430531910.1007/s10032‑016‑0272‑4
     [Google Scholar]
  25. ZhangJ. DuJ. ZhangS. LiuD. HuY. HuJ. WeiS. DaiL. Watch, attend and parse: An end-to-end neural network based approach to handwritten mathematical expression r ecognition.Pattern Recognit.20177119620610.1016/j.patcog.2017.06.017
     [Google Scholar]
  26. MahdaviM. ZanibbiR. Mouch’ereH. Viard-GaudinC. GarainU. "ICDAR 2019 CROHME + TFD: Competition on recognition of handwritten mathematical expressions and typeset formula detection", Proceedings of the 2019 International Conference on Document Analysis and Recognition2019Sydney, NSW, Australia10.1109/ICDAR.2019.00247
     [Google Scholar]
  27. ZhelezniakovD. ZaytsevV. RadyvonenkoO. "Acceleration of Online Recognition of 2D Sequences Using Deep Bidirectional LSTM and Dynamic Programming", Lect. Notes Comput. Sci.20191150743844910.1007/978‑3‑030‑20518‑8_37
     [Google Scholar]
  28. WuJ. YinF. ZhangY. ZhangX. LiuC. Image-to-markup generation via paired adversarial learning.Machine Learning and Knowledge Discovery in Databases.Cham, SwitzerlandSpringer International Publishing2019183410.1007/978‑3‑030‑10925‑7_2
     [Google Scholar]
  29. KrizhevskyA. SutskeverI. HintonG.E. ImageNet classification with deep convolutional neural networks.Commun. ACM2017606849010.1145/3065386
     [Google Scholar]
  30. ZhangJ. DuJ. DaiL. Track, attend and parse (TAP): An endto- end framework for online handwritten mathematical expression recognition.IEEE Trans. Multimed.201921122123310.1109/TMM.2018.2844689
     [Google Scholar]
  31. SinghalP. GuptaS. SinghJ. An integrated approach for analysis of electronic health records using blockchain and deep learningRecent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science)202316911010.2174/2666255816666230509142714
     [Google Scholar]
  32. KusetogullariH. YavariabdiA. HallJ. LavessonN. DIGITNET: A deep handwritten digit detection and recognition method using a new historical handwritten digit dataset.Big Data Research20212310018210.1016/j.bdr.2020.100182
     [Google Scholar]
  33. EndongZ. LichengL. "Design of online handwritten mathematical expression recognition system based on gated recurrent unit recurrent neural network", 2021 4th International Conference on Pattern Recognition and Artificial Intelligence (PRAI),202144645110.1109/PRAI53619.2021.9551034
     [Google Scholar]
  34. CheddadA. KusetogullariH. HilmkilA. SundinL. YavariabdiA. AouacheM. HallJ. SHIBR—the swedish historical birth records: A semi-annotated dataset.Neural Comput. Appl.20213322158631587510.1007/s00521‑021‑06207‑z
     [Google Scholar]
  35. YavariabdiA. KusetogullariH. CelikT. ThummanapallyS. RijwanS. HallJ. Cardis: A swedish historical handwritten character and word dataset.IEEE Access202210553385534910.1109/ACCESS.2022.3175197
     [Google Scholar]
  36. KusetogullariH. YavariabdiA. CheddadA. GrahnH. HallJ. ARDIS: a Swedish historical handwritten digit dataset.Neural Comput. Appl.20203221165051651810.1007/s00521‑019‑04163‑3
     [Google Scholar]
  37. BhartiM. SaxenaS. KumarR. A middleware approach for reliable resource selection on Internet‐of‐Things.Int. J. Commun. Syst.2020335e427810.1002/dac.4278
     [Google Scholar]
  38. JindalH. BhartiM. KasanaS.S. SaxenaS. An ensemble mosaicking and ridgelet based fusion technique for underwater panoramic image reconstruction and its refinement.Multimedia Tools Appl.20238222337193377110.1007/s11042‑023‑14594‑9
     [Google Scholar]
  39. BhartiM. KumarR. SaxenaS. "Architectural survey on internet-of-things", 2019 Fifth International Conference on Image Information Processing (ICIIP)201943744210.1109/ICIIP47207.2019.8985897
     [Google Scholar]
  40. BhartiM. KumarR. SaxenaS. Context‐aware search optimization framework on the internet of things.Concurr. Comput.20183014e442610.1002/cpe.4426
     [Google Scholar]
  41. GuptaA. SrivastavaM. MahantaC. Offline handwritten character recognition using neural network2011 IEEE International Conference on Computer Applications and Industrial Electronics (ICCAIE)201110210710.1109/ICCAIE.2011.6162113
     [Google Scholar]
  42. SahuV.L. KubdeB. "Offline handwritten character recognition techniques using neural network: A review", International journal of science and Research (IJSR)2013218794
     [Google Scholar]
  43. PtuchaR. Petroski SuchF. PillaiS. BrocklerF. SinghV. HutkowskiP. Intelligent character recognition using fully convolutional neural networks.Pattern Recognit.20198860461310.1016/j.patcog.2018.12.017
     [Google Scholar]
  44. WuY.C. YinF. LiuC.L. Improving handwritten Chinese text recognition using neural network language models and convolutional neural network shape models.Pattern Recognit.20176525126410.1016/j.patcog.2016.12.026
     [Google Scholar]
  45. BlucheT. NeyH. KermorvantC. Feature extraction with convolutional neural networks for handwritten word recognition2013 12th international conference on document analysis and recognition201328528910.1109/ICDAR.2013.64
     [Google Scholar]
  46. WuC. FanW. HeY. SunJ. NaoiS. "Handwritten character recognition by alternately trained relaxation convolutional neural network", 2014 14th International Conference on Frontiers in Handwriting Recognition201429129610.1109/ICFHR.2014.56
     [Google Scholar]
  47. DasD. NayakD.R. DashR. MajhiB. ZhangY.D. HWordNet: A holistic convolutional neural network approach for handwritten word recognition.IET Image Process.20201491794180510.1049/iet‑ipr.2019.1398
     [Google Scholar]
  48. DegtyarenkoI. RadyvonenkoO. BokhanK. KhomenkoV. Text/shape classifier for mobile applications with handwriting input.Int. J. Doc. Anal. Recognit.201619436937910.1007/s10032‑016‑0276‑0
     [Google Scholar]
  49. KimH.C. LeeS.W. Document summarization model based on general context in RNN.J. Inf. Process. Syst.20191513781391
     [Google Scholar]
  50. OmK. BoukorosS. NugaliyaddeA. McGillT. DixonM. KoutsakisP. WongK. “Modelling email traffic workloads with RNN and LSTM models”, Hum.-.Cent. Comput. Inf. Sci.202010116
     [Google Scholar]
  51. BhartiM. BalaA. Workflow management in cloud computing.Int. J. Appl. Inf. Syst.201249212610.5120/ijais12‑450788
     [Google Scholar]
  52. KaurJ. KaurK. BhartiM. SharmaP. KaurJ. Reconstruction using compressive sensing: A review.Int. J. Adv. Res. Comput. Commun. Eng.201329
     [Google Scholar]
  53. KaurM. BhartiM. Securing user data on cloud using Fog computingand Decoy technique.International Journal of Advance Research in Computer Science and Management Studies2014210104110
     [Google Scholar]
  54. BhartiM. JindalH. Optimized clustering-based discovery framework on Internet of Things.J. Supercomput.20217721739177810.1007/s11227‑020‑03315‑w
     [Google Scholar]
  55. ShanmugavelS.B. Devi RangaswamyK. MuthukannanM. "A deep learning model to detect fake news about COVID-19", Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science)20231695866
     [Google Scholar]
  56. FarooqO. SinghP. "Data analytics and modeling in iot-fog environment for resource constrained iot-applications: A review", Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science)2022157 96899110.2174/2666255814666210715161630
     [Google Scholar]
  57. RamalingamS.P. ShanmugamP.K. "A comprehensive review on wired and wireless communication technologies and challenges in smart residential buildings", Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science)20221591140116710.2174/2666255814666210119142742
     [Google Scholar]
  58. KumarM.R. ArthiK. "Non-cooperative iris segmentation: A two-stage approach", Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science),20211461719172710.2174/2213275912666191114144935
     [Google Scholar]
  59. OlahC. Understanding LSTM Networks.2015Available from: https://colah.github.io/posts/2015-08-Understanding-LSTMs
    [Google Scholar]
  60. WoodT. Softmax Function.2019Available from https://deepai.org/machine-learning-glossary-and-terms/softmax-layer
    [Google Scholar]
/content/journals/rascs/10.2174/0126662558288845240521075304
Loading
Graphical User Interface for Handwritten Mathematical Expression Employing RNN-based Encoder-decoder Model
Recent Advances in Computer Science and Communications 18, E290524230473 (2025); https://doi.org/10.2174/0126662558288845240521075304
/content/journals/rascs/10.2174/0126662558288845240521075304
/content/journals/rascs/10.2174/0126662558288845240521075304
Loading

Data & Media loading...


  • Article Type:     Research Article
Keyword(s): aspect-ratio; Deep neural network; encoder-decoder; handwritten mathematical expressions; user interface

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test