iBet uBet web content aggregator. Adding the entire web to your favor.
iBet uBet web content aggregator. Adding the entire web to your favor.



Link to original content: https://doi.org/10.1007/978-3-031-49552-6_29
Polarity Prediction in Tourism Cuban Reviews Using Transformer with Estimation of Distribution Algorithms | SpringerLink
Skip to main content

Polarity Prediction in Tourism Cuban Reviews Using Transformer with Estimation of Distribution Algorithms

  • Conference paper
  • First Online:
Progress in Artificial Intelligence and Pattern Recognition (IWAIPR 2023)

Abstract

The tourism sector has benefited from recent research in the area of natural language processing, where digital platforms on the web offer the opportunity for people to express their opinions about the services and places they visit. The texts of the reviews are unstructured data characterized by high dimensionality, variable size, and complex semantic relationships between words, which has led to the development of neural architectures with a larger number of parameters to optimize. The training of deep neural networks has been approached by methods based on partial derivatives of the objective function and presents several theoretical and practical limitations, such as the probability of convergence to local minima. In this paper, a hybrid method based on Distribution Estimation Algorithms is proposed for fine-tuning an mT5-based Transformer for polarity prediction. For this purpose, a new Spanish dataset is proposed for polarity classification compiled from TripAdvisor reviews of Cuba. Different preprocessing variants are applied and compared in the solution and data imbalance is treated by back translation. The proposed method combined with back translation decreases the mean of the absolute error in the mT5-based Transformer for polarity prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Available at: https://drive.google.com/drive/folders/1pZVxp5DgpD7UEgOnWXKhLm5-pX0YYlNo?usp=sharing.

  2. 2.

    A language pair refers to a translation from a source language to a target language. Example: EN-ES, which indicates translating from English to Spanish.

  3. 3.

    Available at: https://huggingface.co/Helsinki-NLP/opus-mt-es-en.

  4. 4.

    Available at: https://huggingface.co/Helsinki-NLP/opus-mt-en-es.

  5. 5.

    Available at: https://colab.research.google.com/drive/1QdhSMLSOq1jXC4OPJfK6PKsmvc1-RrQa?usp=sharing.

References

  1. Chaturvedi, I., Su, C.L., Welsch, R.E.: Fuzzy aggregated topology evolution for cognitive multi-tasks. Cogn. Comput. 13(1), 96–107 (2021). https://doi.org/10.1007/s12559-020-09807-4

    Article  Google Scholar 

  2. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195 (2001)

    Article  Google Scholar 

  3. Janani, R., Vijayarani, S.: Automatic text classification using machine learning and optimization algorithms. Soft. Comput. 25(2), 1129–1145 (2021). https://doi.org/10.1007/s00500-020-05209-8

    Article  Google Scholar 

  4. Junczys-Dowmunt, M., et al.: Marian: fast neural machine translation in C++. In: ACL 2018–56th Annual Meeting of the Association for Computational Linguistics, Proceedings of System Demonstrations, pp. 116–121 (2015)

    Google Scholar 

  5. Kiefer, J., Wolfowitz, J.: Stochastic estimation of the maximum of a regression function. Ann. Math. Stat. 23(3), 462–466 (1952)

    Article  MathSciNet  Google Scholar 

  6. Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–15 (2015)

    Google Scholar 

  7. Larrañaga, P.: A review on estimation of distribution algorithms. In: Larrañaga, P., Lozano, J.A. (eds.) Estimation of Distribution Algorithms. GENA, vol. 2, pp. 57–100. Springer, Boston (2002). https://doi.org/10.1007/978-1-4615-1539-5_3

    Chapter  Google Scholar 

  8. Larrañaga, P., Lozano, J.A.: Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation, vol. 2. Springer, New York (2002). https://doi.org/10.1007/978-1-4615-1539-5

    Book  Google Scholar 

  9. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: 7th International Conference on Learning Representations, ICLR 2019, pp. 1–8 (2019)

    Google Scholar 

  10. Khataei Maragheh, H., Gharehchopogh, F.S., Majidzadeh, K., Sangar, A.B.: A new hybrid based on long short-term memory network with spotted hyena optimization algorithm for multi-label text classification. Mathematics 10(3), 488 (2022)

    Article  Google Scholar 

  11. Rojas-Delgado, J., Trujillo-Rasúa, R., Bello, R.: A continuation approach for training artificial neural networks with meta-heuristics. Pattern Recogn. Lett. 125, 373–380 (2019)

    Article  Google Scholar 

  12. Salinas Chávez, E., Salinas Chávez, E., Mundet i Cerdan, L.: Tourism in Cuba: development, challenges, perspectives. Revista Rosa dos Ventos - Turismo e Hospitalidade 11(1), 23–49 (2019)

    Google Scholar 

  13. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, NIPS, pp. 5999–6009 (2017)

    Google Scholar 

  14. Xue, L., et al.: mT5: a massively multilingual pre-trained text-to-text transformer. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 483–498 (2021)

    Google Scholar 

Download references

Acknowledgments

This work has been partially funded by VLIR-UOS Network University Cooperation Programme-Cuba. We gratefully acknowledge the computing time granted through UCI-HPC and Computational Mathematics Study Center at the University of Informatics Sciences supercomputer resources.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Orlando Grabiel Toledano-López .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Toledano-López, O.G., Álvarez-Carmona, M.Á., Madera, J., Simón-Cuevas, A., López-Rodríguez, Y.A., González Diéz, H. (2024). Polarity Prediction in Tourism Cuban Reviews Using Transformer with Estimation of Distribution Algorithms. In: Hernández Heredia, Y., Milián Núñez, V., Ruiz Shulcloper, J. (eds) Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2023. Lecture Notes in Computer Science, vol 14335. Springer, Cham. https://doi.org/10.1007/978-3-031-49552-6_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49552-6_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49551-9

  • Online ISBN: 978-3-031-49552-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics