Abstract
As the core work of information extraction, relation extraction aims to find medical entity pairs with relations from the medical records. The current methods of relation extraction either ignore the relevance of entity extraction and relation classification, or fail to solve the problem of multiple relations or entities in one sentence. To handle those problems, this paper proposes a cascading pointer Multi-BERT-wwm model based on the probabilistic graph strategy. The model selects an entity randomly from all the predicted entities each time, then predicts other entities and their relations according to that entity. Meanwhile, the pointer labeling network helps to solve the problem of overlapping entities. The Multi-BERT-wwm model is improved based on BERT, which connects a layer of self-attention to the multi-head attention layer in the first six Encoder modules to strengthen the feature extraction ability. In addition, we add the adversarial training to improve the robustness and generalization ability of the model. The experimental results tested on the CMeIE dataset show that compared with the traditional CNN+Attention and BERT methods, our method improves the F1-score by 4.42% and 1.91% respectively in relation extraction task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bekoulis, G., Deleu, J., Demeester, T., Develder, C.: Joint entity recognition and relation extraction as a multi-head selection problem. Expert Syst. Appl. 114, 34–45 (2018)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. STAT 1050, 20 (2015)
Kumar, S.: A survey of deep learning methods for relation extraction. arXiv preprint arXiv:1705.03645 (2017)
Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics: Long Papers, , vol. 1, pp. 2124–2133 (2016)
Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMs on sequences and tree structures. arXiv preprint arXiv:1601.00770 (2016)
Miyato, T., Dai, A.M., Goodfellow, I.: Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725 (2016)
Nguyen, D.Q., Verspoor, K.: Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings. arXiv preprint arXiv:1805.10586 (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wei, Z., Su, J., Wang, Y., Tian, Y., Chang, Y.: A novel cascade binary tagging framework for relational triple extraction. arXiv preprint arXiv:1909.03227 (2019)
Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., Jin, Z.: Classifying relations via long short term memory networks along shortest dependency paths. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1785–1794 (2015)
Yu, X., Lam, W.: Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach. In: Coling 2010: Posters, pp. 1399–1407 (2010)
Zhang, Y., et al.: A hybrid model based on neural networks for biomedical relation extraction. J. Biomed. Inform. 81, 83–92 (2018)
Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., Xu, B.: Joint extraction of entities and relations based on a novel tagging scheme. In: ACL, no. 1 (2017)
Zhou, Z.H.: A brief introduction to weakly supervised learning. Natl. Sci. Rev. 5(1), 44–53 (2018)
Acknowledgment
This work is supported by multiple grants, including: The National Key Research and Development Program of China (2020YFB1313900), National Natural Science Foundation of China (61902386, 62072452), Shenzhen Science and Technology Program (JCYJ20180507182415428), and Research on digital battlefield rescue integrated command Platform and auxiliary Equipment (CX19024).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Y., Liao, X., Chen, L., Kang, H., Cai, Y., Wang, Q. (2021). Multi-BERT-wwm Model Based on Probabilistic Graph Strategy for Relation Extraction. In: Siuly, S., Wang, H., Chen, L., Guo, Y., Xing, C. (eds) Health Information Science. HIS 2021. Lecture Notes in Computer Science(), vol 13079. Springer, Cham. https://doi.org/10.1007/978-3-030-90885-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-90885-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90884-3
Online ISBN: 978-3-030-90885-0
eBook Packages: Computer ScienceComputer Science (R0)