nach oben

International Journal on Digital Libraries

23.01.2024

A BERT-based sequential deep neural architecture to identify contribution statements and extract phrases for triplets from scientific publications

verfasst von: Komal Gupta, Ammaar Ahmad, Tirthankar Ghosal, Asif Ekbal

Erschienen in: International Journal on Digital Libraries

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Research in Natural Language Processing (NLP) is increasing rapidly; as a result, a large number of research papers are being published. It is challenging to find the contributions of the research paper in any specific domain from the huge amount of unstructured data. There is a need for structuring the relevant contributions in Knowledge Graph (KG). In this paper, we describe our work to accomplish four tasks toward building the Scientific Knowledge Graph (SKG). We propose a pipelined system that performs contribution sentence identification, phrase extraction from contribution sentences, Information Units (IUs) classification, and organize phrases into triplets (subject, predicate, object) from the NLP scholarly publications. We develop a multitasking system (ContriSci) for contribution sentence identification with two supporting tasks, viz. Section Identification and Citance Classification. We use the Bidirectional Encoder Representations from Transformers (BERT)—Conditional Random Field (CRF) model for the phrase extraction and train with two additional datasets: SciERC and SciClaim. To classify the contribution sentences into IUs, we use a BERT-based model. For the triplet extraction, we categorize the triplets into five categories and classify the triplets with the BERT-based classifier. Our proposed approach yields the F1 score values of 64.21%, 77.47%, 84.52%, and 62.71% for the contribution sentence identification, phrase extraction, IUs classification, and triplet extraction, respectively, for non-end-to-end setting. The relative improvement for contribution sentence identification, IUs classification, and triplet extraction is 8.08, 2.46, and 2.31 in terms of F1 score for the NLPContributionGraph (NCG) dataset. Our system achieves the best performance (57.54% F1 score) in the end-to-end pipeline with all four sub-tasks combined. We make our codes available at: https://github.com/92Komal/pipeline_triplet_extraction.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

https://kmcs.nii.ac.jp/resource/AASC/AASC.html.

https://github.com/ncg-task/training-data.

https://github.com/ncg-task/trial-data.

https://github.com/ncg-task/test-data.

https://github.com/kermitt2/grobid.

B, I, and O denote the beginning, intermediate, and outside Named Entities (NEs).

https://kmcs.nii.ac.jp/resource/AASC/AASC.html.

https://huggingface.co/allenai/.

https://huggingface.co/docs/transformers/model_doc/bert#dtatransformers.BertForSequenceClassification.

https://competitions.codalab.org/competitions.

Abbas, A., Zhang, L., Khan, S.U.: A literature review on the state-of-the-art in patent analysis. World Patent Inf. 37, 3–13 (2014)CrossRef

Abney, S.: Bootstrapping. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 360–367 (2002)

Al-Zaidy, R.A., Caragea, C., Giles, C.L.: Bi-lstm-crf sequence labeling for keyphrase extraction from scholarly documents. In: Liu, L., White, R.W., Mantrach, A., et al. (eds.), The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019. ACM, pp. 2551–2557. https://doi.org/10.1145/3308558.3313642 (2019)

Alipourfard, N., Arendt, B., Benjamin, D.J., et al.: Systematizing confidence in open research and evidence (score) (2021)

Arora, H., Ghosal, T., Kumar, S., et al.: INNOVATORS at semeval-2021 task-11: A dependency parsing and bert-based model for extracting contribution knowledge from scientific papers. In: Palmer, A., Schneider, N., Schluter, N., et al. (eds.), Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval@ACL/IJCNLP 2021, Virtual Event/Bangkok, Thailand, August 5-6, 2021. Association for Computational Linguistics, pp. 502–510. https://doi.org/10.18653/v1/2021.semeval-1.61 (2021)

Arslan, Y., Allix, K., Veiber, L., et al.: A comparison of pre-trained language models for multi-class text classification in the financial domain. In: Leskovec, J., Grobelnik, M., Najork, M., et al. (eds.), Companion of the Web Conference 2021, Virtual Event/Ljubljana, Slovenia, April 19-23, 2021. ACM/IW3C2, pp 260–268. https://doi.org/10.1145/3442442.3451375 (2021)

Augenstein, I., Das, M., Riedel, S., et al.: Semeval 2017 task 10: Scienceie—extracting keyphrases and relations from scientific publications. In: Bethard, S., Carpuat, M., Apidianaki, M., et al. (eds.), Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval@ACL 2017, Vancouver, Canada, August 3-4, 2017. Association for Computational Linguistics, pp. 546–555. https://doi.org/10.18653/v1/S17-2091 (2017)

Báez, M., Birukou, A., Casati, F., et al.: Addressing information overload in the scientific community. IEEE Int. Comput. 14(6), 31–38 (2010). https://doi.org/10.1109/MIC.2010.107CrossRef

Basha, C.Z., Pravallika, B.N.L., Shankar, E.B.: An efficient face mask detector with pytorch and deep learning. EAI Endorsed. Trans. Pervasive Health Technol. 7(25), e4 (2021). https://doi.org/10.4108/eai.8-1-2021.167843CrossRef

10.

Beltagy, I., Lo, K., Cohan, A.: Scibert: A pretrained language model for scientific text. In: Inui, K., Jiang, J., Ng, V., et al. (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, pp. 3613–3618. https://doi.org/10.18653/v1/D19-1371 (2019)

11.

Bordons, M., Fernández, M.T., Gómez, I.: Advantages and limitations in the use of impact factor measures for the assessment of research performance. Scientometrics 53(2), 195–206 (2002). https://doi.org/10.1023/A:1014800407876CrossRef

12.

Boudin, F.: Unsupervised keyphrase extraction with multipartite graphs. In: Walker, M.A., Ji, H., Stent, A. (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers). Association for Computational Linguistics, pp. 667–672. https://doi.org/10.18653/v1/n18-2105 (2018)

13.

Bougouin, A., Boudin, F., Daille, B.: Topicrank: Graph-based topic ranking for keyphrase extraction. In: 6th International Joint Conference on Natural Language Processing, IJCNLP 2013, Nagoya, Japan, October 14-18, 2013. Asian Federation of Natural Language Processing/ACL, pp. 543–551. https://aclanthology.org/I13-1062/ (2013)

14.

Brack, A., D’Souza, J., Hoppe, A., et al.: Domain-independent extraction of scientific concepts from research articles. In: Jose, J.M., Yilmaz, E., Magalhães, J., et al. (eds.), Advances in Information Retrieval—42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14-17, 2020, Proceedings, Part I, Lecture Notes in Computer Science, vol 12035. Springer, pp. 251–266. https://doi.org/10.1007/978-3-030-45439-5_17 (2020)

15.

Codalab—competition (2021). https://competitions.codalab.org/competitions/25680#results

16.

Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997). https://doi.org/10.1023/A:1007379606734MathSciNetCrossRef

17.

Celebi, M.E., Aydin, K.: Unsupervised Learning Algorithms, vol. 9. Springer, Berlin (2016)CrossRef

18.

Cohan, A., Ammar, W., van Zuylen, M., et al.: Structural scaffolds for citation intent classification in scientific publications. In: Burstein, J., Doran, C., Solorio, T. (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). Association for Computational Linguistics, pp. 3586–3596. https://doi.org/10.18653/v1/n19-1361 (2019)

19.

Dawes, M., Sampson, U.: Knowledge management in clinical practice: a systematic review of information seeking behavior in physicians. Int. J. Med. Inform. 71(1), 9–15 (2003). https://doi.org/10.1016/S1386-5056(03)00023-6CrossRef

20.

D’Souza, J., Auer, S.: Graphing contributions in natural language processing research: intra-annotator agreement on a trial dataset. arXiv:2010.04388 (2020)

21.

D’Souza, J., Auer, S.: Nlpcontributions: An annotation scheme for machine reading of scholarly contributions in natural language processing literature. In: Zhang, C., Mayr, P., Lu, W., et al. (eds.), Proceedings of the 1st Workshop on Extraction and Evaluation of Knowledge Entities from Scientific Documents co-located with the ACM/IEEE Joint Conference on Digital Libraries in 2020, EEKE@JCDL 2020, Virtual Event, China, August 1st, 2020, CEUR Workshop Proceedings, vol. 2658. CEUR-WS.org, pp. 16–27. http://ceur-ws.org/Vol-2658/paper2.pdf (2020)

22.

D’Souza, J., Auer, S.: Sentence, phrase, and triple annotations to build a knowledge graph of natural language processing contributions - A trial dataset. J. Data Inf. Sci. 6(3), 6–34 (2021). https://doi.org/10.2478/jdis-2021-0023CrossRef

23.

D’Souza, J., Auer, S., Pedersen, T.: Semeval-2021 task 11: Nlpcontributiongraph—structuring scholarly NLP contributions for a research knowledge graph. In: Palmer, A., Schneider, N., Schluter, N., et al. (eds.), Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval@ACL/IJCNLP 2021, Virtual Event / Bangkok, Thailand, August 5-6, 2021. Association for Computational Linguistics, pp. 364–376. https://doi.org/10.18653/v1/2021.semeval-1.44 (2021)

24.

Enduri, M.K., Sankar, V.U., Hajarathaiah, K.: Empirical study on citation count prediction of research articles. J. Scientometr. Res. 11(2), 155–163 (2022)CrossRef

25.

Gábor, K., Buscaldi, D., Schumann, A., et al.: Semeval-2018 task 7: Semantic relation extraction and classification in scientific papers. In: Apidianaki, M., Mohammad, S.M., May, J., et al. (eds.), Proceedings of The 12th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018. Association for Computational Linguistics, pp. 679–688. https://doi.org/10.18653/v1/s18-1111 (2018)

26.

Grineva, M.P., Grinev, M.N., Lizorkin, D.: Extracting key terms from noisy and multitheme documents. In: Quemada, J., León, G., Maarek, Y.S., et al. (eds.), Proceedings of the 18th International Conference on World Wide Web, WWW 2009, Madrid, Spain, April 20-24, 2009. ACM, pp. 661–670. https://doi.org/10.1145/1526709.1526798 (2009)

27.

Gupta, S., Manning, C.D.: Analyzing the dynamics of research by extracting key aspects of scientific papers. In: 5th International Joint Conference on Natural Language Processing, IJCNLP 2011, Chiang Mai, Thailand, November 8-13, 2011. The Association for Computer Linguistics, pp. 1–9. https://aclanthology.org/I11-1001/ (2011)

28.

Hasan, K.S., Ng, V.: Conundrums in unsupervised keyphrase extraction: Making sense of the state-of-the-art. In: Huang, C., Jurafsky, D. (eds.), COLING 2010, 23rd International Conference on Computational Linguistics, Posters Volume, 23-27 August 2010, Beijing, China. Chinese Information Processing Society of China, pp. 365–373. https://aclanthology.org/C10-2042/ (2010)

29.

He, P., Liu, X., Gao, J., et al.: Deberta: decoding-enhanced bert with disentangled attention. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https://openreview.net/forum?id=XPZIaotutsD (2021)

30.

Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef

31.

Hulth, A.: Improved automatic keyword extraction given more linguistic knowledge. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2003, Sapporo, Japan, July 11-12, 2003. https://aclanthology.org/W03-1028/ (2003)

32.

Jaiswal, A., George, V.: A modified approach for extraction and association of triplets. In: International Conference on Computing, Communication & Automation. IEEE, pp. 36–40 (2015)

33.

Jivani, M.A.G., Shingala, M.A.H., Virparia, P.V.: The multi-liaison algorithm. IJACSA Int. J. Adv. Comput. Sci. Appl. 2(5) (2011)

34.

Johnson, R., Watkinson, A., Mabe, M.: The STM report. An overview of scientific and scholarly publishing 5th edition October (2018)

35.

Kim, T.K.: T test as a parametric statistic. Korean J. Anesthesiol. 68(6), 540–546 (2015)MathSciNetCrossRef

36.

Klementiev, A., Roth, D., Small, K.: Unsupervised rank aggregation with distance-based models. In: Cohen, W.W., McCallum, A., Roweis, S.T. (eds.), Machine Learning, Proceedings of the Twenty-Fifth International Conference (ICML 2008), Helsinki, Finland, June 5-9, 2008, ACM International Conference Proceeding Series, vol. 307. ACM, pp. 472–479. https://doi.org/10.1145/1390156.1390216 (2008)

37.

Lin, J., Ling, J., Wang, Z., et al.: ECNUICA at semeval-2021 task 11: Rule based information extraction pipeline. In: Palmer, A., Schneider, N., Schluter, N., et al. (eds.), Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval@ACL/IJCNLP 2021, Virtual Event/Bangkok, Thailand, August 5-6, 2021. Association for Computational Linguistics, pp. 1295–1302. https://doi.org/10.18653/v1/2021.semeval-1.185 (2021)

38.

Liu, F., Pennell, D., Liu, F., et al.: Unsupervised approaches for automatic keyword extraction using meeting transcripts. In: Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, May 31–June 5, 2009, Boulder, Colorado, USA. The Association for Computational Linguistics, pp. 620–628. https://aclanthology.org/N09-1070/ (2009)

39.

Liu, H., Sarol, M.J., Kilicoglu, H.: Uiuc_bionlp at semeval-2021 task 11: A cascade of neural models for structuring scholarly NLP contributions. In: Palmer, A., Schneider, N., Schluter, N., et al. (eds.), Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval@ACL/IJCNLP 2021, Virtual Event/Bangkok, Thailand, August 5-6, 2021. Association for Computational Linguistics, pp. 377–386. https://doi.org/10.18653/v1/2021.semeval-1.45 (2021)

40.

Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: Kambhampati, S. (ed.), Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016. IJCAI/AAAI Press, pp. 2873–2879. http://www.ijcai.org/Abstract/16/408 (2016)

41.

Liu, P., Qiu, X., Huang, X.: Adversarial multi-task learning for text classification. In: Barzilay, R., Kan, M. (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers. Association for Computational Linguistics, pp. 1–10. https://doi.org/10.18653/v1/P17-1001 (2017)

42.

Liu, Z., Rao, B.: Characterization of polynomial decay rate for the solution of linear evolution equation. Zeitschrift für angewandte Mathematik und Physik ZAMP 56, 630–644 (2005)MathSciNetCrossRef

43.

Llugsi, R., El Yacoubi, S., Fontaine, A., et al.: Comparison between Adam, AdaMax and Adam W optimizers to implement a Weather Forecast based on Neural Networks for the Andean city of Quito. In: 2021 IEEE Fifth Ecuador Technical Chapters Meeting (ETCM), IEEE, pp. 1–6 (2021)

44.

Luan, Y., He, L., Ostendorf, M., et al.: Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In: Riloff, E., Chiang, D., Hockenmaier, J., et al. (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. Association for Computational Linguistics, pp. 3219–3232. https://doi.org/10.18653/v1/d18-1360 (2018)

45.

Ma, X., Wang, J., Zhang, X.: YNU-HPCC at semeval-2021 task 11: Using a BERT model to extract contributions from NLP scholarly articles. In: Palmer, A., Schneider, N., Schluter, N., et al., (eds.), Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval@ACL/IJCNLP 2021, Virtual Event / Bangkok, Thailand, August 5-6, 2021. Association for Computational Linguistics, pp. 478–484. https://doi.org/10.18653/v1/2021.semeval-1.58 (2021)

46.

MacCartney, B.: Natural Language Inference. Stanford University, Stanford (2009)

47.

Magnusson, I.H., Friedman, S.E.: Extracting fine-grained knowledge graphs of scientific claims: Dataset and transformer-based results. In: Moens, M., Huang, X., Specia, L., et al. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event/Punta Cana, Dominican Republic, 7-11 November, 2021. Association for Computational Linguistics, pp. 4651–4658. https://doi.org/10.18653/v1/2021.emnlp-main.381 (2021)

48.

Mahata, D., Kuriakose, J., Shah, R.R., et al.: Key2vec: Automatic ranked keyphrase extraction from scientific articles using phrase embeddings. In: Walker, M.A., Ji, H., Stent, A. (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers). Association for Computational Linguistics, pp. 634–639. https://doi.org/10.18653/v1/n18-2100 (2018)

49.

Mansuri, I.R., Sarawagi, S.: Integrating unstructured data into relational databases. In: Liu, L., Reuter, A., Whang, K., et al., (eds.), Proceedings of the 22nd International Conference on Data Engineering, ICDE 2006, 3-8 April 2006, Atlanta, GA, USA. IEEE Computer Society, p. 29. https://doi.org/10.1109/ICDE.2006.83 (2006)

50.

Martin, A., Pedersen, T.: Duluth at semeval-2021 task 11: Applying deberta to contributing sentence selection and dependency parsing for entity extraction. In: Palmer, A., Schneider N., Schluter, N., et al. (eds.), Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval@ACL/IJCNLP 2021, Virtual Event / Bangkok, Thailand, August 5-6, 2021. Association for Computational Linguistics, pp. 490–501. https://doi.org/10.18653/v1/2021.semeval-1.60 (2021)

51.

Medelyan, O., Frank, E., Witten, I.H.: Human-competitive tagging using automatic keyphrase extraction. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, 6-7 August 2009, Singapore, A meeting of SIGDAT, a Special Interest Group of the ACL. ACL, pp. 1318–1327. https://aclanthology.org/D09-1137/ (2009)

52.

Mihalcea, R., Tarau, P.: Textrank: bringing order into text. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing , EMNLP 2004, A meeting of SIGDAT, a Special Interest Group of the ACL, held in conjunction with ACL 2004, 25-26 July 2004, Barcelona, Spain. ACL, pp. 404–411. https://aclanthology.org/W04-3252/ (2004)

53.

Mishra, S., Mishra, D.: SVM-BT-RFE: an improved gene selection framework using Bayesian T-test embedded in support vector machine (recursive feature elimination) algorithm. Karbala Int. J. Modern Sci. 1(2), 86–96 (2015)CrossRef

54.

Molchanov, D., Ashukha, A., Vetrov, D.P.: Variational dropout sparsifies deep neural networks. In: Precup, D., Teh, Y.W. (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, Proceedings of Machine Learning Research, vol. 70. PMLR, pp. 2498–2507. http://proceedings.mlr.press/v70/molchanov17a.html (2017)

55.

Nguyen, G.H., Bouzerdoum, A., Phung, S.L.: A supervised learning approach for imbalanced data sets. In: 19th International Conference on Pattern Recognition (ICPR 2008), December 8-11, 2008, Tampa, Florida, USA. IEEE Computer Society, pp. 1–4. https://doi.org/10.1109/ICPR.2008.4761278 (2008)

56.

Pascanu, R., Gülçehre ,Ç., Cho, K., et al.: How to construct deep recurrent neural networks. In: Bengio, Y., LeCun, Y. (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings. arXiv:1312.6026 (2014)

57.

Poibeau, T.: Machine Translation. MIT Press, Cambridge (2017)CrossRef

58.

Qi, P., Zhang, Y., Zhang, Y., et al.: Stanza: A python natural language processing toolkit for many human languages. In: Celikyilmaz, A., Wen, T. (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2020, Online, July 5-10, 2020. Association for Computational Linguistics, pp. 101–108. https://doi.org/10.18653/v1/2020.acl-demos.14 (2020)

59.

Ruder, S.: An overview of multi-task learning in deep neural networks. arXiv:1706.05098 (2017)

60.

Rusu, D., Dali, L., Fortuna, B., et al.: Triplet extraction from sentences. In: Proceedings of the 10th International Multiconference” Information Society-IS, pp. 8–12 (2007)

61.

Sahrawat, D., Mahata, D., Zhang, H., et al.: Keyphrase extraction as sequence labeling using contextualized embeddings. In: Jose, J.M., Yilmaz, E., Magalhães, J., et al. (eds.), Advances in Information Retrieval—42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14-17, 2020, Proceedings, Part II, Lecture Notes in Computer Science, vol. 12036. Springer, pp. 328–335. https://doi.org/10.1007/978-3-030-45442-5_41 (2020)

62.

Sanh, V., Debut, L., Chaumond, J., et al.: Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv:1910.01108 (2019)

63.

Schafer, J.B., Konstan, J.A., Riedl, J.: E-commerce recommendation applications. Data Min. Knowl. Discov. 5, 115–153 (2001)CrossRef

64.

Shailabh, S., Chaurasia, S., Modi, A.: Knowgraph@iitk at semeval-2021 task 11: building knowledgegraph for NLP research. arXiv:2104.01619 (2021)

65.

Sharma, S., Sharma, S., Athaiya, A.: Activation functions in neural networks. Towards Data Sci. 6(12), 310–316 (2017)

66.

Shi, W., Zheng, W., Yu, J.X., et al.: Keyphrase extraction using knowledge graphs. Data Sci. Eng. 2(4), 275–288 (2017). https://doi.org/10.1007/s41019-017-0055-zCrossRef

67.

Souza, F., Nogueira, R.F., de Alencar Lotufo, R.: Portuguese named entity recognition using BERT-CRF. arXiv:1909.10649 (2019)

68.

Sutton, C., McCallum, A., Rohanimanesh, K.: Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data. J. Mach. Learn. Res. 8, 693–723 (2007). https://doi.org/10.5555/1314498.1314523CrossRef

69.

Trewartha, A., Walker, N., Huo, H., et al.: Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science. Patterns 3(4), 100488 (2022). https://doi.org/10.1016/j.patter.2022.100488CrossRef

70.

Vieira, S.M., Kaymak, U., Sousa, J.M.C.: Cohen’s kappa coefficient as a performance measure for feature selection. In: FUZZ-IEEE 2010, IEEE International Conference on Fuzzy Systems, Barcelona, Spain, 18-23 July, 2010, Proceedings. IEEE, pp. 1–8. https://doi.org/10.1109/FUZZY.2010.5584447 (2010)

71.

Wade, A.D.: The semantic scholar academic graph (s2ag). Companion Proc. Web Conf. 2022, 739–739 (2022)

72.

Wan, X., Xiao, J.: Single document keyphrase extraction using neighborhood knowledge. In: Fox, D., Gomes, C.P. (eds.), Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008. AAAI Press, pp. 855–860. http://www.aaai.org/Library/AAAI/2008/aaai08-136.php (2008)

73.

Wang, K., Shen, Z., Huang, C., et al.: Microsoft academic graph: when experts are not enough. Quant. Sci. Stud. 1(1), 396–413 (2020). https://doi.org/10.1162/qss_a_00021CrossRef

74.

Wang, L.L., Lo, K., Chandrasekhar, Y., et al.: CORD-19: the COVID-19 open research dataset. arXiv:2004.10706 (2020)

75.

Wang, M., Zhao, B., Huang, Y.: PTR: phrase-based topical ranking for automatic keyphrase extraction in scientific publications. In: Hirose, A., Ozawa, S., Doya, K., et al. (eds.), Neural Information Processing—23rd International Conference, ICONIP 2016, Kyoto, Japan, October 16-21, 2016, Proceedings, Part IV, pp. 120–128. https://doi.org/10.1007/978-3-319-46681-1_15 (2016)

76.

Wang, R., Liu, W., McDonald, C.: Using word embeddings to enhance keyword identification for scientific publications. In: Sharaf, M.A., Cheema, M.A., Qi, J. (eds.), Databases Theory and Applications—26th Australasian Database Conference, ADC 2015, Melbourne, VIC, Australia, June 4-7, 2015. Proceedings, Lecture Notes in Computer Science, vol. 9093. Springer, pp. 257–268. https://doi.org/10.1007/978-3-319-19548-3_21 (2015)

77.

Welch, B.L.: The generalization of ‘student’s’ problem when several different population varlances are involved. Biometrika 34(1–2), 28–35 (1947)MathSciNet

78.

Wise, C., Ioannidis, V.N., Calvo, M.R., et al.: COVID-19 knowledge graph: accelerating information retrieval and discovery for scientific literature. arXiv:2007.12731 (2020)

79.

Wu, Z., Giles, C.L.: Measuring term informativeness in context. In: Vanderwende III, L. H.D, Kirchhoff, K. (eds.), Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 9-14, 2013, Westin Peachtree Plaza Hotel, Atlanta, Georgia, USA. The Association for Computational Linguistics, pp. 259–269. https://aclanthology.org/N13-1026/ (2013)

80.

Xu, H., AbdelRahman, S., Jiang, M., et al.: An initial study of full parsing of clinical text using the Stanford parser. In: 2011 IEEE International Conference on Bioinformatics and Biomedicine Workshops (BIBMW), IEEE, pp. 607–614 (2011)

81.

Yih, W., Goodman, J., Carvalho, V.R.: Finding advertising keywords on web pages. In: Carr, L., Roure, D.D., Iyengar, A., et al. (eds.), Proceedings of the 15th international conference on World Wide Web, WWW 2006, Edinburgh, Scotland, UK, May 23-26, 2006. ACM, pp. 213–222. https://doi.org/10.1145/1135777.1135813 (2006)

82.

Yu, B., Li, Y., Wang, J.: Detecting causal language use in science findings. In: Inui, K., Jiang, J., Ng, V., et al. (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, pp. 4663–4673. https://doi.org/10.18653/v1/D19-1473 (2019)

83.

Yu, Y., Ng, V.: Wikirank: Improving keyphrase extraction based on background knowledge. arXiv:1803.09000 (2018)

84.

Zhang, G., Su, Y., He, C., et al.: ITNLP at semeval-2021 task 11: Boosting BERT with sampling and adversarial training for knowledge extraction. In: Palmer, A., Schneider, N., Schluter, N., et al. (eds.), Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval@ACL/IJCNLP 2021, Virtual Event/Bangkok, Thailand, August 5-6, 2021. Association for Computational Linguistics, pp. 485–489. https://doi.org/10.18653/v1/2021.semeval-1.59 (2021)

85.

Zhang, J., Li, C., Yin, Y., et al.: Applications of artificial neural networks in microorganism image analysis: a comprehensive review from conventional multilayer perceptron to popular convolutional neural network and potential visual transformer. Artif. Intell. Rev. 56(2), 1013–1070 (2023). https://doi.org/10.1007/s10462-022-10192-7CrossRef

86.

Zhang, Q., Wang, Y., Gong, Y., et al.: Keyphrase extraction using deep recurrent neural networks on twitter. In: Su, J., Carreras, X., Duh, K. (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016. The Association for Computational Linguistics, pp. 836–845. https://doi.org/10.18653/v1/d16-1080 (2016)

87.

Zhang, Y., Milios, E.E., Zincir-Heywood, A.N.: A comparative study on key phrase extraction methods in automatic web site summarization. J. Digit. Inf. Manag. 5(5), 323–332 (2007)

88.

Zhang, Z., Han, X., Liu, Z., et al.: ERNIE: enhanced language representation with informative entities. In: Korhonen, A., Traum, D.R., Màrquez, L. (eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, pp. 1441–1451. https://doi.org/10.18653/v1/p19-1139 (2019)

89.

Zhang, Z., Strubell, E., Hovy, E.H.: A survey of active learning for natural language processing. In: Goldberg, Y., Kozareva, Z., Zhang, Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022. Association for Computational Linguistics, pp. 6166–6190. https://doi.org/10.18653/v1/2022.emnlp-main.414 (2022)

90.

Zhu, X., Lyu, C., Ji, D., et al.: Deep neural model with self-training for scientific Keyphrase extraction. PLoS ONE 15(5), e0232,547 (2020)CrossRef

Titel: A BERT-based sequential deep neural architecture to identify contribution statements and extract phrases for triplets from scientific publications
verfasst von: Komal Gupta
Ammaar Ahmad
Tirthankar Ghosal
Asif Ekbal
Publikationsdatum: 23.01.2024
Verlag: Springer Berlin Heidelberg
Erschienen in: International Journal on Digital Libraries
Print ISSN: 1432-5012
Elektronische ISSN: 1432-1300
DOI: https://doi.org/10.1007/s00799-023-00393-y

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner