Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
ATZ-Fachkongress

Chassis.tech plus 2024


4 Kongresse in einer Veranstaltung

Treffen Sie in München die Fachleute von Chassis, Lenkung, Bremsen sowie Reifen/Räder.
Erweiterte Suche
Anmelden
nach oben
Artificial Intelligence and Law

17.05.2024 | Original Research

Self-training improves few-shot learning in legal artificial intelligence tasks

verfasst von: Yulin Zhou, Yongbin Qin, Ruizhang Huang, Yanping Chen, Chuan Lin, Yuan Zhou

Erschienen in: Artificial Intelligence and Law

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

As the labeling costs in legal artificial intelligence tasks are expensive. Therefore, it becomes a challenge to utilize low cost to train a robust model. In this paper, we propose a LAIAugment approach, which aims to enhance the few-shot learning capability in legal artificial intelligence tasks. Specifically, we first use the self-training approach to label the amount of unlabelled data to enhance the feature learning capability of the model. Moreover, we also search for datasets that are similar to the training set by improving the text similarity function. We conducted experimental analyses for three legal artificial intelligence tasks, including evidence extraction, legal element extraction, and case multi-label prediction, which composed of 3500 judgement documents. The experimental results show that the proposed LAIAugment method has an average F1-score of 72.3% on the three legal AI tasks, which is 1.93% higher than the baseline model. At the same time, it shows a huge improvement in few-shot learning.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
Literatur
Arora S, Liang Y, Ma T (2017) A simple but tough-to-beat baseline for sentence embeddings. In: international conference on learning representations, pp 1–16
Bao Y, Wu M, Chang S, Barzilay R (2019) Few-shot text classification with distributional signatures. arXiv preprint arXiv:​1908.​06039
Bhattacharya P, Paul S, Ghosh K, Ghosh S, Wyner A (2023) Deeprhole: deep learning for rhetorical role labeling of sentences in legal case documents. Artif Intell Law 31(1):53–90CrossRef
Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inform Process Syst 33:1877–1901
Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inform Process Syst 33:1877–1901
Chalkidis I, Fergadiotis M, Malakasiotis P, Aletras N, Androutsopoulos I (2020) Legal-bert: the muppets straight out of law school. Find Assoc Comput Linguis EMNLP 2020:2898–2904
Chen Q, Yang L, Lai J-H, Xie X (2022a) Self-supervised image-specific prototype exploration for weakly supervised semantic segmentation. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4288–4298
Chen X, Shi Z, Qiu X, Huang X-J (2017) Adversarial multi-criteria learning for chinese word segmentation. In: proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: Long Papers), pp 1193–1203
Chen Y-S, Chiang S-W, Wu M-L (2022) A few-shot transfer learning approach using text-label embedding with legal attributes for law article prediction. Appl Intell 52(3):2884–2902CrossRef
Dhani J. S, Bhatt R, Ganesan B, Sirohi P, Bhatnagar V (2021) Similar cases recommendation using legal knowledge graphs. arXiv preprint arXiv:​2107.​04771
Du J, Grave É, Gunel B, Chaudhary V, Celebi O, Auli M, Stoyanov V, Conneau A (2021) Self-training improves pre-training for natural language understanding. In Proceedings of the 2021 Conference of the North American chapter of the association for computational linguistics: human language technologies, pp 5408–5418
Ethayarajh K (2018) Unsupervised random walk sentence embeddings: a strong but simple baseline. In: proceedings of the third workshop on representation learning for NLP, pp 91–100
Feng G, Qin Y, Huang R, Chen Y (2023) Criminal action graph: a semantic representation model of judgement documents for legal charge prediction. Inform Process Manag 60(5):103421CrossRef
Gong J, Chen X, Gui T, Qiu X (2019) Switch-lstms for multi-criteria Chinese word segmentation. Proc AAAI Confer Artif Intell 33:6457–6464
Gu Y, Han X, Liu Z, Huang M (2022) Ppt: Pre-trained prompt tuning for few-shot learning. In: proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers), pp 8410–8423
He H, Wu L, Yan H, Gao Z, Feng Y, Townsend G (2019) Effective neural solution for multi-criteria word segmentation. In: smart intelligent computing and applications: proceedings of the second international conference on SCI 2018, Volume 2, pp 133–142. Springer
Huang Y-X, Dai W-Z, Yang J, Cai L-W, Cheng S, Huang R, Li Y-F, Zhou Z-H. (2020) Semi-supervised abductive learning and its application to theft judicial sentencing. In: 2020 IEEE international conference on data mining (ICDM), pp 1070–1075. IEEE
Karamanolakis G, Mukherjee S, Zheng G, Hassan A (2021) Self-training with weak supervision. In: proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 845–863
Kenton JDM-WC, Toutanova LK (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In proceedings of NAACL-HLT, pp 4171–4186
Liu X, Yin D, Feng Y, Wu Y, and Zhao D (2021) Everything has a cause: leveraging causal inference in legal text analysis. In: proceedings of the 2021 conference of the North American Chapter of the association for computational linguistics: human language technologies, pp 1928–1941
Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:​1907.​11692
Ma J, Ganchev K, Weiss D (2018) State-of-the-art chinese word segmentation with bi-lstms. In: proceedings of the 2018 conference on empirical methods in natural language processing, pp 4902–4908
Mehta S. V, Rao J, Tay Y, Kale M, Parikh A, Strubell E (2022) Improving compositional generalization with self-training for data-to-text generation. In Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers), pp 4205–4219
Meng Y, Zhang Y, Huang J, Wang X, Zhang Y, Ji H, Han J (2021) Distantly-supervised named entity recognition with noise-robust learning and language model augmented self-training. In: proceedings of the 2021 conference on empirical methods in natural language processing, pp 10367–10378
Mi F, Zhou W, Kong L, Cai F, Huang M, Faltings B (2021) Self-training improves pre-training for few-shot learning in task-oriented dialog systems. In: proceedings of the 2021 conference on empirical methods in natural language processing, pp 1887–1898
Mikolov T, Grave É, Bojanowski P, Puhrsch C, Joulin A (2018) Advances in pre-training distributed word representations. In: proceedings of the eleventh international conference on language resources and evaluation (LREC 2018), pp 1–4
Mueller A, Krone J, Romeo S, Mansour S, Mansimov E, Zhang Y, Roth D (2022) Label semantic aware pre-training for few-shot text classification. In: proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: Long Papers), pp 8318–8334
Mukherjee S, Awadallah AH (2020) Uncertainty-aware self-training for text classification with few labels. arXiv preprint arXiv:​2006.​15315
Nguyen D-H, Nguyen B-S, Nghiem NVD, Le DT, Khatun MA, Nguyen M-T, Le H (2021) Robust deep reinforcement learning for extractive legal summarization. In: neural information processing: 28th international conference, ICONIP 2021, Sanur, Bali, Indonesia, December 8–12, 2021, proceedings, Part VI 28, pp 597–604. Springer
Niu Y, Jiao F, Zhou M, Yao T, Xu J, Huang M (2020) A self-training method for machine reading comprehension with soft evidence extraction. In: proceedings of the 58th annual meeting of the association for computational linguistics, pp 3916–3927
Peng B, Li C, Li J, Shayandeh S, Liden L, Gao J (2020a) Soloist: Few-shot task-oriented dialog with a single pretrained auto-regressive model. arXiv preprint arXiv:​2005.​05298, 3
Peng B, Zhu C, Li C, Li X, Li J, Zeng M, Gao J (2020) Few-shot natural language generation for task-oriented dialog. Find Assoc Computat Linguist EMNLP 2020:172–182
Qu X, Zeng J, Liu D, Wang Z, Huai B, Zhou P (2023) Distantly-supervised named entity recognition with adaptive teacher learning and fine-grained student ensemble. Proc AAAI Confer Artif Intell 37:13501–13509
Ratner A, Bach SH, Ehrenberg H, Fries J, Wu S, Ré C (2020) Snorkel: rapid training data creation with weak supervision. VLDB J 29(2):709–730CrossRef
Scudder H (1965) Probability of error of some adaptive pattern-recognition machines. IEEE Trans Inform Theory 11(3):363–371MathSciNetCrossRef
Seker A, Bandel E, Bareket D, Brusilovsky I, Greenfeld R, Tsarfaty R (2022) Alephbert: Language model pre-training and evaluation from sub-word to sentence level. In: proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: Long Papers), pp 46–56
Simmler M, Brunner S, Canova G, Schedler K (2023) Smart criminal justice: exploring the use of algorithms in the swiss criminal justice system. Artif Intell Law 31(2):213–237CrossRef
Su H, Shi W, Shen X, Xiao Z, Ji T, Fang J, Zhou J (2022) Rocbert: Robust chinese bert with multimodal contrastive pretraining. In: proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers), pp 921–931
Tseng H-Y, Lee H-Y, Huang J-B, Yang M-H (2020) Cross-domain few-shot classification via learned feature-wise transformation. arXiv preprint arXiv:​2001.​08735
Vuong YT-H, Bui QM, Nguyen H-T, Nguyen T-T-T, Tran V, Phan X-H, Satoh K, Nguyen L-M. (2022) Sm-bert-cr: a deep learning approach for case law retrieval with supporting model. Artif Intell Law, pp 1–28
Wei C, Sohn K, Mellina C, Yuille A, Yang F (2021) Crest: A class-rebalancing self-training framework for imbalanced semi-supervised learning. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10857–10866
Wieting J, Bansal M, Gimpel K, Livescu K (2015) Towards universal paraphrastic sentence embeddings. arXiv preprint arXiv:​1511.​08198
Wu C-S, Hoi SC, Socher R, Xiong C (2020) Tod-bert: Pre-trained natural language understanding for task-oriented dialogue. In: proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp 917–929
Xia C, Zhang C, Nguyen H, Zhang J, Yu P (2020) Cg-bert: conditional text generation with bert for generalized few-shot intent detection. arXiv preprint arXiv:​2004.​01881
Xiao C, Hu X, Liu Z, Tu C, Sun M (2021) Lawformer: a pre-trained language model for chinese legal long documents. AI Open 2:79–84CrossRef
Xie Q, Luong M-T, Hovy E, Le QV (2020) Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10687–10698
Yang J, Zhang Y, Dong F (2017) Neural word segmentation with rich pretraining. In proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: Long Papers), pp 839–849
Yang L, Zhuo W, Qi L, Shi Y, Gao Y (2022) St++: Make self-training work better for semi-supervised semantic segmentation. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4268–4277
Yao H, Zhang C, Wei Y, Jiang M, Wang S, Huang J, Chawla N, Li Z (2020) Graph few-shot learning via knowledge transfer. Proc AAAI Conf Artif Intell 34:6656–6663
Yu AW, Dohan D, Luong M-T, Zhao R, Chen K, Norouzi M, Le QV (2018) Qanet: combining local convolution with global self-attention for reading comprehension. arXiv preprint arXiv:​1804.​09541
Zhang J, Zhang M, Lu Z, Xiang T (2021) Adargcn: adaptive aggregation gcn for few-shot learning. In: proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 3482–3491
Zhou J, Wang J, Liu G (2019) Multiple character embeddings for chinese word segmentation. In: proceedings of the 57th annual meeting of the association for computational linguistics: student research workshop, pp 210–216
Zhou Y, Liu L, Chen Y, Huang R, Qin Y, Lin C (2023) A novel mrc framework for evidence extracts in judgment documents. Artif Intell Law, pp 1–17
Zoph B, Ghiasi G, Lin T-Y, Cui Y, Liu H, Cubuk ED, Le Q (2020) Rethinking pre-training and self-training. Adv Neural Inform Process Syst 33:3833–3845
Zou Y, Yu Z, Liu X, Kumar B, Wang J (2019) Confidence regularized self-training. In: proceedings of the IEEE/CVF international conference on computer vision, pp 5982–5991
Metadaten
Titel
Self-training improves few-shot learning in legal artificial intelligence tasks
verfasst von
Yulin Zhou
Yongbin Qin
Ruizhang Huang
Yanping Chen
Chuan Lin
Yuan Zhou
Publikationsdatum
17.05.2024
Verlag
Springer Netherlands
Erschienen in
Artificial Intelligence and Law
Print ISSN: 0924-8463
Elektronische ISSN: 1572-8382
DOI
https://doi.org/10.1007/s10506-024-09403-z

Premium Partner

    Bildnachweise