<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="singh-etal-2023-nlms">
    <titleInfo>
        <title>NLMs: Augmenting Negation in Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Rituraj</namePart>
        <namePart type="family">Singh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rahul</namePart>
        <namePart type="family">Kumar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vivek</namePart>
        <namePart type="family">Sridhar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Negation is the fundamental component in a natural language that reverses the semantic meaning of a sentence. It plays an extremely important role across a wide range of applications, yet they are underrepresented in pre-trained language models (LMs), resulting often in wrong inferences. In this work, we try to improve the underlying understanding of the negation in the pre-trained LMs. To augment negation understanding, we propose a language model objective with a weighted cross-entropy loss and elastic weight consolidation regularization. We reduce the mean top 1 error rate for BERT-base to 1.1%, BERT-large to 0.78%, RoBERTA-base to 3.74%, RoBERTA-large to 0.01% on the negated LAMA dataset. It minimizes the BERT error rate by a margin of 8% and also outperform the existing negation models. We also provide empirical evidences that negated augmented models outperform the classical models on original as well as negation benchmarks on natural language inference tasks.</abstract>
    <identifier type="citekey">singh-etal-2023-nlms</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-emnlp.873</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-emnlp.873</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>13104</start>
            <end>13116</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T NLMs: Augmenting Negation in Language Models
%A Singh, Rituraj
%A Kumar, Rahul
%A Sridhar, Vivek
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Findings of the Association for Computational Linguistics: EMNLP 2023
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F singh-etal-2023-nlms
%X Negation is the fundamental component in a natural language that reverses the semantic meaning of a sentence. It plays an extremely important role across a wide range of applications, yet they are underrepresented in pre-trained language models (LMs), resulting often in wrong inferences. In this work, we try to improve the underlying understanding of the negation in the pre-trained LMs. To augment negation understanding, we propose a language model objective with a weighted cross-entropy loss and elastic weight consolidation regularization. We reduce the mean top 1 error rate for BERT-base to 1.1%, BERT-large to 0.78%, RoBERTA-base to 3.74%, RoBERTA-large to 0.01% on the negated LAMA dataset. It minimizes the BERT error rate by a margin of 8% and also outperform the existing negation models. We also provide empirical evidences that negated augmented models outperform the classical models on original as well as negation benchmarks on natural language inference tasks.
%R 10.18653/v1/2023.findings-emnlp.873
%U https://aclanthology.org/2023.findings-emnlp.873
%U https://doi.org/10.18653/v1/2023.findings-emnlp.873
%P 13104-13116

Markdown (Informal)

[NLMs: Augmenting Negation in Language Models](https://aclanthology.org/2023.findings-emnlp.873) (Singh et al., Findings 2023)

• NLMs: Augmenting Negation in Language Models (Singh et al., Findings 2023)

ACL

• Rituraj Singh, Rahul Kumar, and Vivek Sridhar. 2023. NLMs: Augmenting Negation in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13104–13116, Singapore. Association for Computational Linguistics.