2023 Unsupervised Abstract Top Conference Papers Collection

write on top

I have confirmed the major research route with my teacher, and I am currently studying papers and codes in related fields of [Unsupervised Abstract], hoping to use them in our thesis research

This blog post sorts out the main context of each article, and initially marks the ideas and sections for in-depth study in the later period.
Problem + motivation + problem-solving method

Preface: I searched all the papers in [Text Abstracts] in the top journals in the past three years, and made a preliminary classification according to the paper name & abstract. Now look at the unsupervised papers and leave a preliminary impression (dblp seems not to be able to search
twice It’s not easy to search for papers by periodicals)
(The teacher took the relevant literature that he spent two afternoons sorting out, and got a new method to find references O(∩_∩)O~)

ACL-2023

Aspect-aware Unsupervised Extractive Opinion Summarization for unsupervised opinion summary (not found)

Unsupervised Extractive Summarization of Emotion Triggers * Unsupervised Extractive Summarization of Emotion Triggers (reasons) (with code)

Paper address and code

https://arxiv.org/pdf/2306.01444v1.pdf
https:/lgithub.com/tsosea2/CovidET-EXT

Task

Understanding what triggers emotions in a large-scale crisis (in the context of the COVID-19 crisis) can:
1. provide a basis for expressing emotions
and 2. improve understanding of the ongoing disaster.

Develop new unsupervised learning models that can jointly detect emotions and summarize their triggers (summarize the reasons for triggering emotions to form a summary)

motivation

A recent approach (Zhan et al., 2022) trains supervised models to detect emotions and explain emotion triggers (events and reviews) through abstract summarization.
However, obtaining timely and qualitative abstract summaries is expensive and extremely time-consuming , requiring highly trained expert annotators. In time-sensitive, high-stakes situations, this can hinder a necessary response.
Instead, we pursue unsupervised systems that extract triggers from text.

Dataset COVIDET-EXT

COVIDET-EXT:
1. Based on COVIDET
by Zhan et al. (2022) 2. Augmented with manually annotated extraction summaries corresponding to each abstract summaries.

Description:
1. A dataset containing 1883 Reddit posts about the COVID-19 pandemic, manually annotated with 7 fine-grained sentiments (from COVIDET) and their corresponding extraction triggers (Fig. 1).
2. For each emotion present in a post, the sentences summarizing the triggers of the emotion are highlighted, resulting in a total of 6,741 extracted summaries.

COVIDET-EXT provides an ideal testbed to facilitate the development of extraction (supervised or unsupervised) techniques for emotion detection and trigger summarization tasks in crisis contexts.
Qualitative analysis of the dataset shows good agreement among the annotators, and subsequent human verification of the annotations also shows high correctness.

step

1. Introduce the COVIDET-EXT dataset
2. Develop a new unsupervised learning model that can jointly detect emotions and summarize their triggers
Our best method, named Pagerank for emotion perception, combines emotional information from external sources with language understanding modules combined

Figure 3: Our emotion-aware PageRank graph. EAP builds a word graph from an article and then runs separate biased pageranks, one for each emotion, to score each candidate sentence under each emotion. This score is combined with an emotion-aware language understanding module to produce a final ranking for each sentence under each emotion.

EAP treats words rather than sentences as nodes in the graph and computes a sentiment-specific score for each word using multiple independent biased PageRanks (Haveli-wala, 2003), which is combined with a sentence similarity module , yielding a sentence score for each emotion, indicating the salience of the sentence under each emotion.

Formula representation

Index performance

Emotion-agnostic baseline.

Two standard heuristic baselines, namely:
1) Extract the first sentence in the post (1 send)
2) Extract the first three sentences in the post (3 sends).

Three methods based on graph centrality measures are designed:
3) PacSum (Zheng and Lapata, 2019),
4) PreSum (Liu and Lapata, 2019) and word-level
5) TextRank (Mihalcea and Tarau, 2004).
Note that these methods are sentiment-agnostic and the generated summaries will be the same for different sentiments.

Emotion-specific baselines.

Two lexicon-based approaches were first employed:
6) EmoLex - used the EmoLex (Muhammad and Turney, 2013) vocabulary to identify lexical cues indicative of emotional expression.
If a sentence contains a word related to emotion e, we consider this sentence to express eoe and the final summary contains all sentences expressing e.
7) EmoIntensity - utilizes the NRC Emotion Intensity Dictionary (Muhammad, 2018) to build a more fine-grained approach to identify whether a sentence expresses emotion or not.
For each sentence and sentiment, we compute the average sentiment word strength and compare it with a predefined threshold t. If the average strength of e is higher than t, we label the sentence as e. t is a tunable parameter we choose based on validation set performance.

Results: Outperforms strong baselines.

(Insert a sister article EMNLP-2022 from the same team as the previous paper) Why Do You Feel This Way? Summarizing Triggers of Emotions in Social Media Posts Why do you feel this way? (Generative) Summarize the emotional triggers in social media posts (reason)

Paper address and data set, code

https://arxiv.org/pdf/2210.12531.pdf
One and two are swapped, three and four are swapped

Dataset COVIDET and code:
https://github.com/honglizhan/CovidET.

motivation

Crises such as the coVID-19 pandemic continue to threaten our world and affect the emotions of billions of people around the world in different ways.
Understanding the triggers that drive people's emotions is critical.
Posts on social media can be a good source for this kind of analysis, but these texts often contain multiple emotions, with triggers scattered across multiple sentences .

This paper adopts a novel perspective, emotion detection and trigger word summarization , which aims to detect perceived emotions in text and summarize the events and their evaluations that trigger each emotion.

data set

The time span of CoVIDET is from June 2021 to January 2022, and it records various major events that occurred during the epidemic and people's emotional evaluation of these events.

This paper presents CovIDET (Emotions during Covid19 and their triggers), a dataset of 1883 English-language Reddit posts related to COVID-19, which contains manually annotated ①perceived emotions, and each article is annotated with 7 fine-
grained Sentiment Labels
② Abstract summaries of (emotional) triggers described in posts. For each emotion, the annotators provided a concise, abstract summary describing the trigger for that emotion.

Compared to previous sentiment studies that only considered sentence-level text (Sosea and Caragea, 2020; Demszky et al., 2020) or (short) tweets (Sosea et al., 2022; Abdul-Mageed and Ungar, 2017), CovIDET has Challenging because it contains significantly longer text.

Figure 1: An example from COVIDET identifying perceived
emotions and summarizing their triggers.

method

A robust baseline is developed to jointly detect emotions and summarize emotional triggers.
Separate emotion detection and trigger summarization models are employed, as well as a joint model designed to simultaneously detect emotion and generate trigger summaries.

COvIDET has various unique properties, from its long sequences and valuable context to the nature of the task itself.
Human evaluation of trigger summaries tailored for sentiment-triggered summaries
shows that the proposed model is effective in capturing latent triggers of posts

A robust baseline is developed to jointly detect emotions and summarize emotional triggers.

Figure 6: Architecture of the joint model for sentiment detection and trigger summarization.

result

The analysis shows that CoVIDET presents new challenges in sentiment-specific summaries and multi-emotion detection in long social media posts.
Therefore, general sentiment detection or summarization models lag significantly behind in performance compared to our approach.

Unsupervised Summarization Re-ranking Unsupervised Summary Re-ranking

the code

https://github.com/ntunlp/SummScore

background

With the rise of task-specific pre-training objectives, abstract summarization models such as PEGASUS provide attractive zero-shot performance in downstream summarization tasks.
However, the performance of such unsupervised models still lags significantly behind supervised models.
Similar to the supervised setting, we note that the quality of the summarization candidates for these models varies greatly, while only one candidate is retained as the summary output.

This paper proposes to rerank abstract candidates in an unsupervised manner, aiming to close the performance gap between unsupervised and supervised models. The method is compared to four widely adopted summary benchmarks in ROUGE

main challenge

The main challenge:
the reorderer must also be free of any supervision. The proposed model does not train any neural model, but simply computes features indicative of summary quality to score each candidate summary, some of which also leverage the source documents.
A weighted average of these features is used for candidate reranking, and several methods for estimating feature weights are explored.

The method is called SummScore:
1. Lightweight, fast and easy to use because it does not rely on neural networks.
2. Since it is completely unsupervised, the reranked results can provide finer self-supervision for the pre-trained model, supplementing the pre-training with several rounds of self-training.

Pros:
1. This paper presents SummScore, the first system for reranking summary candidates in an unsupervised setting and in an unsupervised manner.
2. Demonstrate the power of SummScore through continuous performance improvement: up to 7.27% for PEGASUS and 6.86% for ChatGPT The average return is 7.51% ROUGE.
3. Using re-ranker, an original and effective self-training method is derived, and the basic unsupervised summary model is continuously improved, which improves PEGASUS from 35.47 to 39.76ROUGE-1 (+12.09%).

Disentangling Text Representation With Counter-Template For Unsupervised Opinion Summarization

ACL-2022

Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization (with code)

Point of departure

Recently, Schumann et al. (2020) proposed an edit-based approach for unsupervised summarization. Their model maximizes a heuristically defined scoring function that evaluates the quality (fluency and semantics) of generated summaries, achieving higher performance than cycle-consistency methods.
1. However, search methods are slow during inference, as each data sample requires hundreds of search steps.
2. Furthermore, their method can only select words from the input sentence whose word order remains the same. Therefore, it is limited and may produce noisy summaries due to the local optimality of the search algorithm.

advantage

To address the above shortcomings, this paper proposes a non-autoregressive unsupervised summarization method (NAUS). The idea is to perform the search in Schumann et al. (2020) and, inspired by Li et al. (2020), train a machine learning model to smooth this noise and speed up the inference process. Unlike Li et al. (2020), this paper proposes to utilize a non-autoregressive decoder, which generates all output tokens in parallel due to the following observations: 1.
Non-autoregressive models are several times faster than autoregressive generation, which is very convenient when deploying the system. important.
2. The input and output of the summary task have a strong correspondence. Non-autoregressive generation supports encoder-only architectures, which can better exploit this input-output correspondence and even outperform autoregressive models in terms of summarization.
3. For non-autoregressive models, a length control algorithm based on dynamic programming can be designed to satisfy the length constraint, which is common in abstraction applications but difficult to implement in autoregressive models.

method

This paper presents a non-autoregressive unsupervised summarization (NAUS) method that does not require parallel data for training.
1. NAUS first performs an edit-based search on the heuristically defined scores and generates a summary as the pseudo ground truth.
2. Then, train an encoder-only non-autoregressive Transformer based on the search results.

This paper also proposes a dynamic programming method for length-controlled decoding, which is important for summarization tasks.

Index performance

Experiments on two datasets show that NAUS achieves state-of-the-art performance in unsupervised summarization, but greatly improves inference efficiency. Additionally, the algorithm is able to perform explicit length transfer digest generation.

the code

https://github.com/MANGA-UOFA/NAUS

ACL-2021

Unsupervised Extractive Summarization-Based Representations for Accurate and Explainable Collaborative Filtering

Improving Unsupervised Extractive Summarization with Facet-Aware Modeling

EMNLP-2022

Unsupervised Opinion Summarisation in the Wasserstein Space

Unsupervised Entity Linking with Guided Summarization and Multiple-Choice Selection

Learning From the Source Document: Unsupervised Abstractive Summarization. 4194-4205

Unsupervised Multi-Granularity Summarization. 4980-4995

NeurIPS-2021

WWW-2023

Reference
XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languages
​​https://dl.acm.org/doi/10.1145/3543507

AAAI-2021

Unsupervised Opinion Summarization with Content Planning Unsupervised opinion summary based on content planning (with code)

Code: https://github.com/rktampplayo/plansum

data set

The recent success of deep learning techniques in generative summarization is based on the availability of large-scale datasets.
Such training data is neither available nor readily available when summarizing reviews (e.g., for products or movies), motivating the development of methods that rely on synthetic datasets for supervised training.

(Advantages) Explicitly incorporating content planning
into the summarization model, 1. can produce higher quality output,
and 2. create more natural synthetic datasets, similar to real-world document-summarization pairs.

Take the form of aspect and sentiment distributions that are generalized from data that do not require access to expensive annotations.
Synthetic datasets are created by sampling pseudo-reviews from a Dirichlet LDA topic distribution parameterized by a content planner.

summary generation

Whereas the model generates summaries based on input reviews and induced content plans.

Supplementary knowledge points: hinge loss

Reference: https://zhuanlan.zhihu.com/p/347456667

The derivation of Logistic Loss in LR is more statistical, so the derivation of Hinge Loss in SVM is more like doing mathematical modeling.

In space, it is in the middle of the positive and negative samples, and may be more robust to local perturbations of positive and negative samples.
So how to mathematicalize this idea?
Margin: The point closest to the decision boundary, the distance to the decision boundary
If a hyperplane that can maximize Margin can be found under constraints, then the above idea can be realized so that it is located in the middle of the positive and negative samples.

train

Evaluate

Experiments on three datasets (Wang and Ling 2016; Chuand Liu 2019; Bra2inskas, Lapata, and Titov 2019) representing different domains (movie, business, and product reviews) summarization requirements (short summaries vs long summaries) show that the The proposed method outperforms competing models in generating informative, coherent, and fluent summaries to capture opinion consensus.

2

In this paper, we propose ES-COFILT, the first collaborative filtering model based on extractive summarization. The proposed model specifically produces extractive summaries for each item and user.
Unlike other types of explanations, summary-level explanations closely resemble real-life explanations.
Advantages: Unified representation and interpretation. Extractive summaries represent both items and users.
Model: Integrates BERT, K-means embedding custthng and multi-layer perceptron to learn sentence embedding, representation interpretation and user-item interaction respectively.
Improvement: Accuracy of rating prediction and scalability of users/1tem. Experimental results show that the prediction accuracy of this model is better than other mainstream recommendation models.
Furthermore, we present a comprehensive collection. Criteria for evaluating the interpretability of explanations in real-life situations. Interpretability studies show that summary-level explanations have advantages and preferences over other explanation types.

Unsupervised Abstractive Dialogue Summarization for Tete-a-Tetes Unsupervised Abstract Dialogue Summarization for Face-to-face Dialogues

Unsupervised Summarization for Chat Logs with Topic-Oriented Ranking and Context-Aware Auto-Encoders

Unsupervised Abstractive Dialogue Summarization for Tete-a-Tetes Unsupervised Abstract Dialogue Summarization for Face-to-face Dialogues

other

Judging Extractive and Generative Summarization

Extraction: Extractive Summarization
Generative: Abstractive Summary

main method

Unsupervised extractive summarization.

Extractive summarization aims to condense a piece of text by identifying and extracting a small number of important sentences (Allahyari et al., 2017; Liu and Lapata, 2019; El-Kassas et al., 2021), preserving the original meaning of the text.
The most popular method in unsupervised extraction of summaries:
using graph-based methods to calculate the saliency of sentences contained in the summaries (Mi-halcea and Tarau, 2004; Zheng and Rapata, 2019). These methods represent sentences in documents as nodes in an undirected graph whose edges are weighted using sentence similarity. Score and rank veracity of sentences in the graph using node cen- verity, computed recursively using PageRank (Page et al., 1999).

Evaluate

Assessing informativeness: Unary and binary overlap (ROUGE-1 and Rouge-2)
Measuring fluency: Longest common subsequence (ROUGE-L)