Paper link: https://arxiv.org/pdf/2011.05367.pdf
Summary
Social media is deliberately used for malicious purposes, including political manipulation and disinformation. Most research has focused on high-resource languages. However, malicious actors share content across countries and languages, including low-resource languages.
Here we investigate whether and to what extent malicious actors can be detected in low-resource language settings . We found that after the 2016 US presidential election, as part of Twitter's crackdown on interference, a large number of accounts that posted in Tagalog were suspended.
By combining text embeddings and transfer learning, our framework can accurately detect malicious users posting in Tagalog without prior knowledge of or training on malicious content in the language .
We first learn embedding models for each language independently, i.e. high-resource language (English) and low-resource language (Tagalog).
Then, we learn a mapping between the two latent spaces to transfer the detection model .
We demonstrate that the proposed method significantly outperforms state-of-the-art models, including BERT, and yields significant advantages in settings with very limited training data—a norm when dealing with detecting malicious activity in online platforms.
introduction
Disinformation and political manipulation have a long history: For example, in 1984, long before the age of social media, a story that claimed HIV was created by the US government as a biological weapon went viral around the world. Today, social media amplifies and accelerates the flow of information like never before. Online social networks (OSNs) such as Twitter and Facebook have been facing a massive increase in malicious content, which undermines the authenticity and authenticity of online discourse [1, 21, 24, 32, 74, 77].
Various studies have shown that OSNs have been used for malicious purposes, compromising multiple components of our society [42, 75], ranging from geopolitical events [22, 27, 44, 58, 63] to public health [14, 25 , 28, 52, 78]. Bots and trolls are major players in social media manipulation and disinformation campaigns [3, 11, 26, 45, 62], often in a coordinated fashion [29, 51, 53, 64, 67, 76].
Particular attention has been paid to the risk of large-scale manipulation of public opinion in political contexts, best exemplified by cyber interference in the 2016 US Presidential Discussion election [4, 6]. Since then, OSN has worked hard to combat abuse and maintain trust and a healthy dialogue on its platform. Despite efforts, troll and bot activity appears to persist [36,43,72]. For example, Twitter identified and suspended malicious accounts from various countries, including Russia, Iran, Bangladesh, and Venezuela [71], suggesting a concerted effort to manipulate online speech across countries and languages. More recently, Pieri et al. [58] documented evidence of platform abuse and subsequent Twitter meddling in the context of the ongoing conflict between Ukraine and Russia [57]. While others have explored various strategies for malicious users in high-resource languages [41, 43, 69, 70] to achieve their detection [12, 13, 33, 49], here we propose a method using transfer A new approach learned to detect malicious users automatically identifies misbehaving accounts in low-resource languages.
Contribution of this work
Our aim is to investigate whether and to what extent textual content can be used as a proxy to detect malicious activity on social media, paying particular attention to accounts sharing messages in low-resource languages. Overall, we aim to answer two main research questions:
RQ1: Can we classify accounts as malicious based solely on what they share? We explore the effectiveness of learning word representations from tweets to identify suspended accounts.
RQ2: Can we learn a model from a high-resource language (English) and transfer knowledge to a low-resource language (Tagalog) to detect suspended accounts? We investigate whether learning a mapping between two independently trained word embeddings is beneficial for identifying misbehaving accounts.
In the Numbers: US 2016 Presidential Election
In this study, we use Twitter as a test platform to detect the activity of malicious accounts focused on the 2016 US presidential election. The dataset consists of approximately 42 million tweets by nearly 6 million distinct users and was first published by [6]. Tweets collected via the Twitter Streaming API using 23 election keywords (5 for Donald Trump, 4 for Hillary Clinton, 3 for third-party candidates, 11 for general election terms). The collection was conducted between September 16, 2016 and October 21, 2016. From the collected set of tweets, duplicate tweets were removed, which might have been caught through unexpected redundant queries to the Twitter API. Table 1 lists the most popular keywords and the number of related tweets. While all keywords are in English, tweets in other languages were also collected.
We identified more than 60 different languages, with the highest number of tweets written in European languages. Among them, nearly 5 million users posted more than 37.6 million tweets in English. We found a large number of tweets in Tagalog, an Austronesian language that is the first language of a quarter of the Filipino population and the second language of more than half of the Filipino population. Tagalog is the fourth most commonly used language in the US by number of speakers[73], behind English, Spanish, and Chinese, and the #1 low resource in our data by number of tweets language. The United States also has one of the largest Filipino immigrant populations outside the Philippines. Furthermore, the underresourced status of Tagalog was further confirmed by an analysis of the size of its Wikipedia - a common metric for estimating the amount of digital resources in a language. Wikipedia in Tagalog is currently ranked 101 by number of articles, in stark contrast to its popularity in our dataset. We therefore focused our attention on Tagalog as the target language for this work.
4 Methodology
4.1 Word Representation
To learn word embeddings and train classification models, we use the FastText framework. Instead of treating words as atomic units of text, FastText represents words as a bag of character n-grams [8], where each n-gram has its own vector representation, and words are represented as their constituent character n-grams Sum. This allows the model to adapt to morphologically rich and large-vocabulary languages, and to generalize better from smaller training corpora.
Although neural network based models have achieved considerable success in text classification tasks, they are still quite expensive to train and deploy. FastText utilizes hierarchical softmax to serve as a fast approximation to a softmax classifier for computing the probability distribution of a given class [38]. Using feature pruning, quantization, hashing, and retraining to significantly reduce model size without sacrificing accuracy or speed, this approach allows training models on large text corpora much faster than neural network-based approaches [37].
4.2 Transfer Learning
Traditional machine learning methods for natural language processing focus on training specialized models for specific tasks. However, this requires a lot of data, which is difficult to obtain for low-resource languages. This has historically fueled the cycle by triggering more research on high-resource languages (mainly European languages), which in turn created more resources for those languages. Transfer learning has recently emerged as a method of taking knowledge learned from a source language (or source task) and using it to improve performance in a target language (or target task).
To address the scarcity of target language data analyzed in this work, we use MUSE, a framework for aligning monolingual word embeddings from different languages in the same space and allowing knowledge transfer between them. MUSE uses Procrustes alignment to learn a mapping from source space to target space to minimize the distance between similar words in two languages [40]. It takes as input two sets of pretrained monolingual word embeddings (such as those learned by FastText), one set for each language, and can learn the mapping between them in a supervised or unsupervised manner. Supervised methods require the use of bilingual dictionaries to help align two embeddings together by identifying pairs of similar words that should be close together in a shared space. In the absence of such a dictionary, unsupervised alternatives leverage inverse training to initialize a linear mapping between source and target spaces and generate a synthetic parallel dictionary. [15] show that this method can be used to perform unsupervised word translation without using any parallel data, and the results even outperform previous supervised methods in some cases.
4.3 Learning tasks
Monolingual text classification. In the first approach, we train separate text classification models from scratch using each language's own dataset. For classification purposes, we use the FastText framework, which represents text as Bag-of-Words (BoW) and averages their individual representations into a combined text representation. This text representation is then used as input to a linear classifier with a softmax function that computes the probability distribution of the label classes for prediction.