今日 cs.CL方向共计14篇文章。
推理分析(1篇)
[1]：An Efficient Model Inference Algorithm for Learning-based Testing of Reactive Systems
标题：一种有效的基于学习的反应系统测试模型推理算法
作者：Muddassar A. Sindhu
备注：29 pages, 2 figures
链接：https://arxiv.org/abs/2008.06268
摘要：Learning-based testing (LBT) is an emerging methodology to automate iterative black-box requirements testing of software systems. The methodology involves combining model inference with model checking techniques. However, a variety of optimisations on model inference are necessary in order to achieve scalable testing for large systems. In this paper we describe the IKL learning algorithm which is an active incremental learning algorithm for deterministic Kripke structures. We formally prove the correctness of IKL. We discuss the optimisations it incorporates to achieve scalability of testing. We also evaluate a black box heuristic for test termination based on convergence of IKL learning.模型(1篇)
[1]：Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems
标题：面向任务对话系统的少镜头学习语言模型
作者：Andrea Madotto
备注：Blog and Code Available
链接：https://arxiv.org/abs/2008.06239
摘要：Task-Oriented dialogue systems use four connected modules such as Natural Language Understanding (NLU), Dialogue State Tracker (DST), Dialogue Policy (DP) and Natural Language Generator (NLG). A research challenge is to learn each module with the least amount of samples (i.e., few-shots) given the high cost related to the data collection. The most common and effective technique to solve this problem is transferring learning, where large language models, either pre-trained on text or task-specific data, are fine-tuned on the few samples. These methods require fine-tuning steps and a set of parameters for each task. Differently, language models such as GPT-2 (Radford et al., 2019) and GPT-3 Brown et al., 2020) allows few-shot learning by priming the model with few-examples. In this paper, we evaluate the few-shot ability of Language Models such as GPT-2 by priming in the NLU, DST, DP and NLG tasks. Importantly, we highlight the current limitations of this approach and we discuss the possible implication to future work.其他(12篇)
[1]：Predicting Event Time by Classifying Sub-Level Temporal Relations Induced from a Unified Representation of Time Anchors
标题：基于时间锚统一表示的子层时间关系分类预测事件时间
作者：Fei Cheng, Yusuke Miyao
备注：9 pages, 4 figures
链接：https://arxiv.org/abs/2008.06452
摘要：Extracting event time from news articles is a challenging but attractive task. In contrast to the most existing pair-wised temporal link annotation, Reimers et al.(2016) proposed to annotate the time anchor (a.k.a. the exact time) of each event. Their work represents time anchors with discrete representations of Single-Day/Multi-Day and Certain/Uncertain. This increases the complexity of modeling the temporal relations between two time anchors, which cannot be categorized into the relations of Allen's interval algebra (Allen, 1990).
In this paper, we propose an effective method to decompose such complex temporal relations into sub-level relations by introducing a unified quadruple representation for both Single-Day/Multi-Day and Certain/Uncertain time anchors. The temporal relation classifiers are trained in a multi-label classification manner. The system structure of our approach is much simpler than the existing decision tree model (Reimers et al., 2018), which is composed by a dozen of node classifiers. Another contribution of this work is to construct a larger event time corpus (256 news documents) with a reasonable Inter-Annotator Agreement (IAA), for the purpose of overcoming the data shortage of the existing event time corpus (36 news documents). The empirical results show our approach outperforms the state-of-the-art decision tree model and the increase of data size obtained a significant improvement of performance.
[2]：ANDES at SemEval-2020 Task 12: A jointly-trained BERT multilingual model for offensive language detection
标题：安第斯在SemEval-2020任务12：联合训练的BERT多语言攻击性语言检测模型
作者：Juan Manuel Pérez, Aymé Arango, Franco Luque
备注：Github repo:this https URL
链接：https://arxiv.org/abs/2008.06408
摘要：This paper describes our participation in SemEval-2020 Task 12: Multilingual Offensive Language Detection. We jointly-trained a single model by fine-tuning Multilingual BERT to tackle the task across all the proposed languages: English, Danish, Turkish, Greek and Arabic. Our single model had competitive results, with a performance close to top-performing systems in spite of sharing the same parameters across all languages. Zero-shot and few-shot experiments were also conducted to analyze the transference performance among these languages. We make our code public for further research
[3]：Graph-based Modeling of Online Communities for Fake News Detection
标题：基于图的网络社区虚假新闻检测建模
作者：Shantanu Chandra, Pushkar Mishra, Helen Yannakoudakis, Ekaterina Shutova
链接：https://arxiv.org/abs/2008.06274
摘要：Over the past few years, there has been substantial effort towards automated detection of fake news. Existing research has modeled the structure, style and content of news articles, as well as the demographic traits of users. However, no attention has been directed towards modeling the properties of online communities that interact with fake news. In this work, we propose a novel approach via graph-based modeling of online communities. Our method aggregates information with respect to: 1) the nature of the content disseminated, 2) content-sharing behavior of users, and 3) the social network of those users. We empirically demonstrate that this yields significant improvements over existing text and user-based techniques for fake news detection.
[4]：Unsupervised vs. transfer learning for multimodal one-shot matching of speech and images
标题：无监督学习与转移学习在多模式一次语音图像匹配中的应用
作者：Leanne Nortje, Herman Kamper
备注：Accepted at Interspeech 2020
链接：https://arxiv.org/abs/2008.06258
摘要：We consider the task of multimodal one-shot speech-image matching. An agent is shown a picture along with a spoken word describing the object in the picture, e.g. cookie, broccoli and ice-cream. After observing one paired speech-image example per class, it is shown a new set of unseen pictures, and asked to pick the "ice-cream". Previous work attempted to tackle this problem using transfer learning: supervised models are trained on labelled background data not containing any of the one-shot classes. Here we compare transfer learning to unsupervised models trained on unlabelled in-domain data. On a dataset of paired isolated spoken and visual digits, we specifically compare unsupervised autoencoder-like models to supervised classifier and Siamese neural networks. In both unimodal and multimodal few-shot matching experiments, we find that transfer learning outperforms unsupervised training. We also present experiments towards combining the two methodologies, but find that transfer learning still performs best (despite idealised experiments showing the benefits of unsupervised learning).
[5]：Speech To Semantics: Improve ASR and NLU Jointly via All-Neural Interfaces
标题：通过NLU联合改进ASR语言的所有语义
作者：Milind Rao, Anirudh Raju, Pranav Dheram, Bach Bui, Ariya Rastrow
备注：Proceedings of INTERSPEECH
链接：https://arxiv.org/abs/2008.06173
摘要：We consider the problem of spoken language understanding (SLU) of extracting natural language intents and associated slot arguments or named entities from speech that is primarily directed at voice assistants. Such a system subsumes both automatic speech recognition (ASR) as well as natural language understanding (NLU). An end-to-end joint SLU model can be built to a required specification opening up the opportunity to deploy on hardware constrained scenarios like devices enabling voice assistants to work offline, in a privacy preserving manner, whilst also reducing server costs.
We first present models that extract utterance intent directly from speech without intermediate text output. We then present a compositional model, which generates the transcript using the Listen Attend Spell ASR system and then extracts interpretation using a neural NLU model. Finally, we contrast these methods to a jointly trained end-to-end joint SLU model, consisting of ASR and NLU subsystems which are connected by a neural network based interface instead of text, that produces transcripts as well as NLU interpretation. We show that the jointly trained model shows improvements to ASR incorporating semantic information from NLU and also improves NLU by exposing it to ASR confusion encoded in the hidden layer.
[6]：Studying Dishonest Intentions in Brazilian Portuguese Texts
标题：巴西葡萄牙语文本中的不诚实意图研究
作者：Francielle Alves Vargas, Thiago Alexandre Salgueiro Pardo
链接：https://arxiv.org/abs/2008.06079
摘要：Previous work in the social sciences, psychology and linguistics has show that liars have some control over the content of their stories, however their underlying state of mind may "leak out" through the way that they tell them. To the best of our knowledge, no previous systematic effort exists in order to describe and model deception language for Brazilian Portuguese. To fill this important gap, we carry out an initial empirical linguistic study on false statements in Brazilian news. We methodically analyze linguistic features using thethis http URLcorpus, which includes both fake and true news. The results show that they present substantial lexical, syntactic and semantic variations, as well as punctuation and emotion distinctions.
[7]：Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model
标题：基于BERT模型的社交媒体仇恨言语检测与种族偏见缓解
作者：Marzieh Mozafari, Reza Farahbakhsh, Noel Crespi
备注：This paper has been accepted in the PLOS ONE journal in August 2020
链接：https://arxiv.org/abs/2008.06460
摘要：Disparate biases associated with datasets and trained classifiers in hateful and abusive content identification tasks have raised many concerns recently. Although the problem of biased datasets on abusive language detection has been addressed more frequently, biases arising from trained classifiers have not yet been a matter of concern. Here, we first introduce a transfer learning approach for hate speech detection based on an existing pre-trained language model BERT and evaluate the proposed model on two publicly available datasets that have been annotated for racism, sexism, hate or offensive content on Twitter. Next, we introduce a bias alleviation mechanism to mitigate the effect of bias in training set during the fine-tuning of our pre-trained BERT-based model for hate speech detection. Toward that end, we use a regularization method to reweight input samples, thereby decreasing the effects of high correlated training set' s n-grams with class labels, and then fine-tune our pre-trained BERT-based model with the new re-weighted samples. To evaluate our bias alleviation mechanism, we employed a cross-domain approach in which we use the trained classifiers on the aforementioned datasets to predict the labels of two new datasets from Twitter, AAE-aligned and White-aligned groups, which indicate tweets written in African-American English (AAE) and Standard American English (SAE), respectively. The results show the existence of systematic racial bias in trained classifiers, as they tend to assign tweets written in AAE from AAE-aligned group to negative classes such as racism, sexism, hate, and offensive more often than tweets written in SAE from White-aligned. However, the racial bias in our classifiers reduces significantly after our bias alleviation mechanism is incorporated. This work could institute the first step towards debiasing hate speech and abusive language detection systems.
[8]：Partial Orders, Residuation, and First-Order Linear Logic
标题：偏序、残差与一阶线性逻辑
作者：Richard Moot
备注：33 pages
链接：https://arxiv.org/abs/2008.06351
摘要：We will investigate proof-theoretic and linguistic aspects of first-order linear logic. We will show that adding partial order constraints in such a way that each sequent defines a unique linear order on the antecedent formulas of a sequent allows us to define many useful logical operators. In addition, the partial order constraints improve the efficiency of proof search.
[9]：Annotating for Hate Speech: The MaNeCo Corpus and Some Input from Critical Discourse Analysis
标题：仇恨言语的诠释：曼内克语料库与批评性话语分析的输入
作者：Stavros Assimakopoulos, Rebecca Vella Muskat, Lonneke van der Plas, Albert Gatt
备注：10 pages, 1 table. Appears in Proceedings of the 12th edition of the Language Resources and Evaluation Conference (LREC'20)
链接：https://arxiv.org/abs/2008.06222
摘要：This paper presents a novel scheme for the annotation of hate speech in corpora of Web 2.0 commentary. The proposed scheme is motivated by the critical analysis of posts made in reaction to news reports on the Mediterranean migration crisis and LGBTIQ+ matters in Malta, which was conducted under the auspices of the EU-funded C.O.N.T.A.C.T. project. Based on the realization that hate speech is not a clear-cut category to begin with, appears to belong to a continuum of discriminatory discourse and is often realized through the use of indirect linguistic means, it is argued that annotation schemes for its detection should refrain from directly including the label 'hate speech,' as different annotators might have different thresholds as to what constitutes hate speech and what not. In view of this, we suggest a multi-layer annotation scheme, which is pilot-tested against a binary +/- hate speech classification and appears to yield higher inter-annotator agreement. Motivating the postulation of our scheme, we then present the MaNeCo corpus on which it will eventually be used; a substantial corpus of on-line newspaper comments spanning 10 years.
[10]：Adaptable Multi-Domain Language Model for Transformer ASR
标题：变压器ASR的自适应多域语言模型
作者：Taewoo Lee, Min-Joong Lee, Tae Gyoon Kang, Seokyeoung Jung, Minseok Kwon, Yeona Hong, Jungin Lee, Kyoung-Gu Woo, Ho-Gyeong Kim, Jiseung Jeong, Jihyun Lee, Hosik Lee, Young Sang Choi
备注：5 pages
链接：https://arxiv.org/abs/2008.06208
摘要：We propose an adapter based multi-domain Transformer based language model (LM) for Transformer ASR. The model consists of a big size common LM and small size adapters. The model can perform multi-domain adaptation with only the small size adapters and its related layers. The proposed model can reuse the full fine-tuned LM which is fine-tuned using all layers of an original model. The proposed LM can be expanded to new domains by adding about 2% of parameters for a first domain and 13% parameters for after second domain. The proposed model is also effective in reducing the model maintenance cost because it is possible to omit the costly and time-consuming common LM pre-training process. Using proposed adapter based approach, we observed that a general LM with adapter can outperform a dedicated music domain LM in terms of word error rate (WER).
[11]：A Hybrid BERT and LightGBM based Model for Predicting Emotion GIF Categories on Twitter
标题：基于BERT和LightGBM的Twitter情感GIF类别预测模型
作者：Ye Bi, Shuo Wang, Zhongrui Fan
备注：4 pages, ACL 2020 EmotionGIF Challenge Technical Report
链接：https://arxiv.org/abs/2008.06176
摘要：The animated Graphical Interchange Format (GIF) images have been widely used on social media as an intuitive way of expression emotion. Given their expressiveness, GIFs offer a more nuanced and precise way to convey emotions. In this paper, we present our solution for the EmotionGIF 2020 challenge, the shared task of SocialNLP 2020. To recommend GIF categories for unlabeled tweets, we regarded this problem as a kind of matching tasks and proposed a learning to rank framework based on Bidirectional Encoder Representations from Transformer (BERT) and LightGBM. Our team won the 4th place with a Mean Average Precision @ 6 (MAP@6) score of 0.5394 on the round 1 leaderboard.
[12]：End-to-End Trainable Self-Attentive Shallow Network for Text-Independent Speaker Verification
标题：面向文本无关说话人验证的端到端可训练自聚焦浅网络
作者：Hyeonmook Park, Jungbae Park, Sang Wan Lee
备注：5 pages, 3 figures, 3 tables
链接：https://arxiv.org/abs/2008.06146
摘要：Generalized end-to-end (GE2E) model is widely used in speaker verification (SV) fields due to its expandability and generality regardless of specific languages. However, the long-short term memory (LSTM) based on GE2E has two limitations: First, the embedding of GE2E suffers from vanishing gradient, which leads to performance degradation for very long input sequences. Secondly, utterances are not represented as a properly fixed dimensional vector. In this paper, to overcome issues mentioned above, we propose a novel framework for SV, end-to-end trainable self-attentive shallow network (SASN), incorporating a time-delay neural network (TDNN) and a self-attentive pooling mechanism based on the self-attentive x-vector system during an utterance embedding phase. We demonstrate that the proposed model is highly efficient, and provides more accurate speaker verification than GE2E. For VCTK dataset, with just less than half the size of GE2E, the proposed model showed significant performance improvement over GE2E of about 63%, 67%, and 85% in EER (Equal error rate), DCF (Detection cost function), and AUC (Area under the curve), respectively. Notably, when the input length becomes longer, the DCF score improvement of the proposed model is about 17 times greater than that of GE2E.
