Generating and Measuring Similar Sentences Using Long Short-Term Memory and Generative Adversarial Networks

Abstract

The two problems of measuring the semantic similarity (MSS) between two sentences and generating a similar sentence (GSS) for a given one are particularly challenging. Since these two problems naturally have logical connections, this article proposes algorithms to deal with them together. The main contributions of this article are in four aspects. 1) We propose a new algorithm called the syntactic and semantic long short-term memory (SSLSTM) for computing sentence similarity. The sentence model used by SSLSTM computes a representation vector of a given sentence by merging the results of separately running two LSTM networks, one with the given sentence and the other with a related sentence that is generated based on the semantic features of the words and syntactic features of the given sentence. The semantic similarity score of two sentences is calculated based on the distance between the two representations vectors. 2) A new GAN framework is proposed called the sentence similarity generative adversarial network (SSGAN). A GSS algorithm and an MSS algorithm are incorporated as modules in the generator and discriminator of SSGAN. A unique design of SSGAN is that, with one input triple to the GAN, the generator will produce three additional items, and the discriminator will process them. Three versions of SSGAN are proposed: the classic SSGAN (C-SSGAN), the hybrid SSGAN (H-SSGAN), and the black-box SSGAN (B-SSGAN). 3) Two new paradigms of GAN emerge from the design patterns of the black-box SSGAN and hybrid SSGAN, called the black-box GAN (B-GAN) and the hybrid GAN (H-GAN), respectively, which have the potentials to be generally applied to other NLP problems. 4) A series of experiments of different settings are designed to test the effects of B-SSGAN, and the results show that B-SSGAN has considerable boosting effects on both the chosen MSS and GSS algorithms. Several experiments are executed to compare SSLSTM with some representative and state-of-the-art MSS algorithms. The results show that SSLSTM has advantages in terms of the amount of error and the overall performance. There are new design features in these experiments. The performances of a GSS algorithm are measured by using an MSS algorithm. Multiple performance measures are considered to describe the algorithms’ performance holistically, including the efficiency of achieved performance relative to training time, which indicates that a CNN-based algorithm (SSCNN) is the most training-efficient in the comparison.

Keywords

• Long short-term memory
• generative adversarial networks
• semantics
• syntax
• deep learning
• neural network