S2‐Net: Machine reading comprehension with SRU‐based self‐matching networks

Abstract

Machine reading comprehension is the task of understanding a given context and finding the correct response in that context. A simple recurrent unit (SRU) is a model that solves the vanishing gradient problem in a recurrent neural network (RNN) using a neural gate, such as a gated recurrent unit (GRU) and long short‐term memory (LSTM); moreover, it removes the previous hidden state from the input gate to improve the speed compared to GRU and LSTM. A self‐matching network, used in R‐Net, can have a similar effect to coreference resolution because the self‐matching network can obtain context information of a similar meaning by calculating the attention weight for its own RNN sequence. In this paper, we construct a dataset for Korean machine reading comprehension and propose an S2‐Net model that adds a self‐matching layer to an encoder RNN using multilayer SRU. The experimental results show that the proposed S2‐Net model has performance of single 68.82% EM and 81.25% F1, and ensemble 70.81% EM, 82.48% F1 in the Korean machine reading comprehension test dataset, and has single 71.30% EM and 80.37% F1 and ensemble 73.29% EM and 81.54% F1 performance in the SQuAD dev dataset.

Keywords

• machine reading comprehension
• question answering
• simple recurrent unit
• self‐matching network
• Korean machine reading comprehension
• S