Mixture-of-Experts Model for Hypernymy Discrimination

Abstract

Hypernymy discrimination is an essential and challenging task in NLP.Traditional supervised methods usually model all the hypernymies in the global semantic space,which has achieved fair performance.However,the distributed semantic representation of hypernymies is rather complex,and their manifestations may differ significantly in different areas of the semantic space,making it difficult to learn the global model.This paper employs the mixture-of-experts framework as a solution.It works on the basis of a divide-and-conquer strategy,which divides the semantic space into multiple subspaces,and each subspace corres-ponds to a local expert(model).A number of localized experts(models) focus on their own domains(or subspaces) to learn their specialties,and a gating mechanism determines the space partitioning and the expert aggregation.Experimental results show that the mixture-of-experts model outperforms the traditional global ones on public datasets.

Keywords

• hypernymy discrimination|mixture-of-experts|local model