Additive Compositionality of Word Vectors

Yeon Seonwoo1, Sungjoon Park2, Dongkwan Kim3, Alice Oh3
1KAIST, Korea Advanced Institute of Science and Technology, 2Korea Advanced Institute of Science and Technology, 3KAIST


Abstract

Additive compositionality of word embedding models has been studied from empirical and theoretical perspectives. Existing research on justifying additive compositionality of existing word embedding models requires a rather strong assumption of uniform word distribution. In this paper, we relax that assumption and propose more realistic conditions for proving additive compositionality, and we develop a novel word and sub-word embedding model that satisfies additive compositionality under those conditions. We then empirically show our model's improved semantic representation performance on word similarity and noisy sentence similarity.