Graph routing between capsules

Yang Li, Wei Zhao, Erik Cambria, Suhang Wang, Steffen Eger

Research output: Contribution to journalArticlepeer-review

8 Scopus citations

Abstract

Routing methods in capsule networks often learn a hierarchical relationship for capsules in successive layers, but the intra-relation between capsules in the same layer is less studied, while this intra-relation is a key factor for the semantic understanding in text data. Therefore, in this paper, we introduce a new capsule network with graph routing to learn both relationships, where capsules in each layer are treated as the nodes of a graph. We investigate strategies to yield adjacency and degree matrix with three different distances from a layer of capsules, and propose the graph routing mechanism between those capsules. We validate our approach on five text classification datasets, and our findings suggest that the approach combining bottom-up routing and top-down attention performs the best. Such an approach demonstrates generalization capability across datasets. Compared to the state-of-the-art routing methods, the improvements in accuracy in the five datasets we used were 0.82, 0.39, 0.07, 1.01, and 0.02, respectively.

Original languageEnglish (US)
Pages (from-to)345-354
Number of pages10
JournalNeural Networks
Volume143
DOIs
StatePublished - Nov 2021

All Science Journal Classification (ASJC) codes

  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Graph routing between capsules'. Together they form a unique fingerprint.

Cite this