Leveraging Dynamic Graph Word Embedding for Efficient Contextual Representations

Sentence structure consists of a complex structure of words and relationships between words, which can be hard to represent with only a sequential representation and makes it challenging to learn long-range dependencies. This research presents a novel dynamic word embedding method designed to improve text classification performance. The method leverages a next word prediction model trained on a massive text corpus to extract dynamic text representations. These representations capture the evolving meaning of words based on context and are then combined with static embeddings like Word2Vec. A noteworthy approach is that the method incorporates an undirected graph model to capture contextual relationships between words. Three variations of the method are explored: ELMo-Like Baseline Dynamic, ARMA Graph Dynamic, and ARMA+ELMo Dynamic. Experiments utilizing deep learning models for sentiment analysis and disaster tweet classification demonstrate the effectiveness of the proposed approach.