Deep belief networks for sentiment analysis

Abstract

Sentiment analysis is a highly popular issue both in academic and engineering fields. Nowadays there is an increasingly large amount of online opinion resources, so people are inclined to develop some systems that can automatically determine the polarities of opinions, especially in the decision-making process of a company. On the other hand, deep learning is a recently developed popular topic and has received much attention in machine learning area. Deep belief network (DBN) is one important deep learning model, which has proved powerful in many domains including natural language processing. However, there still exist some big challenges for DBNs in sentiment analysis because of the complexity to express opinions. Therefore, this study tries to improve DBNs in sentiment analysis area from the following three aspects: (1) The neuron models are investigated in DBNs for sentiment prediction. We perform various experiments and apply both total accuracy and F-measure to evaluate the performance, which proves that Gaussian neuron model with specific parameter setting has better efiect. (2) In addition to the traditional bag-of-words representation for each sentence, the word positional information is considered in the input. We propose a new word positional contribution form and another novel word-to-segment matrix representation for text to incorporate the positional information into DBNs for sentiment analysis. Finally, we evaluate the performance via the total accuracy. The results show that the word positional information of sentences helps to improve the performance of DBNs for sentiment classification. (3) We propose a new method to improve the DBN learning algorithm based on the unsupervised training phase of restricted Boltzmann machines (RBMs). That is, the RBM generates the hidden layer in an unsupervised fashion, and then we use this hidden layer as the output of a single-layer neural network, which is trained via the delta rule (DR). The new weights trained from DR are then transmitted into the whole network for initialization of back propagation (BP). This way keeps more correction signal for each layer in the BP algorithm compared to the ordinary DBN training. Our experimental results demonstrate that this updated learning method performs better than the ordinary learning for sentiment classification.