Skip to main content

Posts

2018


LSTM and GRU

·1 min
LSTM #The avoid the problem of vanishing gradient and exploding gradient in vanilla RNN, LSTM was published, which can remember information for longer periods of time. Here is the structure of LSTM: The calculate procedure are: \[\begin{aligned} f_t&=\sigma(W_f\cdot[h_{t-1},x_t]+b_f)\\ i_t&=\sigma(W_i\cdot[h_{t-1},x_t]+b_i)\\ o_t&=\sigma(W_o\cdot[h_{t-1},x_t]+b_o)\\ \tilde{C_t}&=tanh(W_C\cdot[h_{t-1},x_t]+b_C)\\ C_t&=f_t\ast C_{t-1}+i_t\ast \tilde{C_t}\\ h_t&=o_t \ast tanh(C_t) \end{aligned}\]

Models and Architectures in Word2vec

·3 mins
Generally, word2vec is a language model to predict the words probability based on the context. When build the model, it create word embedding for each word, and word embedding is widely used in many NLP tasks. Models #CBOW (Continuous Bag of Words) #Use the context to predict the probability of current word.

2017


Semi-supervised text classification using doc2vec and label spreading

·2 mins
Here is a simple way to classify text without much human effort and get a impressive performance. It can be divided into two steps: Get train data by using keyword classification Generate a more accurate classification model by using doc2vec and label spreading Keyword-based Classification #Keyword based classification is a simple but effective method.

Parameters in doc2vec

·2 mins
Here are some parameter in gensim’s doc2vec class. window #window is the maximum distance between the predicted word and context words used for prediction within a document. It will look behind and ahead. In skip-gram model, if the window size is 2, the training samples will be this:(the blue word is the input word)

Brief Introduction of Label Propagation Algorithm

·2 mins
As I said before, I’m working on a text classification project. I use doc2vec to convert text into vectors, then I use LPA to classify the vectors. LPA is a simple, effective semi-supervised algorithm. It can use the density of unlabeled data to find a hyperplane to split the data.

Enable C Extension for gensim on Windows

·1 min
These days, I’m working on some text classification works, and I use gensim ’s doc2vec function. When using gensim, it shows this warning message: C extension not loaded for Word2Vec, training will be slow. I search this on Internet and found that gensim has rewrite some part of the code using cython rather than numpy to get better performance.

Some Useful Shell Tools

·2 mins
Here are some shell tools I use, which can boost your productivity. Mordern-unix is a great repo that list lots of modern unix tools. Prezto #A zsh configuration framework. Provides auto completion, prompt theme and lots of modules to work with other useful tools.

Start

·1 min
Over the years, I have read so many programmers’ blogs, which has helped me a lot. Now I think it’s the time to start my own blog. I hope this can enforce myself to review what I have learned, and it would even be better if someone can benefit from it.