You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
NNSegmentation is a package for Word Segmentation using neural networks based on package LibN3L.
It includes different combination of Neural network architectures (TNN, RNN, GatedNN, LSTM and GRNN) with Objective function(Softmax, CRF Max-Margin, CRF Maximum Likelihood).
It also provides the capability of combination of Sparse feature along with above models.
In addition, this package can easily support various user-defined neural network structures.
Open CMakeLists.txt and change "../LibN3L/" into the directory of your LibN3L package.
cmake . make
Example
This example shows how to train three Chinese word segmentation models for the pku corpus of the Sighan Bakeoff 2005 dataset.
These models are
SparseCRFMMLabeler which only considers the sparse features and works like a CRF model
LSTMCRFMMLabeler which only uses neural embeddings as input and employs CRF Maximum Likelihood as training objective.
SparseLSTMCRFMMLabeler which supports both neural embeddings and sparse features and also employs CRF Maximum Likelihood as training objective.
This example data contains
Sparse Features "train.feats", "dev.feats" and "test.feats". The training features and dev features are extracted only from a subset of the pku corpus.