mmseg - maximum matching segment Chinese text.
mmseg -d dict_file [option]...
[corpus_file]...
mmseg is a tool for segmenting Chinese text into words
using maximum matching algorithm. mmseg segments corpus_file,
or standard input if no filename is specified, and write the segmented
result to standard output.
- -d dict_file
- Use dict_file as lexicon. A default lexicon can be found at
/usr/share/sunpinyin-slm/dict.utf8.
- -f,--format
(text|bin)
- Output Format, can be 'text' or 'bin'. default 'bin'. Normally, in text
mode, word text are output, while in binary mode, binary short integer of
the word-ids are written to stdout.
- -s, --stok
STOK_ID
- Sentence token id. Default 10. It will be written to output in binary mode
after every sentence.
- -i, --show-id
- Show Id info. Under text output format mode, attach id after known words.
If under binary mode, print id(s) in text.
- -a, --ambiguious-id
AMBI-ID
- Ambiguious means ABC => A BC or AB
C. If specified (AMBI-ID != 0), The sequence ABC will
not be segmented, in binary mode, the AMBI-ID is written out; in
text mode,
"<ambi>ABC</ambi>" will be
output. Default is 0.
Under binary mode, consecutive id of 0 are merged into one
0. Under text mode, no space are inserted between unknown-words.
Originally written by Phill.Zhang <phill.zhang@sun.com>.
Currently maintained by Kov.Chai <tchaikov@gmail.com>.