DOKK / manpages / debian 10 / libplucene-perl / Plucene::Analysis::WhitespaceTokenizer.3pm.en
Plucene::Analysis::WhitespaceTokenizer(3pm) User Contributed Perl Documentation Plucene::Analysis::WhitespaceTokenizer(3pm)

Plucene::Analysis::WhitespaceTokenizer - white space tokenizer

        # isa Plucene::Analysis::CharTokenizer

A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

2018-04-02 perl v5.26.1