DOKK / manpages / debian 12 / libplucene-perl / Plucene::Analysis::WhitespaceTokenizer.3pm.en
Plucene::Analysis::WhitespaceTokenizer(3pm) User Contributed Perl Documentation Plucene::Analysis::WhitespaceTokenizer(3pm)

Plucene::Analysis::WhitespaceTokenizer - white space tokenizer

        # isa Plucene::Analysis::CharTokenizer

A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

2022-12-04 perl v5.36.0