/usr/share/perl5/Plucene/Analysis/LetterTokenizer.pm is in libplucene-perl 1.25-3.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | package Plucene::Analysis::LetterTokenizer;
=head1 NAME
Plucene::Analysis::LetterTokenizer - Letter tokenizer
=head1 SYNOPSIS
# isa Plucene::Analysis::CharTokenizer
=head1 DESCRIPTION
This is the letter tokenizer class, which divides text at non-letters.
Note: this does a decent job for most European languages, but does a
terrible job for some Asian languages, where words are not separated
by spaces
=cut
use strict;
use warnings;
use base 'Plucene::Analysis::CharTokenizer';
sub token_re { qr/[[:alpha:]]+/ }
1;
|