Class CrossProductTokenization

  • All Implemented Interfaces:
    java.io.Serializable, Annotatable, SymbolTokenization, Changeable

    public class CrossProductTokenization
    extends WordTokenization
    Tokenization for cross-product alphabets. This class handles the general case of tokens of the form (foo bar baz), where each element is handled by a sub-tokenization. By default, these will be the "name" tokenizations of each of the sub-alphabets, but any tokenization can be used.
    Since:
    1.2
    Author:
    Thomas Down, Greg Cox
    See Also:
    Serialized Form