Uses of Class
org.apache.lucene.analysis.TokenStream

Packages that use TokenStream
org.apache.lucene.analysis API and code to convert text into indexable/searchable tokens. 
org.apache.lucene.analysis.standard A fast grammar-based tokenizer constructed with JFlex. 
org.apache.lucene.document The logical representation of a Document for indexing and searching. 
 

Uses of TokenStream in org.apache.lucene.analysis
 

Subclasses of TokenStream in org.apache.lucene.analysis
 class CachingTokenFilter
          This class can be used if the Tokens of a TokenStream are intended to be consumed more than once.
 class CharTokenizer
          An abstract base class for simple, character-oriented tokenizers.
 class ISOLatin1AccentFilter
          A filter that replaces accented characters in the ISO Latin 1 character set (ISO-8859-1) by their unaccented equivalent.
 class KeywordTokenizer
          Emits the entire input as a single token.
 class LengthFilter
          Removes words that are too long and too short from the stream.
 class LetterTokenizer
          A LetterTokenizer is a tokenizer that divides text at non-letters.
 class LowerCaseFilter
          Normalizes token text to lower case.
 class LowerCaseTokenizer
          LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together.
 class PorterStemFilter
          Transforms the token stream as per the Porter stemming algorithm.
 class SinkTokenizer
          A SinkTokenizer can be used to cache Tokens for use in an Analyzer
 class StopFilter
          Removes stop words from a token stream.
 class TeeTokenFilter
          Works in conjunction with the SinkTokenizer to provide the ability to set aside tokens that have already been analyzed.
 class TokenFilter
          A TokenFilter is a TokenStream whose input is another token stream.
 class Tokenizer
          A Tokenizer is a TokenStream whose input is a Reader.
 class WhitespaceTokenizer
          A WhitespaceTokenizer is a tokenizer that divides text at whitespace.
 

Fields in org.apache.lucene.analysis declared as TokenStream
protected  TokenStream TokenFilter.input
          The source of tokens for this filter.
 

Methods in org.apache.lucene.analysis that return TokenStream
 TokenStream WhitespaceAnalyzer.reusableTokenStream(String fieldName, Reader reader)
           
 TokenStream StopAnalyzer.reusableTokenStream(String fieldName, Reader reader)
           
 TokenStream SimpleAnalyzer.reusableTokenStream(String fieldName, Reader reader)
           
 TokenStream PerFieldAnalyzerWrapper.reusableTokenStream(String fieldName, Reader reader)
           
 TokenStream KeywordAnalyzer.reusableTokenStream(String fieldName, Reader reader)
           
 TokenStream Analyzer.reusableTokenStream(String fieldName, Reader reader)
          Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method.
 TokenStream WhitespaceAnalyzer.tokenStream(String fieldName, Reader reader)
           
 TokenStream StopAnalyzer.tokenStream(String fieldName, Reader reader)
          Filters LowerCaseTokenizer with StopFilter.
 TokenStream SimpleAnalyzer.tokenStream(String fieldName, Reader reader)
           
 TokenStream PerFieldAnalyzerWrapper.tokenStream(String fieldName, Reader reader)
           
 TokenStream KeywordAnalyzer.tokenStream(String fieldName, Reader reader)
           
abstract  TokenStream Analyzer.tokenStream(String fieldName, Reader reader)
          Creates a TokenStream which tokenizes all the text in the provided Reader.
 

Constructors in org.apache.lucene.analysis with parameters of type TokenStream
CachingTokenFilter(TokenStream input)
           
ISOLatin1AccentFilter(TokenStream input)
           
LengthFilter(TokenStream in, int min, int max)
          Build a filter that removes words that are too long or too short from the text.
LowerCaseFilter(TokenStream in)
           
PorterStemFilter(TokenStream in)
           
StopFilter(TokenStream in, Set stopWords)
          Constructs a filter which removes words from the input TokenStream that are named in the Set.
StopFilter(TokenStream input, Set stopWords, boolean ignoreCase)
          Construct a token stream filtering the given input.
StopFilter(TokenStream input, String[] stopWords)
          Construct a token stream filtering the given input.
StopFilter(TokenStream in, String[] stopWords, boolean ignoreCase)
          Constructs a filter which removes words from the input TokenStream that are named in the array of words.
TeeTokenFilter(TokenStream input, SinkTokenizer sink)
           
TokenFilter(TokenStream input)
          Construct a token stream filtering the given input.
 

Uses of TokenStream in org.apache.lucene.analysis.standard
 

Subclasses of TokenStream in org.apache.lucene.analysis.standard
 class StandardFilter
          Normalizes tokens extracted with StandardTokenizer.
 class StandardTokenizer
          A grammar-based tokenizer constructed with JFlex
 

Methods in org.apache.lucene.analysis.standard that return TokenStream
 TokenStream StandardAnalyzer.reusableTokenStream(String fieldName, Reader reader)
           
 TokenStream StandardAnalyzer.tokenStream(String fieldName, Reader reader)
          Constructs a StandardTokenizer filtered by a StandardFilter, a LowerCaseFilter and a StopFilter.
 

Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream
StandardFilter(TokenStream in)
          Construct filtering in.
 

Uses of TokenStream in org.apache.lucene.document
 

Methods in org.apache.lucene.document that return TokenStream
 TokenStream Field.tokenStreamValue()
          The value of the field as a TokesStream, or null.
 TokenStream Fieldable.tokenStreamValue()
          The value of the field as a TokenStream, or null.
 

Methods in org.apache.lucene.document with parameters of type TokenStream
 void Field.setValue(TokenStream value)
          Expert: change the value of this field.
 

Constructors in org.apache.lucene.document with parameters of type TokenStream
Field(String name, TokenStream tokenStream)
          Create a tokenized and indexed field that is not stored.
Field(String name, TokenStream tokenStream, Field.TermVector termVector)
          Create a tokenized and indexed field that is not stored, optionally with storing term vectors.
 



Copyright © 2000-2008 Apache Software Foundation. All Rights Reserved.