TeeSinkTokenFilter
insteadpublic class SinkTokenizer extends Tokenizer
TeeTokenFilter
and SinkTokenizer
only work with the old TokenStream API.
If you switch to the new API, you need to use TeeSinkTokenFilter
instead, which offers
the same functionality.TeeTokenFilter
AttributeSource.AttributeFactory, AttributeSource.State
Modifier and Type | Field and Description |
---|---|
protected Iterator |
iter
Deprecated.
|
protected List |
lst
Deprecated.
|
Constructor and Description |
---|
SinkTokenizer()
Deprecated.
|
SinkTokenizer(int initCap)
Deprecated.
|
SinkTokenizer(List input)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
void |
add(Token t)
Deprecated.
Override this method to cache only certain tokens, or new tokens based
on the old tokens.
|
void |
close()
Deprecated.
By default, closes the input Reader.
|
List |
getTokens()
Deprecated.
Get the tokens in the internal List.
|
Token |
next(Token reusableToken)
Deprecated.
Returns the next token out of the list of cached tokens
|
void |
reset()
Deprecated.
Reset the internal data structures to the start at the front of the list of tokens.
|
correctOffset, reset
end, getOnlyUseNewAPI, incrementToken, next, setOnlyUseNewAPI
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString
public SinkTokenizer(List input)
public SinkTokenizer()
public SinkTokenizer(int initCap)
public List getTokens()
reset()
method to be called in order for them
to be made available. Also, this Tokenizer does nothing to protect against ConcurrentModificationException
s
in the case of adds happening while next(org.apache.lucene.analysis.Token)
is being called.
WARNING: Since this SinkTokenizer can be reset and the cached tokens made available again, do not modify them. Modify clones instead.Token
spublic Token next(Token reusableToken) throws IOException
next
in class TokenStream
reusableToken
- a Token
that may or may not be used to return;
this parameter should never be null (the callee is not required to
check for null before using it, but it is a good idea to assert that
it is not null.)Token
in the Sink.IOException
public void add(Token t)
t
- The Token
to add to the sinkpublic void close() throws IOException
Tokenizer
close
in class Tokenizer
IOException
public void reset() throws IOException
next(Token)
reset
in class TokenStream
IOException
Copyright © 2000-2012 Apache Software Foundation. All Rights Reserved.