it.unimi.dsi.mg4j.search
Interface DocumentIterator

All Superinterfaces:
IntIterator, Iterable<Interval>, Iterator<Integer>
All Known Subinterfaces:
IndexIterator
All Known Implementing Classes:
AbstractCompositeDocumentIterator, AbstractDocumentIterator, AbstractIndexIterator, AbstractIntersectionDocumentIterator, AbstractOrderedIntervalDocumentIterator, AbstractUnionDocumentIterator, AlignDocumentIterator, AndDocumentIterator, BitStreamHPIndexReader.BitStreamHPIndexReaderIndexIterator, BitStreamIndexReader.BitStreamIndexReaderIndexIterator, CachingDocumentIterator, ConsecutiveDocumentIterator, DifferenceDocumentIterator, DocumentalConcatenatedClusterDocumentIterator, DocumentalConcatenatedClusterIndexIterator, DocumentalMergedClusterDocumentIterator, DocumentalMergedClusterIndexIterator, DocumentIterators.EmptyDocumentIterator, GammaDeltaGammaDeltaBitStreamHPIndexReader.BitStreamHPIndexReaderIndexIterator, GammaDeltaGammaDeltaBitStreamIndexReader.BitStreamIndexReaderIndexIterator, Index.EmptyIndexIterator, LowPassDocumentIterator, MultiTermIndexIterator, NotDocumentIterator, OrderedAndDocumentIterator, OrDocumentIterator, PayloadPredicateDocumentIterator, SkipGammaDeltaGammaDeltaBitStreamIndexReader.BitStreamIndexReaderIndexIterator

public interface DocumentIterator
extends IntIterator, Iterable<Interval>

An iterator over documents (pointers) and their intervals.

Warning: the semantics of nextDocument() has changed significantly in MG4J 1.2.

Warning: from MG4J 1.2, most methods throw an IOException (such exceptions used to be catched and wrapped into a RuntimeException).

Warning: the semantics of skipTo(int) has changed significantly in MG4J 1.1.

Each call to nextDocument() will return a document pointer, or -1 if no more documents are available. Just after the call to nextDocument(), intervalIterator(Index) will return an interval iterator enumerating intervals in the last returned document for the specified index. The latter method may return, as a special result, a special TRUE value: this means that albeit the current document satisfies the query, there is only a generic empty witness to prove it (see TRUE for some elaboration).

Note that this class implements IntIterator. Nonetheless, for performance reasons, the preferred access to the document pointers is nextDocument().

The iterator() method must be an alias for intervalIterator(), and shares the same limitations.

A document iterator is usually structured as composite, with operators as internal nodes and IndexIterators as leaves. The methods accept(DocumentIteratorVisitor) and acceptOnTruePaths(DocumentIteratorVisitor) implement the visitor pattern.

The dispose() method is intended to recursively release all resources associated to a composite document iterator. Note that this is not always what you want, as you might be, say, pooling index readers to reduce the number of file open/close operations. For this reason, we intentionally avoid calling the method “close”.

Warning: the interval enumeration can be carried out only just after a call to nextDocument(). Subsequent calls to nextDocument() or even to Iterator.hasNext() will reset the internal state of the iterator. In particular, trying to enumerate intervals after a call to Iterator.hasNext() will usually throw an IllegalStateException.


Method Summary
 boolean accept(DocumentIteratorVisitor visitor)
          Accepts a visitor.
 boolean acceptOnTruePaths(DocumentIteratorVisitor visitor)
          Accepts a visitor after a call to nextDocument(), limiting recursion to true paths.
 void dispose()
          Disposes this document iterator, releasing all resources.
 int document()
          Returns the last document returned by nextDocument().
 ReferenceSet<Index> indices()
          Returns the set of indices over which this iterator is built.
 IntervalIterator intervalIterator()
          Returns the interval iterator of this document iterator for single-index queries.
 IntervalIterator intervalIterator(Index index)
          Returns the interval iterator of this document iterator for the given index.
 Reference2ReferenceMap<Index,IntervalIterator> intervalIterators()
          Returns an unmodifiable map from indices to interval iterators.
 IntervalIterator iterator()
          An alias for intervalIterator(), that has the same limitations (i.e., it will work only if there is just one index), and that catches IOExceptions.
 int nextDocument()
          Returns the next document provided by this document iterator, or -1 if no more documents are available.
 int nextInt()
          Deprecated. As of MG4J 1.2, the suggested way of iterating over document iterators is nextDocument(), which has been modified so to provide fully lazy iteration. After a couple of releases, however, this annotation will be removed, as it is very practical to have document iterators implementing IntIterator. Its main purpose is to warn people about performance issues solved by nextDocument().
 int skipTo(int n)
          Skips all documents smaller than n.
 
Methods inherited from interface it.unimi.dsi.fastutil.ints.IntIterator
skip
 
Methods inherited from interface java.util.Iterator
hasNext, next, remove
 

Method Detail

intervalIterator

IntervalIterator intervalIterator()
                                  throws IOException
Returns the interval iterator of this document iterator for single-index queries.

This is a commodity method that can be used only for queries built over a single index.

Returns:
an interval iterator.
Throws:
IllegalStateException - if this document iterator is not built on a single index.
IOException
See Also:
intervalIterator(Index)

intervalIterator

IntervalIterator intervalIterator(Index index)
                                  throws IOException
Returns the interval iterator of this document iterator for the given index.

After a call to nextDocument(), this iterator can be used to retrieve the intervals in the current document (the one returned by nextDocument()) for the index index.

Note that if all indices have positions, it is guaranteed that at least one index will return an interval. However, for disjunctive queries it cannot be guaranteed that all indices will return an interval.

Indices without positions always return IntervalIterators.TRUE. Thus, in presence of indices without positions it is possible that no intervals at all are available.

Parameters:
index - an index (must be one over which the query was built).
Returns:
an interval iterator over the current document in index.
Throws:
IOException

intervalIterators

Reference2ReferenceMap<Index,IntervalIterator> intervalIterators()
                                                                 throws IOException
Returns an unmodifiable map from indices to interval iterators.

After a call to nextDocument(), this map can be used to retrieve the intervals in the current document. An invocation of Map.get(java.lang.Object) on this map with argument index yields the same result as intervalIterator(index).

Returns:
a map from indices to interval iterators over the current document.
Throws:
UnsupportedOperationException - if this index does not contain positions.
IOException
See Also:
intervalIterator(Index)

indices

ReferenceSet<Index> indices()
Returns the set of indices over which this iterator is built.

Returns:
the set of indices over which this iterator is built.

nextInt

@Deprecated
int nextInt()
Deprecated. As of MG4J 1.2, the suggested way of iterating over document iterators is nextDocument(), which has been modified so to provide fully lazy iteration. After a couple of releases, however, this annotation will be removed, as it is very practical to have document iterators implementing IntIterator. Its main purpose is to warn people about performance issues solved by nextDocument().

Returns the next document.

Specified by:
nextInt in interface IntIterator
See Also:
nextDocument()

nextDocument

int nextDocument()
                 throws IOException
Returns the next document provided by this document iterator, or -1 if no more documents are available.

Warning: the specification of this method has significantly changed as of MG4J 1.2. The special return value -1 is used to mark the end of iteration (a NoSuchElementException would have been thrown before in that case, so ho harm should be caused by this change). The reason for this change is providing fully lazy iteration over documents. Fully lazy iteration does not provide an hasNext() method—you have to actually ask for the next element and check the return value. Fully lazy iteration is much lighter on method calls (half) and in most (if not all) MG4J classes leads to a much simpler logic. Moreover, nextDocument() can be specified as throwing an IOException, which avoids the pernicious proliferation of try/catch blocks in very short, low-level methods (it was having a detectable impact on performance).

Returns:
the next document, or -1 if no more documents are available.
Throws:
IOException

document

int document()
Returns the last document returned by nextDocument().

Returns:
the last document returned by nextDocument(), or -1 if no document has been returned yet.

skipTo

int skipTo(int n)
           throws IOException
Skips all documents smaller than n.

Define the current document k associated with this document iterator as follows:

If k is larger than or equal to n, then this method does nothing and returns k. Otherwise, a call to this method is equivalent to

 while( ( k = nextDocument() ) < n && k != -1 );
 return k == -1 ? Integer.MAX_VALUE : k;
 

Thus, when a result kInteger.MAX_VALUE is returned, the state of this iterator will be exactly the same as after a call to nextDocument() that returned k. In particular, the first document larger than or equal to n (when returned by this method) will not be returned by the next call to nextDocument().

Parameters:
n - a document pointer.
Returns:
a document pointer larger than or equal to n if available, Integer.MAX_VALUE otherwise.
Throws:
IOException

accept

boolean accept(DocumentIteratorVisitor visitor)
               throws IOException
Accepts a visitor.

A document iterator is usually structured as composite, with operators as internal nodes and IndexIterators as leaves. This method implements the visitor pattern.

Parameters:
visitor - the visitor.
Returns:
true if the visit should continue.
Throws:
IOException

acceptOnTruePaths

boolean acceptOnTruePaths(DocumentIteratorVisitor visitor)
                          throws IOException
Accepts a visitor after a call to nextDocument(), limiting recursion to true paths.

After a call to nextDocument(), a document iterator is positioned over a document. This call is equivalent to accept(DocumentIteratorVisitor), but visits only along true paths.

We define a true path as a path from the root of the composite that passes only through nodes whose associated subtree is positioned on the same document of the root. Note that OrDocumentIterators detach exhausted iterators from the composite tree, so true paths define the subtree that is causing the current document to satisfy the query represented by this document iterator.

For more elaboration, and the main application of this method, see CounterCollectionVisitor.

Parameters:
visitor - the visitor.
Returns:
true if the visit should continue.
Throws:
IOException
See Also:
accept(DocumentIteratorVisitor), CounterCollectionVisitor

dispose

void dispose()
             throws IOException
Disposes this document iterator, releasing all resources.

This method should propagate down to the underlying index iterators, where it should release resources such as open files and network connections. If you're doing your own resource tracking and pooling, then you do not need to call this method.

Throws:
IOException

iterator

IntervalIterator iterator()
An alias for intervalIterator(), that has the same limitations (i.e., it will work only if there is just one index), and that catches IOExceptions.

Specified by:
iterator in interface Iterable<Interval>
Returns:
an interval iterator.