it.unimi.dsi.mg4j.tool
Class Concatenate
java.lang.Object
it.unimi.dsi.mg4j.tool.Combine
it.unimi.dsi.mg4j.tool.Concatenate
public final class Concatenate
- extends Combine
Concatenates several indices.
This implementation of Combine
concatenates
the involved indices: document 0 of the first index is document 0 of the
final collection, but document 0 of the second index is numbered after
the number of documents in the first index, and so on. The resulting
index is exactly what you would obtain by concatenating the document
sequences at the origin of each index.
Note that this class can be used also with a single index, making it possible to recompress easily
an index using different compression flags.
- Since:
- 1.0
- Author:
- Sebastiano Vigna
Fields inherited from class it.unimi.dsi.mg4j.tool.Combine |
DEFAULT_BUFFER_SIZE, frequency, hasCounts, hasPayloads, hasPositions, index, indexIterator, indexReader, indexWriter, inputBasename, maxCount, numberOfDocuments, numberOfOccurrences, numIndices, position, size, termQueue, usedIndex |
Constructor Summary |
Concatenate(String outputBasename,
String[] inputBasename,
boolean metadataOnly,
int bufferSize,
Map<CompressionFlags.Component,CompressionFlags.Coding> writerFlags,
boolean interleaved,
boolean skips,
int quantum,
int height,
int skipBufferSize,
long logInterval)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Concatenate
public Concatenate(String outputBasename,
String[] inputBasename,
boolean metadataOnly,
int bufferSize,
Map<CompressionFlags.Component,CompressionFlags.Coding> writerFlags,
boolean interleaved,
boolean skips,
int quantum,
int height,
int skipBufferSize,
long logInterval)
throws IOException,
ConfigurationException,
URISyntaxException,
ClassNotFoundException,
SecurityException,
InstantiationException,
IllegalAccessException,
InvocationTargetException,
NoSuchMethodException
- Throws:
IOException
ConfigurationException
URISyntaxException
ClassNotFoundException
SecurityException
InstantiationException
IllegalAccessException
InvocationTargetException
NoSuchMethodException
combineNumberOfDocuments
protected int combineNumberOfDocuments()
- Description copied from class:
Combine
- Combines the number of documents.
- Specified by:
combineNumberOfDocuments
in class Combine
- Returns:
- the number of documents of the combined index.
combineSizes
protected int combineSizes()
throws IOException
- Description copied from class:
Combine
- Combines size lists.
- Specified by:
combineSizes
in class Combine
- Returns:
- the maximum size of a document in the combined index.
- Throws:
IOException
combine
protected int combine(int numUsedIndices)
throws IOException
- Description copied from class:
Combine
- Combines several indices.
When this method is called, exactly numUsedIndices
entries
of Combine.usedIndex
contain, in increasing order, the indices containing
inverted lists for the current term. Implementations of this method must
combine the inverted list, save the total global count for the current
term and return the resulting frequency.
- Specified by:
combine
in class Combine
- Parameters:
numUsedIndices
- the number of valid entries in Combine.usedIndex
.
- Returns:
- the frequency of the combined lists.
- Throws:
IOException
main
public static void main(String[] arg)
throws ConfigurationException,
SecurityException,
com.martiansoftware.jsap.JSAPException,
IOException,
URISyntaxException,
ClassNotFoundException,
InstantiationException,
IllegalAccessException,
InvocationTargetException,
NoSuchMethodException
- Throws:
ConfigurationException
SecurityException
com.martiansoftware.jsap.JSAPException
IOException
URISyntaxException
ClassNotFoundException
InstantiationException
IllegalAccessException
InvocationTargetException
NoSuchMethodException