Class V2MasterTableDataSource
- java.lang.Object
-
- com.mckoi.database.MasterTableDataSource
-
- com.mckoi.database.V2MasterTableDataSource
-
public final class V2MasterTableDataSource extends MasterTableDataSource
A MasterTableDataSource that is backed by a non-shared com.mckoi.store.Store object. The store interface allows us a great deal of flexibility because we can map a store around different underlying devices. For example, a store could map to a memory region, a memory mapped file, or a standard random access file.Unlike V1MasterTableDataSource, this manages data and index information in a single store which can be backed by a single file in the file system.
The structure of the store comprises of a header block that contains the following information;
HEADER BLOCK +-------------------------------+ | version | | table id | | table sequence id | | pointer to DataTableDef | | pointer to DataIndexSetDef | | pointer to index block | | LIST BLOCK HEADER pointer | +-------------------------------+
Each record is comprised of a header which contains offsets to the fields in the record, and a serializable of the fields themselves.
-
-
Field Summary
Fields Modifier and Type Field Description private java.lang.String
file_name
The file name of this store in the conglomerate path.private long
first_delete_chain_record
The first delete chain element.private boolean
has_shutdown
Set to true when the VM has shutdown and writes should no longer be possible on the object.private MutableArea
header_area
The header area itself.private long
index_header_p
Points to the index header area.private IndexSetStore
index_store
An IndexSetStore object that manages the indexes for this table.private long
list_header_p
Points to the block list header area.private FixedRecordList
list_structure
The structure that manages the pointers to the records.private short
s_run_file_hits
private long
sequence_id
The current sequence id.private Store
store
The backing store object.-
Fields inherited from class com.mckoi.database.MasterTableDataSource
blob_store_interface, cache, column_count, column_rid_list, DATA_CELL_CACHING, delete_hits_key, file_hits_key, garbage_collector, index_def, insert_hits_key, is_closed, root_lock_key, table_def, table_id, table_indices, total_hits_key
-
-
Constructor Summary
Constructors Constructor Description V2MasterTableDataSource(TransactionSystem system, StoreSystem store_system, OpenTransactionList open_transactions, BlobStoreInterface blob_store_interface)
The Constructor.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description private long
addToRecordList(long record_p)
Finds a free place to add a record and returns an index to the record here.private long
addToRecordList(long index, long record_p)
Adds a record to the given position in the fixed structure.void
checkAndRepair(java.lang.String file_name, UserTerminal terminal)
Performs a complete check and repair of the table.private boolean
checkAndRepairRecord(int row_index, java.util.List all_areas, UserTerminal terminal)
Checks and repairs a record if it requires repairing.(package private) void
checkForCleanup()
Checks to determine if it is safe to clean up any resources in the table, and if it is safe to do so, the space is reclaimed.(package private) void
close(boolean pending_drop)
Closes this master table in the file system.(package private) void
commitIndexSet(IndexSet index_set)
Commits changes made to an IndexSet returned by the 'createIndexSet' method.(package private) void
copy(int table_id, MasterTableDataSource src_master_table, IndexSet index_set)
Creates a new v2 master table data source that is a copy of the given MasterTableDataSource object.private void
copyRecordFrom(MasterTableDataSource src_master_table, int record_id)
Copies the record at the given index in the source table to the same record index in this table.(package private) void
create(int table_id, DataTableDef table_def)
Create this master table in the file system at the given path.(package private) IndexSet
createIndexSet()
Creates and returns an IndexSet object that is used to create indices for this table source.(package private) long
currentUniqueID()
Atomically returns the current 'unique_id' value for this table.(package private) void
dispose(boolean pending_drop)
Disposes of all in-memory resources associated with this table and invalidates this object.(package private) boolean
drop()
Disposes and drops this table.private void
dropAllBlobReferences()
Scans the table and drops ALL blob references in this table.(package private) boolean
exists(java.lang.String identity)
Returns true if the master table data source with the given source identity exists.private static java.io.DataInputStream
getDIn(java.io.InputStream in)
Convenience - wraps the given input stream around a buffered data input stream.private static java.io.DataOutputStream
getDOut(java.io.OutputStream out)
Convenience - wraps the given output stream around a buffered data output stream.(package private) java.lang.String
getSourceIdent()
Returns a string that uniquely identifies this table within the conglomerate context.private void
growListStructure()
Grows the list structure to accomodate more entries.(package private) int
internalAddRow(RowData data)
Adds a new row to this table and returns an index that is used to reference this row by the 'getCellContents' method.(package private) void
internalDeleteRow(int row_index)
Removes the row at the given index so that any resources associated with the row may be immediately available to be recycled.(package private) TObject
internalGetCellContents(int column, int row)
Returns the cell contents of the given cell in the table.(package private) boolean
isWorthCompacting()
Returns true if a compact table is necessary.(package private) long
nextUniqueID()
Atomically returns the next 'unique_id' value from this table.void
open(java.lang.String file_name)
Opens an existing master table from the file system at the path of the conglomerate this belongs to.(package private) int
rawRowCount()
Returns the raw count or rows in the table, including uncommited, committed and deleted rows.(package private) int
readRecordType(int row_index)
Reads the record state for the given record in the table.private void
readStoreHeaders()
Read the store headers and initialize any internal object state.(package private) boolean
recordDeleted(int row_index)
Returns true if the record with the given index is deleted from the table.private void
removeAllBlobReferencesForRecord(long record_p)
Removes all blob references in the record area pointed to by 'record_p'.void
scanForLeaks()
Looks for any leaks in the file.(package private) void
setUniqueID(long value)
Sets the unique id for this store.private void
setupInitialStore()
Sets up an initial store (should only be called from the 'create' method).(package private) void
shutdownHookCleanup()
Called by the 'shutdown hook' on the conglomerate.private void
skipStream(java.io.InputStream in, long amount)
java.lang.String
toString()
For diagnostic.private long
writeRecordToStore(RowData data)
Writes a record to the store and returns a pointer to the area that represents the new record.(package private) int
writeRecordType(int row_index, int row_state)
Sets the record type for the given record in the table and returns the previous state of the record.-
Methods inherited from class com.mckoi.database.MasterTableDataSource
addRootLock, addRow, buildIndex, buildIndexes, cachedTableName, clearAllRootLocks, commitTransactionChange, createSelectableSchemeForColumn, createSelectableSchemeForIndex, createTableDataSourceAtCommit, createTableDataSourceAtCommit, Debug, doOpeningScan, findAllJournalsSince, getCellContents, getDataIndexSetDef, getDataTableDef, getName, getRawDiagnosticTable, getSchema, getSystem, getTableID, getTableName, hardCheckAndReclaimRow, hardRemoveRow, hasTransactionChangesPending, isClosed, isReadOnly, isRootLocked, loadInternal, makeTableFileName, mergeJournalChanges, minimalTableDataSource, recordTypeInfo, removeRootLock, rollbackTransactionChange, setupDataIndexSetDef, setupDataTableDef, storeSystem, transactionChangeString
-
-
-
-
Field Detail
-
file_name
private java.lang.String file_name
The file name of this store in the conglomerate path.
-
store
private Store store
The backing store object.
-
index_store
private IndexSetStore index_store
An IndexSetStore object that manages the indexes for this table.
-
sequence_id
private long sequence_id
The current sequence id.
-
index_header_p
private long index_header_p
Points to the index header area.
-
list_header_p
private long list_header_p
Points to the block list header area.
-
header_area
private MutableArea header_area
The header area itself.
-
list_structure
private FixedRecordList list_structure
The structure that manages the pointers to the records.
-
first_delete_chain_record
private long first_delete_chain_record
The first delete chain element.
-
has_shutdown
private boolean has_shutdown
Set to true when the VM has shutdown and writes should no longer be possible on the object.
-
s_run_file_hits
private short s_run_file_hits
-
-
Constructor Detail
-
V2MasterTableDataSource
public V2MasterTableDataSource(TransactionSystem system, StoreSystem store_system, OpenTransactionList open_transactions, BlobStoreInterface blob_store_interface)
The Constructor.
-
-
Method Detail
-
getDOut
private static java.io.DataOutputStream getDOut(java.io.OutputStream out)
Convenience - wraps the given output stream around a buffered data output stream.
-
getDIn
private static java.io.DataInputStream getDIn(java.io.InputStream in)
Convenience - wraps the given input stream around a buffered data input stream.
-
setupInitialStore
private void setupInitialStore() throws java.io.IOException
Sets up an initial store (should only be called from the 'create' method).- Throws:
java.io.IOException
-
readStoreHeaders
private void readStoreHeaders() throws java.io.IOException
Read the store headers and initialize any internal object state. This is called by the 'open' method.- Throws:
java.io.IOException
-
create
void create(int table_id, DataTableDef table_def) throws java.io.IOException
Create this master table in the file system at the given path. This will initialise the various file objects and result in a new empty master table to store data in.- Throws:
java.io.IOException
-
exists
boolean exists(java.lang.String identity) throws java.io.IOException
Returns true if the master table data source with the given source identity exists.- Throws:
java.io.IOException
-
open
public void open(java.lang.String file_name) throws java.io.IOException
Opens an existing master table from the file system at the path of the conglomerate this belongs to. This will set up the internal state of this object with the data read in.- Throws:
java.io.IOException
-
close
void close(boolean pending_drop) throws java.io.IOException
Closes this master table in the file system. This frees up all the resources associated with this master table.This method is typically called when the database is shut down.
- Throws:
java.io.IOException
-
copy
void copy(int table_id, MasterTableDataSource src_master_table, IndexSet index_set) throws java.io.IOException
Creates a new v2 master table data source that is a copy of the given MasterTableDataSource object.- Parameters:
table_id
- the table id to given the new table.src_master_table
- the table to copy.index_set
- the view of the table to be copied.- Throws:
java.io.IOException
-
writeRecordToStore
private long writeRecordToStore(RowData data) throws java.io.IOException
Writes a record to the store and returns a pointer to the area that represents the new record. This does not manipulate the fixed structure in any way. This method only allocates an area to store the record and serializes the record. It is the responsibility of the callee to add the record into the general file structure.Note that if the RowData contains any references to Blob objects then a reference count to the blob is generated at this point.
- Throws:
java.io.IOException
-
copyRecordFrom
private void copyRecordFrom(MasterTableDataSource src_master_table, int record_id) throws java.io.IOException
Copies the record at the given index in the source table to the same record index in this table. Note that this may need to expand the fixed list record heap as necessary to copy the record into the given position. The record is NOT copied into the first free record position.- Throws:
java.io.IOException
-
removeAllBlobReferencesForRecord
private void removeAllBlobReferencesForRecord(long record_p) throws java.io.IOException
Removes all blob references in the record area pointed to by 'record_p'. This should only be used when the record is be reclaimed.- Throws:
java.io.IOException
-
dropAllBlobReferences
private void dropAllBlobReferences() throws java.io.IOException
Scans the table and drops ALL blob references in this table. This is used when a table is dropped when is still contains elements referenced in the BlobStore. This will decrease the reference count in the BlobStore for all blobs. In effect, this is like calling 'delete' on all the data in the table.This method should only be called when the table is about to be deleted from the file system.
- Throws:
java.io.IOException
-
scanForLeaks
public void scanForLeaks() throws java.io.IOException
Looks for any leaks in the file. This works by walking through the file and index area graph and 'remembering' all areas that were read. The store is then checked that all other areas except these are deleted.Assumes the master table is open.
- Throws:
java.io.IOException
-
checkAndRepair
public void checkAndRepair(java.lang.String file_name, UserTerminal terminal) throws java.io.IOException
Performs a complete check and repair of the table. The table must not have been opened before this method is called. The given UserTerminal parameter is an implementation of a user interface that is used to ask any questions and output the results of the check.- Throws:
java.io.IOException
-
checkAndRepairRecord
private boolean checkAndRepairRecord(int row_index, java.util.List all_areas, UserTerminal terminal) throws java.io.IOException
Checks and repairs a record if it requires repairing. Returns true if the record is valid, or false otherwise (record is/was deleted).- Throws:
java.io.IOException
-
growListStructure
private void growListStructure() throws java.io.IOException
Grows the list structure to accomodate more entries. The new entries are added to the free chain pool. Assumes we are synchronized over list_structure.- Throws:
java.io.IOException
-
addToRecordList
private long addToRecordList(long index, long record_p) throws java.io.IOException
Adds a record to the given position in the fixed structure. If the place is already used by a record then an exception is thrown, otherwise the record is set.- Throws:
java.io.IOException
-
addToRecordList
private long addToRecordList(long record_p) throws java.io.IOException
Finds a free place to add a record and returns an index to the record here. This may expand the record space as necessary if there are no free record slots to use.- Throws:
java.io.IOException
-
getSourceIdent
java.lang.String getSourceIdent()
Description copied from class:MasterTableDataSource
Returns a string that uniquely identifies this table within the conglomerate context. For example, the filename of the table. This string can be used to open and initialize the table also.- Specified by:
getSourceIdent
in classMasterTableDataSource
-
writeRecordType
int writeRecordType(int row_index, int row_state) throws java.io.IOException
Description copied from class:MasterTableDataSource
Sets the record type for the given record in the table and returns the previous state of the record. This is used to change the state of a row in the table.- Specified by:
writeRecordType
in classMasterTableDataSource
- Throws:
java.io.IOException
-
readRecordType
int readRecordType(int row_index) throws java.io.IOException
Description copied from class:MasterTableDataSource
Reads the record state for the given record in the table.- Specified by:
readRecordType
in classMasterTableDataSource
- Throws:
java.io.IOException
-
recordDeleted
boolean recordDeleted(int row_index) throws java.io.IOException
Description copied from class:MasterTableDataSource
Returns true if the record with the given index is deleted from the table. A deleted row can not be read.- Specified by:
recordDeleted
in classMasterTableDataSource
- Throws:
java.io.IOException
-
rawRowCount
int rawRowCount() throws java.io.IOException
Description copied from class:MasterTableDataSource
Returns the raw count or rows in the table, including uncommited, committed and deleted rows. This is basically the maximum number of rows we can iterate through.- Specified by:
rawRowCount
in classMasterTableDataSource
- Throws:
java.io.IOException
-
internalDeleteRow
void internalDeleteRow(int row_index) throws java.io.IOException
Description copied from class:MasterTableDataSource
Removes the row at the given index so that any resources associated with the row may be immediately available to be recycled.- Specified by:
internalDeleteRow
in classMasterTableDataSource
- Throws:
java.io.IOException
-
createIndexSet
IndexSet createIndexSet()
Description copied from class:MasterTableDataSource
Creates and returns an IndexSet object that is used to create indices for this table source. The IndexSet represents a snapshot of the table and the given point in time.NOTE: Not synchronized because we synchronize in the IndexStore object.
- Specified by:
createIndexSet
in classMasterTableDataSource
-
commitIndexSet
void commitIndexSet(IndexSet index_set)
Description copied from class:MasterTableDataSource
Commits changes made to an IndexSet returned by the 'createIndexSet' method. This method also disposes the IndexSet so it is no longer valid.- Specified by:
commitIndexSet
in classMasterTableDataSource
-
internalAddRow
int internalAddRow(RowData data) throws java.io.IOException
Description copied from class:MasterTableDataSource
Adds a new row to this table and returns an index that is used to reference this row by the 'getCellContents' method.Note that this method will not effect the master index or column schemes. This is a low level mechanism for adding unreferenced data into a conglomerate. The data is referenced by committing the change where it eventually migrates into the master index and schemes.
- Specified by:
internalAddRow
in classMasterTableDataSource
- Throws:
java.io.IOException
-
checkForCleanup
void checkForCleanup()
Description copied from class:MasterTableDataSource
Checks to determine if it is safe to clean up any resources in the table, and if it is safe to do so, the space is reclaimed.- Specified by:
checkForCleanup
in classMasterTableDataSource
-
skipStream
private void skipStream(java.io.InputStream in, long amount) throws java.io.IOException
- Throws:
java.io.IOException
-
internalGetCellContents
TObject internalGetCellContents(int column, int row)
Description copied from class:MasterTableDataSource
Returns the cell contents of the given cell in the table. It is the responsibility of the implemented method to perform caching as it deems fit. Some representations may not require such extensive caching as others.- Specified by:
internalGetCellContents
in classMasterTableDataSource
-
currentUniqueID
long currentUniqueID()
Description copied from class:MasterTableDataSource
Atomically returns the current 'unique_id' value for this table.- Specified by:
currentUniqueID
in classMasterTableDataSource
-
nextUniqueID
long nextUniqueID()
Description copied from class:MasterTableDataSource
Atomically returns the next 'unique_id' value from this table.- Specified by:
nextUniqueID
in classMasterTableDataSource
-
setUniqueID
void setUniqueID(long value)
Description copied from class:MasterTableDataSource
Sets the unique id for this store. This must only be used under extraordinary circumstances, such as restoring from a backup, or converting from one file to another.- Specified by:
setUniqueID
in classMasterTableDataSource
-
dispose
void dispose(boolean pending_drop) throws java.io.IOException
Description copied from class:MasterTableDataSource
Disposes of all in-memory resources associated with this table and invalidates this object. If 'pending_drop' is true then the table is to be disposed pending a call to 'drop'. If 'pending_drop' is true then any persistant resources that are allocated may be freed.- Specified by:
dispose
in classMasterTableDataSource
- Throws:
java.io.IOException
-
drop
boolean drop() throws java.io.IOException
Description copied from class:MasterTableDataSource
Disposes and drops this table. If the dispose failed for any reason, it returns false, otherwise true. If the drop failed, it should be retried at a later time.- Specified by:
drop
in classMasterTableDataSource
- Throws:
java.io.IOException
-
shutdownHookCleanup
void shutdownHookCleanup()
Description copied from class:MasterTableDataSource
Called by the 'shutdown hook' on the conglomerate. This method should block until the table can by put into a safe mode and then prevent any further access to the object after it returns. It must operate very quickly.- Specified by:
shutdownHookCleanup
in classMasterTableDataSource
-
isWorthCompacting
boolean isWorthCompacting()
Description copied from class:MasterTableDataSource
Returns true if a compact table is necessary. By default, we return true however it is recommended this method is overwritten and the table tested.- Overrides:
isWorthCompacting
in classMasterTableDataSource
-
toString
public java.lang.String toString()
For diagnostic.- Overrides:
toString
in classjava.lang.Object
-
-