com.sleepycat.je.cleaner
Class UtilizationProfile

java.lang.Object
  extended by com.sleepycat.je.cleaner.UtilizationProfile
All Implemented Interfaces:
EnvConfigObserver

public class UtilizationProfile
extends java.lang.Object
implements EnvConfigObserver

The UP tracks utilization summary information for all log files.

Unlike the UtilizationTracker, the UP is not accessed under the log write latch and is instead synchronized on itself for protecting the cache. It is not accessed during the primary data access path, except for when flushing (writing) file summary LNs. This occurs in the following cases:

  1. The summary information is flushed at the end of a checkpoint. This allows tracking to occur in memory in between checkpoints, and replayed during recovery.
  2. When committing the truncateDatabase and removeDatabase operations, the summary information is flushed because detail tracking for those operations is not replayed during recovery
  3. The evictor will ask the UtilizationTracker to flush the largest summary if the memory taken by the tracker exeeds its budget.

The cache is populated by the RecoveryManager just before performing the initial checkpoint. The UP must be open and populated in order to respond to requests to flush summaries and to evict tracked detail, even if the cleaner is disabled.

WARNING: While synchronized on this object, eviction is not permitted. If it were, this could cause deadlocks because the order of locking would be the UP object and then the evictor. During normal eviction the order is to first lock the evictor and then the UP, when evicting tracked detail.

The methods in this class synchronize to protect the cached summary information. Some methods also access the UP database. However, because eviction must not occur while synchronized, UP database access is not performed while synchronized except in one case: when inserting a new summary record. In that case we disallow eviction during the database operation.


Field Summary
(package private)  int minAge
          Minumum age to qualify for cleaning.
(package private)  int minFileUtilization
          Minimum utilization threshold for an individual log file that triggers cleaning.
(package private)  int minUtilization
          Minimum overall utilization threshold that triggers cleaning.
 
Constructor Summary
UtilizationProfile(EnvironmentImpl env, UtilizationTracker tracker)
          Creates an empty UP.
 
Method Summary
 void clearCache()
          Clears the cache of file summary info.
 void countAndLogSummaries(java.util.Collection summaries)
          Count the given tracked info as obsolete and then log the summaries.
 void envConfigUpdate(DbConfigManager cm)
          Process notifications of mutable property changes.
 void flushFileSummary(TrackedFileSummary tfs)
          Updates and stores the FileSummary for a given tracked file, if flushing of the summary is allowed.
(package private)  java.lang.Long getBestFileForCleaning(FileSelector fileSelector, boolean forceCleaning, java.util.Set lowUtilizationFiles)
          Returns the best file that qualifies for cleaning, or null if no file qualifies.
(package private)  java.lang.Long getCheapestFileToClean(java.util.SortedSet files)
          Returns the cheapest file to clean from the given list of files.
(package private)  DatabaseImpl getFileSummaryDb()
          For unit testing.
 java.util.SortedMap getFileSummaryMap(boolean includeTrackedFiles)
          Returns a copy of the current file summary map, optionally including tracked summary information, for use by the DbSpace utility and by unit tests.
(package private)  int getNumberOfFiles()
          Returns the number of files in the profile.
(package private)  TrackedFileSummary getObsoleteDetail(java.lang.Long fileNum, PackedOffsets packedOffsets, boolean logUpdate)
          Returns the stored/packed obsolete offsets and the tracked obsolete offsets for the given file.
(package private)  long getTotalLogSize()
          Returns an approximation of the total log size.
 boolean isRMWFixEnabled()
           
 boolean populateCache()
          Populate the profile for file selection.
(package private)  void removeFile(java.lang.Long fileNum)
          Removes a file from the utilization database and the profile, after it has been deleted by the cleaner.
static int utilization(long obsoleteSize, long totalSize)
          Calculate the utilization percentage.
 boolean verifyFileSummaryDatabase()
          Checks that all FSLN offsets are indeed obsolete.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

minUtilization

int minUtilization
Minimum overall utilization threshold that triggers cleaning. Is non-private for unit tests.


minFileUtilization

int minFileUtilization
Minimum utilization threshold for an individual log file that triggers cleaning. Is non-private for unit tests.


minAge

int minAge
Minumum age to qualify for cleaning. If the first active LSN file is 5 and the mininum age is 2, file 4 won't qualify but file 3 will. Must be greater than zero because we never clean the first active LSN file. Is non-private for unit tests.

Constructor Detail

UtilizationProfile

public UtilizationProfile(EnvironmentImpl env,
                          UtilizationTracker tracker)
                   throws DatabaseException
Creates an empty UP.

Throws:
DatabaseException
Method Detail

envConfigUpdate

public void envConfigUpdate(DbConfigManager cm)
                     throws DatabaseException
Process notifications of mutable property changes.

Specified by:
envConfigUpdate in interface EnvConfigObserver
Throws:
DatabaseException

isRMWFixEnabled

public boolean isRMWFixEnabled()
See Also:
EnvironmentParams.CLEANER_RMW_FIX, FileSummaryLN.postFetchInit(com.sleepycat.je.dbi.DatabaseImpl, long)

getNumberOfFiles

int getNumberOfFiles()
               throws DatabaseException
Returns the number of files in the profile.

Throws:
DatabaseException

getTotalLogSize

long getTotalLogSize()
Returns an approximation of the total log size. Used for stats.


getCheapestFileToClean

java.lang.Long getCheapestFileToClean(java.util.SortedSet files)
                                throws DatabaseException
Returns the cheapest file to clean from the given list of files. This method is used to select the first file to be cleaned in the batch of to-be-cleaned files.

Throws:
DatabaseException

getBestFileForCleaning

java.lang.Long getBestFileForCleaning(FileSelector fileSelector,
                                      boolean forceCleaning,
                                      java.util.Set lowUtilizationFiles)
                                throws DatabaseException
Returns the best file that qualifies for cleaning, or null if no file qualifies.

Parameters:
fileSelector - is used to determine valid cleaning candidates.
forceCleaning - is true to always select a file, even if its utilization is above the minimum utilization threshold.
lowUtilizationFiles - is a returned set of files that are below the minimum utilization threshold.
Throws:
DatabaseException

utilization

public static int utilization(long obsoleteSize,
                              long totalSize)
Calculate the utilization percentage.


countAndLogSummaries

public void countAndLogSummaries(java.util.Collection summaries)
                          throws DatabaseException
Count the given tracked info as obsolete and then log the summaries.

Throws:
DatabaseException

getFileSummaryMap

public java.util.SortedMap getFileSummaryMap(boolean includeTrackedFiles)
                                      throws DatabaseException
Returns a copy of the current file summary map, optionally including tracked summary information, for use by the DbSpace utility and by unit tests. The returned map's key is a Long file number and its value is a FileSummary.

Throws:
DatabaseException

clearCache

public void clearCache()
Clears the cache of file summary info. The cache starts out unpopulated and is populated on the first call to getBestFileForCleaning.


removeFile

void removeFile(java.lang.Long fileNum)
          throws DatabaseException
Removes a file from the utilization database and the profile, after it has been deleted by the cleaner.

Throws:
DatabaseException

flushFileSummary

public void flushFileSummary(TrackedFileSummary tfs)
                      throws DatabaseException
Updates and stores the FileSummary for a given tracked file, if flushing of the summary is allowed.

Throws:
DatabaseException

getObsoleteDetail

TrackedFileSummary getObsoleteDetail(java.lang.Long fileNum,
                                     PackedOffsets packedOffsets,
                                     boolean logUpdate)
                               throws DatabaseException
Returns the stored/packed obsolete offsets and the tracked obsolete offsets for the given file. The tracked summary object returned can be used to test for obsolete offsets that are being added during cleaning by other threads participating in lazy migration. The caller must call TrackedFileSummary.setAllowFlush(true) when cleaning is complete. This method performs eviction and is not synchronized.

Parameters:
logUpdate - if true, log any updates to the utilization profile. If false, only retrieve the new information.
Throws:
DatabaseException

populateCache

public boolean populateCache()
                      throws DatabaseException
Populate the profile for file selection. This method performs eviction and is not synchronized. It must be called before recovery is complete so that synchronization is unnecessary. It must be called before the recovery checkpoint so that the checkpoint can flush file summary information.

Throws:
DatabaseException

getFileSummaryDb

DatabaseImpl getFileSummaryDb()
For unit testing.


verifyFileSummaryDatabase

public boolean verifyFileSummaryDatabase()
                                  throws DatabaseException
Checks that all FSLN offsets are indeed obsolete. Assumes that the system is quiesent (does not lock LNs). This method is not synchronized (because it doesn't access fileSummaryMap) and eviction is allowed.

Returns:
true if no verification failures.
Throws:
DatabaseException


Copyright 2004,2008 Oracle. All rights reserved.