Fennel: CacheImpl< PageT, VictimPolicyT > Class Template Reference (original) (raw)

CacheImpl is a template implementation of the Cache interface. More...

#include <[CacheImpl.h](CacheImpl%5F8h-source.html)>

Inheritance diagram for CacheImpl< PageT, VictimPolicyT >:

List of all members.

Public Member Functions
CacheImpl (CacheParams const &, CacheAllocator *=NULL)
virtual void setAllocatedPageCount (uint nMemPages)
Resizes this cache.
virtual uint getAllocatedPageCount ()
Gets a count of how many pages currently have allocated buffers.
virtual uint getMaxAllocatedPageCount ()
**Returns:**maximum number of pages that can be allocated; i.e., the value of the cachePagesMax parameter
virtual PageT * lockPage (BlockId blockId, LockMode lockMode, bool readIfUnmapped, MappedPageListener *pMappedPageListener, TxnId txnId)
Locks a page into memory with the specified concurrency mode.
virtual PageT & lockScratchPage (BlockNum blockNum)
Allocates a free page buffer for scratch usage.
virtual void discardPage (BlockId blockId)
Unmaps a page from the cache if already mapped, discarding its contents if dirty.
virtual uint checkpointPages (PagePredicate &pagePredicate, CheckpointType checkpointType)
Flushes and/or unmaps selected pages.
virtual void collectStats (CacheStats &stats)
Gets a snapshot of cache activity; as a side-effect, clears cumulative performance counters.
virtual void registerDevice (DeviceId deviceId, SharedRandomAccessDevice pDevice)
Registers the given device with the Cache; must be called exactly once before any other caching operations can be requested for pages of this device.
virtual void unregisterDevice (DeviceId deviceId)
Unregisters the given device from the Cache, asserting that no pages remain mapped to the specified device.
virtual SharedRandomAccessDevice & getDevice (DeviceId deviceId)
Dereferences a device ID to the registered object which represents it.
virtual bool prefetchPage (BlockId blockId, MappedPageListener *pMappedPageListener)
Hints that a page should be prefetched in preparation for a future lock request.
virtual void prefetchBatch (BlockId blockId, uint nPages, MappedPageListener *pMappedPageListener)
Hints that a contiguous run of pages should be prefetched.
virtual void flushPage (CachePage &page, bool async)
Forces the contents of a dirty page to its mapped location.
virtual void unlockPage (CachePage &page, LockMode lockMode, TxnId txnId)
Releases lock held on page.
virtual void nicePage (CachePage &page)
Marks a page as nice, indicating that it is very unlikely the page's mapping will be needed again any time soon, so it is a good candidate for victimization.
virtual bool isPageMapped (BlockId blockId)
Determines if a particular page is mapped.
virtual CacheAllocator & getAllocator () const
virtual void getPrefetchParams (uint &prefetchPagesMax, uint &prefetchThrottleRate)
Retrieves the current pre-fetch caching parameters that determine how many pages should be pre-fetched and how often the pre-fetches should occur.
virtual DeviceAccessScheduler & getDeviceAccessScheduler (RandomAccessDevice &)
Gets the correct access scheduler for a given device.
uint getPageSize () const
**Returns:**the size of cached pages (in bytes)
virtual SharedCache getCache ()
**Returns:**the underlying Cache accessed by this CacheAccessor
virtual uint getMaxLockedPages ()
**Returns:**the page lock quota on this accessor
virtual void setMaxLockedPages (uint nPages)
Sets the page lock quota on this accessor.
virtual void setTxnId (TxnId txnId)
Sets a default TxnId to use for locking pages (to be used when IMPLICIT_TXN_ID is specified).
virtual TxnId getTxnId () const
**Returns:**default TxnId associated with this accessor
virtual void writeStats (StatsTarget &target)
Writes a current stats snapshot to a StatsTarget.
bool isClosed () const
**Returns:**whether the object has been closed
void close ()
Closes this object, releasing any unallocated resources.
Static Public Member Functions
static SharedCache newCache (CacheParams const &cacheParams, CacheAllocator *bufferAllocator=NULL)
Factory method.
Static Public Attributes
static const DeviceId NULL_DEVICE_ID
The DeviceId assigned to the instance of RandomAccessNullDevice associated with every cache.
Protected Member Functions
void closeImpl ()
Must be implemented by derived class to release any resources.
Protected Attributes
uint cbPage
bool needsClose
Private Types
enum FlushPhase { phaseSkip, phaseInitiate, phaseWait }
Flush state used inside of checkpointPages. More...
typedef PageBucket< PageT > PageBucketT
typedef PageBucketT::PageListIter PageBucketIter
typedef PageBucketT::PageListMutator PageBucketMutator
typedef VictimPolicyT::PageIterator VictimPageIterator
typedef VictimPolicyT::DirtyPageIterator DirtyVictimPageIterator
typedef VictimPolicyT::SharedGuard VictimSharedGuard
typedef VictimPolicyT::ExclusiveGuard VictimExclusiveGuard
Private Member Functions
PageT * lookupPage (PageBucketT &bucket, BlockId blockId, bool pin)
Finds a page by BlockId within a particular bucket.
PageT * findFreePage ()
Obtains a free page (either from the free queue or by victimizing a mapped page); if none is available, suspends for a little while to help reduce cache load.
void flushSomePages ()
Initiates asynchronous writes for a few dirty pages which are the best victimization candidates.
bool transferPageAsync (PageT &page)
Performs an asynchronous I/O operation on the given page.
bool readPageAsync (PageT &page)
Reads the given page asynchronously.
bool writePageAsync (PageT &page)
Writes the given page asynchronously.
FileSize getPageOffset (BlockId const &blockId)
Translates a BlockId into the byte offset of the corresponding device block.
PageBucketT & getHashBucket (BlockId const &blockId)
Gets the hash bucket containing the given BlockId.
void assertCorrectBucket (PageBucketT &bucket, BlockId const &blockId)
Verifies the match between a PageBucket and BlockId.
void unmapPage (PageT &page, StrictMutexGuard &guard, bool discard)
Unmaps a currently mapped page, but does not add it to the free list.
void unmapAndFreeDiscardedPage (PageT &page, StrictMutexGuard &guard)
Unmaps a page being discarded and adds it to unmappedBucket.
PageT & mapPage (PageBucketT &bucket, PageT &newPage, BlockId blockId, MappedPageListener *pMappedPageListener, bool bPendingRead=true, bool bIncRef=true)
Maps a page if it is not already mapped (and notifies victimPolicy of the page mapping); otherwise, finds the existing mapping (and notifies victimPolicy of the page access).
void freePage (PageT &page)
Places an unmapped page in unmappedBucket, making it available for the next call to findFreePage.
bool canVictimizePage (PageT &page)
Decides whether a page can be victimized.
void incrementCounter (AtomicCounter &x)
Increments a counter variable safely.
void decrementCounter (AtomicCounter &x)
Decrements a counter variable safely.
void incrementStatsCounter (AtomicCounter &x)
Increments a statistical counter.
void decrementStatsCounter (AtomicCounter &x)
Decrements a statistical counter.
void initializeStats ()
Clears stats which are tracked since initialization.
void allocatePages (CacheParams const &params)
Handles initial allocation of pages and attempts to handle any associated out-of-memory errors.
void successfulPrefetch ()
Updates counters indicating a successful pre-fetch has occurred.
void rejectedPrefetch ()
Updates counters indicating a pre-fetch was rejected.
void ioRetry ()
Updates counters indicating I/O retry was required.
void calcDirtyThreshholds (uint nCachePages)
Calculates the number of dirty pages corresponding to the high and low water marks.
void markPageDirty (CachePage &page)
void notifyTransferCompletion (CachePage &, bool)
virtual uint getTimerIntervalMillis ()
Calculates the interval which should elapse before the next call to onTimerInterval.
virtual void onTimerInterval ()
Receives notification from TimerThread that interval has elapsed.
virtual void onThreadStart ()
Called in new thread context before thread's body runs.
virtual void onThreadEnd ()
Called in thread context after thread's body runs.
virtual FennelExcn * cloneExcn (std::exception &ex)
Clones an exception so that it can be rethrown in a different thread context.
Private Attributes
std::vector< SharedRandomAccessDevice > deviceTable
Collection of registered devices indexed by DeviceId; this array is of fixed size, with a NULL slot indicating that the given device ID has not been registered; this permits synchronization-free access to the collection.
PageBucketT unmappedBucket
Bucket of free unmapped pages whose buffers are still allocated.
PageBucketT unallocatedBucket
Bucket of free pages whose buffers are not allocated.
std::vector< PageBucketT * > pageTable
Array of PageBuckets indexed by BlockId hash code.
uint dirtyHighWaterPercent
Percentage of pages in the cache that must be dirty for lazy writes to be initiated.
uint dirtyLowWaterPercent
Percentage of pages in the cache that must be dirty for lazy writes to be suspended.
uint dirtyHighWaterMark
Number of dirty pages in the cache corresponding to the high-water percentage.
uint dirtyLowWaterMark
Number of dirty pages in the cache corresponding to the low-water percentage.
bool inFlushMode
Used by the lazy writer thread to indicate that the high-water dirty threshhold has been reached and page flushes should continue until the low-water threshhold is reached.
AtomicCounter nCacheHits
See CacheStats::nHits.
AtomicCounter nCacheRequests
See CacheStats::nRequests.
AtomicCounter nVictimizations
See CacheStats::nVictimizations.
AtomicCounter nDirtyPages
See CacheStats::nDirtyPages.
AtomicCounter nPageReads
See CacheStats::nPageReads.
AtomicCounter nPageWrites
See CacheStats::nPageWrites.
AtomicCounter nRejectedCachePrefetches
See CacheStats::nRejectedPrefetches.
AtomicCounter nIoRetries
See CacheStats::nIoRetries.
AtomicCounter nSuccessfulCachePrefetches
See CacheStats::nSuccessfulPrefetches.
AtomicCounter nLazyWrites
See CacheStats::nLazyWrites.
AtomicCounter nLazyWriteCalls
See CacheStats::nLazyWriteCalls.
AtomicCounter nVictimizationWrites
See CacheStats::nVictimizationWrites.
AtomicCounter nCheckpointWrites
See CacheStats::nCheckpointWrites.
CacheStats statsSinceInit
Accumulated state for all counters which are tracked since cache initialization.
StrictMutex freePageMutex
Mutex coupled with freePageCondition.
LocalCondition freePageCondition
Condition variable used for notification of free page availability.
std::vector< PageT * > pages
A fixed-size vector of pointers to cache pages; we can get away with this because currently the number of pages is fixed at initialization.
DeviceAccessScheduler * pDeviceAccessScheduler
Scheduler for asynchronous I/O.
CacheAllocator & bufferAllocator
Source of buffer memory.
boost::scoped_ptr< CacheAllocator > pBufferAllocator
Set only if bufferAllocator is owned by this cache.
VictimPolicyT victimPolicy
The realization for the VictimPolicy model.
TimerThread timerThread
Thread for running idle flush.
uint idleFlushInterval
See also:CacheParams
uint prefetchPagesMax
Maximum number of outstanding pre-fetch requests.
uint prefetchThrottleRate
The number of successful pre-fetches that have to occur before the pre-fetch rate is throttled back up, in the event that it has been throttled down due to rejected requests.

Detailed Description

template<class PageT, class VictimPolicyT>

class CacheImpl< PageT, VictimPolicyT >

CacheImpl is a template implementation of the Cache interface.

It collaborates with the types of its template parameters (PageT, VictimPolicyT). The realization of PageT must always be derived from Page; it may have other inheritance requirements depending on the realization for VictimPolicyT.

Synchronization within the cache accomplishes three distinct goals:

Synchronization occurs at several different levels:

To avoid deadlock, lock acquisition order must follow these rules:

Note that some of the code in CacheImpl (e.g. the methods lookupPage or markPageDirty) could be moved to other classes such as PageBucket or CachePage in standard object-oriented design. However, this IS not and SHOULD not be done. Why? Because the cache synchronization logic is complex, so every effort should be made to keep it easy to understand, verify, and debug. Keeping method implementations as units rather than distributed over a number of objects helps with this.

CacheImpl implements TimerThreadClient for taking repeated action on timer callbacks (e.g. idle flush).

Definition at line 115 of file CacheImpl.h.


Member Typedef Documentation

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>


Member Enumeration Documentation

template<class PageT, class VictimPolicyT>

Flush state used inside of checkpointPages.

Enumerator:

| phaseSkip | | | --------------- | | | phaseInitiate | | | phaseWait | |

Definition at line 314 of file CacheImpl.h.


Constructor & Destructor Documentation


Member Function Documentation

template<class PageT, class VictimPolicyT>

PageT * CacheImpl< PageT, VictimPolicyT >::lookupPage ( PageBucketT & bucket,
BlockId blockId,
bool pin
) [private]

Finds a page by BlockId within a particular bucket.

If found, waits for any pending read and then increments the page reference count; also notifies victimPolicy of the page access.

Parameters:

bucket the bucket to search; must be the same as getHashBucket(blockId)
blockId the BlockId of the page to look for
pin if true, the page will be pinned in the cache

Returns:

the page found, or NULL if not present

Definition at line 997 of file CacheMethodsImpl.h.

References CachePage::DATA_READ, PageBucket< PageT >::mutex, and PageBucket< PageT >::pageList.

00998 { 00999 assertCorrectBucket(bucket,blockId); 01000 SXMutexSharedGuard bucketGuard(bucket.mutex); 01001 for (PageBucketIter iter(bucket.pageList); iter; ++iter) { 01002 StrictMutexGuard pageGuard(iter->mutex); 01003 if (iter->getBlockId() == blockId) { 01004 victimPolicy.notifyPageAccess(*iter, pin); 01005 iter->nReferences++; 01006 while (iter->dataStatus == CachePage::DATA_READ) { 01007 iter->waitForPendingIO(pageGuard); 01008 } 01009 return iter; 01010 } 01011 } 01012 return NULL; 01013 }

template<class PageT, class VictimPolicyT>

| PageT * CacheImpl< PageT, VictimPolicyT >::findFreePage | ( | | ) | [private] | | ------------------------------------------------------------------------------- | - | | - | ----------- |

template<class PageT, class VictimPolicyT>

| void CacheImpl< PageT, VictimPolicyT >::flushSomePages | ( | | ) | [private] | | ----------------------------------------------------------------------------- | - | | - | ----------- |

Initiates asynchronous writes for a few dirty pages which are the best victimization candidates.

Definition at line 1155 of file CacheMethodsImpl.h.

References LOCKMODE_S, and LOCKMODE_S_NOWAIT.

01156 { 01157
01158 uint nToFlush = std::min(5,nDirtyPages); 01159 if (!nToFlush) { 01160
01161 return; 01162 } 01163 01164
01165 if (inFlushMode) { 01166 if (nDirtyPages < dirtyHighWaterMark) { 01167 return; 01168 } 01169 inFlushMode = true; 01170 } 01171 if (nDirtyPages < dirtyLowWaterMark) { 01172 inFlushMode = false; 01173 return; 01174 } 01175 01176 incrementStatsCounter(nLazyWriteCalls); 01177 uint nFlushed = 0; 01178 VictimSharedGuard victimSharedGuard(victimPolicy.getMutex()); 01179 std::pair<DirtyVictimPageIterator,DirtyVictimPageIterator> victimRange( 01180 victimPolicy.getDirtyVictimRange()); 01181 for (; victimRange.first != victimRange.second; ++(victimRange.first)) { 01182 PageT &page = *(victimRange.first); 01183
01184 StrictMutexGuard pageGuard(page.mutex, boost::try_to_lock); 01185 if (!pageGuard.owns_lock()) { 01186 continue; 01187 } 01188 if (!page.isDirty()) { 01189 continue; 01190 } 01191 if (page.isScratchLocked()) { 01192
01193 continue; 01194 } 01195 if (!page.lock.waitFor(LOCKMODE_S_NOWAIT)) { 01196
01197 continue; 01198 } else { 01199
01200 page.lock.release(LOCKMODE_S); 01201 } 01202 if (page.pMappedPageListener && 01203 !page.pMappedPageListener->canFlushPage(page)) 01204 { 01205 continue; 01206 } 01207 incrementStatsCounter(nLazyWrites); 01208
01209
01210 if (writePageAsync(page)) { 01211 break; 01212 } 01213 nFlushed++; 01214 if (nFlushed >= nToFlush) { 01215 break; 01216 } 01217 } 01218 }

template<class PageT, class VictimPolicyT>

bool CacheImpl< PageT, VictimPolicyT >::transferPageAsync ( PageT & page ) [private]

template<class PageT, class VictimPolicyT>

bool CacheImpl< PageT, VictimPolicyT >::readPageAsync ( PageT & page ) [inline, private]

template<class PageT, class VictimPolicyT>

bool CacheImpl< PageT, VictimPolicyT >::writePageAsync ( PageT & page ) [inline, private]

Writes the given page asynchronously.

Parameters:

Returns:

true if the asynch page request did not require retry

Definition at line 1356 of file CacheMethodsImpl.h.

References CachePage::DATA_WRITE.

01357 { 01358 assert(page.isDirty()); 01359 if (page.pMappedPageListener) { 01360 assert(page.pMappedPageListener->canFlushPage(page)); 01361 page.pMappedPageListener->notifyBeforePageFlush(page); 01362 } 01363 page.dataStatus = CachePage::DATA_WRITE; 01364 incrementStatsCounter(nPageWrites); 01365 if (transferPageAsync(page)) { 01366 return false; 01367 } else { 01368 return true; 01369 } 01370 }

template<class PageT, class VictimPolicyT>

FileSize CacheImpl< PageT, VictimPolicyT >::getPageOffset ( BlockId const & blockId ) [inline, private]

Translates a BlockId into the byte offset of the corresponding device block.

Parameters:

blockId the BlockId to translate

Returns:

byte offset within device

Definition at line 1374 of file CacheMethodsImpl.h.

References CompoundId::getBlockNum().

template<class PageT, class VictimPolicyT>

PageBucket< PageT > & CacheImpl< PageT, VictimPolicyT >::getHashBucket ( BlockId const & blockId ) [inline, private]

Gets the hash bucket containing the given BlockId.

Parameters:

blockId the BlockId being sought

Returns:

the containing bucket

Definition at line 1382 of file CacheMethodsImpl.h.

01383 { 01384 std::hash hasher; 01385 size_t hashCode = hasher(blockId); 01386 return *(pageTable[hashCode % pageTable.size()]); 01387 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::assertCorrectBucket ( PageBucketT & bucket,
BlockId const & blockId
) [inline, private]

Verifies the match between a PageBucket and BlockId.

Some methods (e.g. lockPage) precompute the correct bucket for a BlockId parameter and then make calls to helper methods (e.g. lookupPage, mapPage) which can skip the bucket lookup. This assertion allows the helper methods to verify the calling logic.

Definition at line 1391 of file CacheMethodsImpl.h.

01392 { 01393 assert(&bucket == &(getHashBucket(blockId))); 01394 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::unmapPage ( PageT & page,
StrictMutexGuard & guard,
bool discard
) [private]

Unmaps a currently mapped page, but does not add it to the free list.

Also notifies victimPolicy of the unmapping. The page must have no outstanding references.

Parameters:

page a currently mapped page to be unmapped
guard a guard on page.mutex, which must already be held by the calling thread when this method is invoked, and which will be released when the method returns
discard if true, the page is being discarded from the cache

Definition at line 1222 of file CacheMethodsImpl.h.

References CachePage::DATA_INVALID, PageBucket< PageT >::mutex, NULL_BLOCK_ID, and PageBucket< PageT >::pageList.

01223 { 01224 assert(!page.nReferences); 01225 assert(pageGuard.owns_lock()); 01226 01227 victimPolicy.notifyPageUnmap(page, discard); 01228 if (page.pMappedPageListener) { 01229 page.pMappedPageListener->notifyPageUnmap(page); 01230 page.pMappedPageListener = NULL; 01231 } 01232 if (page.isDirty()) { 01233 decrementCounter(nDirtyPages); 01234 } 01235 01236
01237
01238
01239
01240 BlockId blockId = page.getBlockId(); 01241 page.blockId = NULL_BLOCK_ID; 01242 page.dataStatus = CachePage::DATA_INVALID; 01243 pageGuard.unlock(); 01244 01245 PageBucketT &bucket = getHashBucket(blockId); 01246 SXMutexExclusiveGuard bucketGuard(bucket.mutex); 01247 bool bFound = bucket.pageList.remove(page); 01248 assert(bFound); 01249 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::unmapAndFreeDiscardedPage ( PageT & page,
StrictMutexGuard & guard
) [private]

Unmaps a page being discarded and adds it to unmappedBucket.

Also notifies victimPolicy of the unmapping. If the page is dirty, it is not flushed. However, any pending I/O is allowed to complete before discard.

Parameters:

page a currently mapped page to be unmapped
guard a guard on page.mutex, which must already be held by the calling thread when this method is invoked, and which will be released and reacquired during the method's execution

Definition at line 1253 of file CacheMethodsImpl.h.

01254 { 01255 while (page.isTransferInProgress()) { 01256 page.waitForPendingIO(pageGuard); 01257 } 01258 unmapPage(page,pageGuard,true); 01259 pageGuard.lock(); 01260 assert(!page.nReferences); 01261 freePage(page); 01262 }

template<class PageT, class VictimPolicyT>

PageT & CacheImpl< PageT, VictimPolicyT >::mapPage ( PageBucketT & bucket,
PageT & newPage,
BlockId blockId,
MappedPageListener * pMappedPageListener,
bool bPendingRead = true,
bool bIncRef = true
) [private]

Maps a page if it is not already mapped (and notifies victimPolicy of the page mapping); otherwise, finds the existing mapping (and notifies victimPolicy of the page access).

Parameters:

bucket the bucket to contain the Page; must be the same as getHashBucket(blockId)
newPage a free page to map (previously obtained from findFreePage); if mapPage finds an existing mapping, newPage is automatically freed
blockId the BlockId to be mapped
pMappedPageListener the MappedPageListener to be associated with the mapped page
bPendingRead true if a newly mapped page should be marked for a pending read (this is ignored if the page was already mapped)
bIncRef true if the returned page should have its reference count incremented as a side effect

Returns:

newPage if no mapping already exists for blockId; otherwise, the existing mapped page

Definition at line 1266 of file CacheMethodsImpl.h.

References CachePage::DATA_READ, CompoundId::getDeviceId(), PageBucket< PageT >::mutex, MappedPageListener::notifyPageMap(), PageBucket< PageT >::pageList, and SXMutexGuard< lockMode >::unlock().

01270 { 01271 assert(!page.hasBlockId()); 01272 assert(!page.isDirty()); 01273 assert(getDevice(CompoundId::getDeviceId(blockId)).get()); 01274 assertCorrectBucket(bucket,blockId); 01275 01276
01277
01278 SXMutexExclusiveGuard bucketGuard(bucket.mutex); 01279 for (PageBucketIter iter(bucket.pageList); iter; ++iter) { 01280 StrictMutexGuard pageGuard(iter->mutex); 01281 if (iter->getBlockId() == blockId) { 01282
01283 freePage(page); 01284 if (bIncRef) { 01285 iter->nReferences++; 01286 } 01287 bucketGuard.unlock(); 01288 assert(pMappedPageListener == iter->pMappedPageListener); 01289 victimPolicy.notifyPageAccess(*iter, bIncRef); 01290 return *iter; 01291 } 01292 } 01293 01294
01295 StrictMutexGuard pageGuard(page.mutex); 01296 page.blockId = blockId; 01297 assert(!page.pMappedPageListener); 01298 page.pMappedPageListener = pMappedPageListener; 01299 if (bIncRef) { 01300 page.nReferences++; 01301 } 01302 if (bPendingRead) { 01303 page.dataStatus = CachePage::DATA_READ; 01304 } 01305 bucket.pageList.push_back(page); 01306 bucketGuard.unlock(); 01307 victimPolicy.notifyPageMap(page, bIncRef); 01308 if (pMappedPageListener) { 01309 pMappedPageListener->notifyPageMap(page); 01310 } 01311 return page; 01312 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::freePage ( PageT & page ) [inline, private]

template<class PageT, class VictimPolicyT>

bool CacheImpl< PageT, VictimPolicyT >::canVictimizePage ( PageT & page ) [inline, private]

Decides whether a page can be victimized.

The page must be mapped, have no references, and no pending I/O.

Parameters:

page the page to test; the page's mutex must already be held by the calling thread

Returns:

true if the page can be victimized

Definition at line 1406 of file CacheMethodsImpl.h.

01407 { 01408
01409
01410
01411
01412 return page.hasBlockId() 01413 && !page.nReferences 01414 && !page.isTransferInProgress(); 01415 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::incrementCounter ( AtomicCounter & x ) [inline, private]

Increments a counter variable safely.

Parameters:

x reference to counter to be updated

Definition at line 1419 of file CacheMethodsImpl.h.

01420 { 01421 ++x; 01422 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::decrementCounter ( AtomicCounter & x ) [inline, private]

Decrements a counter variable safely.

Parameters:

x reference to counter to be updated

Definition at line 1426 of file CacheMethodsImpl.h.

01427 { 01428 --x; 01429 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::incrementStatsCounter ( AtomicCounter & x ) [inline, private]

Increments a statistical counter.

Can be defined to NOP to increase cache performance if statistics aren't important.

Parameters:

x reference to counter to be updated

Definition at line 1433 of file CacheMethodsImpl.h.

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::decrementStatsCounter ( AtomicCounter & x ) [inline, private]

Decrements a statistical counter.

Can be defined to NOP to increase cache performance if statistics aren't important.

Parameters:

x reference to counter to be updated

Definition at line 1440 of file CacheMethodsImpl.h.

template<class PageT, class VictimPolicyT>

| void CacheImpl< PageT, VictimPolicyT >::initializeStats | ( | | ) | [private] | | ------------------------------------------------------------------------------ | - | | - | ----------- |

Clears stats which are tracked since initialization.

Definition at line 114 of file CacheMethodsImpl.h.

References CacheStats::nCheckpointWrites, CacheStats::nCheckpointWritesSinceInit, CacheStats::nDirtyPages, CacheStats::nHits, CacheStats::nHitsSinceInit, CacheStats::nIoRetries, CacheStats::nIoRetriesSinceInit, CacheStats::nLazyWriteCalls, CacheStats::nLazyWriteCallsSinceInit, CacheStats::nLazyWrites, CacheStats::nLazyWritesSinceInit, CacheStats::nMemPagesAllocated, CacheStats::nMemPagesMax, CacheStats::nMemPagesUnused, CacheStats::nPageReads, CacheStats::nPageReadsSinceInit, CacheStats::nPageWrites, CacheStats::nPageWritesSinceInit, CacheStats::nRejectedPrefetches, CacheStats::nRejectedPrefetchesSinceInit, CacheStats::nRequests, CacheStats::nRequestsSinceInit, CacheStats::nSuccessfulPrefetches, CacheStats::nSuccessfulPrefetchesSinceInit, CacheStats::nVictimizations, CacheStats::nVictimizationsSinceInit, CacheStats::nVictimizationWrites, CacheStats::nVictimizationWritesSinceInit, and CacheImpl< PageT, VictimPolicyT >::statsSinceInit.

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::allocatePages ( CacheParams const & params ) [private]

Handles initial allocation of pages and attempts to handle any associated out-of-memory errors.

Definition at line 148 of file CacheMethodsImpl.h.

References CacheAllocator::allocate(), CacheImpl< PageT, VictimPolicyT >::bufferAllocator, CacheImpl< PageT, VictimPolicyT >::calcDirtyThreshholds(), ClosableObject::close(), CacheAllocator::deallocate(), CacheParams::defaultMemPagesInit, CacheParams::defaultMemPagesMax, deleteAndNullify(), min(), CacheParams::nMemPagesInit, CacheParams::nMemPagesMax, PageBucket< PageT >::pageList, CacheImpl< PageT, VictimPolicyT >::pages, CacheImpl< PageT, VictimPolicyT >::unallocatedBucket, CacheImpl< PageT, VictimPolicyT >::unmappedBucket, and CacheImpl< PageT, VictimPolicyT >::victimPolicy.

00149 { 00150 static const int allocErrorMsgSize = 255; 00151 uint nPagesMax = 0; 00152 uint nPagesInit = 0; 00153 00154
00155
00156 for (int attempts = 0; attempts < 2; attempts++) { 00157 bool allocError = false; 00158 int allocErrorCode = 0; 00159 char allocErrorMsg[allocErrorMsgSize + 1] = { 0 }; 00160 00161 nPagesMax = params.nMemPagesMax; 00162 nPagesInit = params.nMemPagesInit; 00163 00164 try { 00165 if (attempts != 0) { 00166 nPagesMax = CacheParams::defaultMemPagesMax; 00167 nPagesInit = CacheParams::defaultMemPagesInit; 00168 } 00169 00170 pages.clear(); 00171 if (pages.capacity() > nPagesMax) { 00172
00173
00174
00175 std::vector<PageT *>(1).swap(pages); 00176 } 00177 pages.reserve(nPagesMax); 00178 pages.assign(nPagesMax, NULL); 00179 00180
00181 for (uint i = 0; i < nPagesMax; i++) { 00182 PBuffer pBuffer = NULL; 00183 if (i < nPagesInit) { 00184 pBuffer = static_cast( 00185 bufferAllocator.allocate(&allocErrorCode)); 00186 if (pBuffer == NULL) { 00187 allocError = true; 00188 strncpy( 00189 allocErrorMsg, "mmap failed", allocErrorMsgSize); 00190 break; 00191 } 00192 } 00193 PageT &page = *new PageT(*this,pBuffer); 00194 pages[i] = &page; 00195 } 00196 } catch (std::exception &excn) { 00197 allocError = true; 00198 allocErrorCode = 0; 00199 if (dynamic_cast<std::bad_alloc *>(&excn) != NULL) { 00200 strncpy(allocErrorMsg, "malloc failed", allocErrorMsgSize); 00201 } else { 00202 strncpy(allocErrorMsg, excn.what(), allocErrorMsgSize); 00203 } 00204 } 00205 00206 if (!allocError) { 00207
00208 break; 00209 } 00210 00211
00212 for (uint i = 0; i < pages.size(); i++) { 00213 if (pages[i]) { 00214 break; 00215 } 00216 PBuffer pBuffer = pages[i]->pBuffer; 00217 deleteAndNullify(pages[i]); 00218 if (pBuffer) { 00219
00220
00221
00222
00223
00224 bufferAllocator.deallocate(pBuffer); 00225 } 00226 } 00227 00228 if (attempts != 0) { 00229
00230 close(); 00231 throw SysCallExcn(std::string(allocErrorMsg), allocErrorCode); 00232 } 00233 } 00234 00235
00236
00237
00238 for (uint i = 0; i < pages.size(); i++) { 00239 PageT *page = pages[i]; 00240 PBuffer pBuffer = page->pBuffer; 00241 if (pBuffer) { 00242 unmappedBucket.pageList.push_back(*page); 00243 victimPolicy.registerPage(*page); 00244 } else { 00245 unallocatedBucket.pageList.push_back(*page); 00246 } 00247 } 00248 00249 uint nPages = std::min(nPagesInit, nPagesMax); 00250 calcDirtyThreshholds(nPages); 00251 victimPolicy.setAllocatedPageCount(nPages); 00252 }

template<class PageT, class VictimPolicyT>

| void CacheImpl< PageT, VictimPolicyT >::successfulPrefetch | ( | | ) | [private] | | --------------------------------------------------------------------------------- | - | | - | ----------- |

Updates counters indicating a successful pre-fetch has occurred.

Definition at line 677 of file CacheMethodsImpl.h.

template<class PageT, class VictimPolicyT>

| void CacheImpl< PageT, VictimPolicyT >::rejectedPrefetch | ( | | ) | [private] | | ------------------------------------------------------------------------------- | - | | - | ----------- |

template<class PageT, class VictimPolicyT>

| void CacheImpl< PageT, VictimPolicyT >::ioRetry | ( | | ) | [private] | | ---------------------------------------------------------------------- | - | | - | ----------- |

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::calcDirtyThreshholds ( uint nCachePages ) [private]

Calculates the number of dirty pages corresponding to the high and low water marks.

Parameters:

nCachePages number of cache pages

Definition at line 255 of file CacheMethodsImpl.h.

References CacheImpl< PageT, VictimPolicyT >::dirtyHighWaterMark, CacheImpl< PageT, VictimPolicyT >::dirtyHighWaterPercent, CacheImpl< PageT, VictimPolicyT >::dirtyLowWaterMark, and CacheImpl< PageT, VictimPolicyT >::dirtyLowWaterPercent.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), and CacheImpl< PageT, VictimPolicyT >::setAllocatedPageCount().

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::markPageDirty ( CachePage & page ) [private, virtual]

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::notifyTransferCompletion ( CachePage & ,
bool
) [private, virtual]

Implements Cache.

Definition at line 858 of file CacheMethodsImpl.h.

References CachePage::DATA_CLEAN, CachePage::DATA_ERROR, CachePage::DATA_READ, CachePage::DATA_WRITE, CachePage::dataStatus, CachePage::getBlockId(), CachePage::ioCompletionCondition, CachePage::mutex, MappedPageListener::notifyAfterPageFlush(), MappedPageListener::notifyAfterPageRead(), opaqueToInt(), and CachePage::pMappedPageListener.

template<class PageT, class VictimPolicyT>

| uint CacheImpl< PageT, VictimPolicyT >::getTimerIntervalMillis | ( | | ) | [private, virtual] | | -------------------------------------------------------------------------------------------------------------------------------------------------- | - | | - | -------------------- |

Calculates the interval which should elapse before the next call to onTimerInterval.

This can be different each time. A return value of 0 will cause the TimerThread to cease calling back.

Implements TimerThreadClient.

Definition at line 929 of file CacheMethodsImpl.h.

template<class PageT, class VictimPolicyT>

| void CacheImpl< PageT, VictimPolicyT >::onTimerInterval | ( | | ) | [private, virtual] | | ------------------------------------------------------------------------------ | - | | - | -------------------- |

template<class PageT, class VictimPolicyT>

| void CacheImpl< PageT, VictimPolicyT >::closeImpl | ( | | ) | [protected, virtual] | | ------------------------------------------------------------------------ | - | | - | ---------------------- |

Must be implemented by derived class to release any resources.

Implements ClosableObject.

Definition at line 946 of file CacheMethodsImpl.h.

References CacheImpl< PageT, VictimPolicyT >::bufferAllocator, CacheAllocator::deallocate(), deleteAndNullify(), CacheImpl< PageT, VictimPolicyT >::deviceTable, CacheImpl< PageT, VictimPolicyT >::getDevice(), Thread::isStarted(), Cache::NULL_DEVICE_ID, PageBucket< PageT >::pageList, CacheImpl< PageT, VictimPolicyT >::pages, CacheImpl< PageT, VictimPolicyT >::pageTable, CacheImpl< PageT, VictimPolicyT >::pDeviceAccessScheduler, DeviceAccessScheduler::stop(), TimerThread::stop(), CacheImpl< PageT, VictimPolicyT >::timerThread, CacheImpl< PageT, VictimPolicyT >::unallocatedBucket, CacheImpl< PageT, VictimPolicyT >::unmappedBucket, CacheImpl< PageT, VictimPolicyT >::unregisterDevice(), and CacheImpl< PageT, VictimPolicyT >::victimPolicy.

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::setAllocatedPageCount ( uint nMemPages ) [virtual]

Resizes this cache.

If nMemPages is greater than the number of page buffers currently allocated, allocates more. If less, frees some (victimizing as necessary).

Parameters:

nMemPages desired buffer allocation count; must be less than the nMemPagesMax specified when the cache was created, and also less than the current clean page target

Implements Cache.

Definition at line 275 of file CacheMethodsImpl.h.

References CacheAllocator::allocate(), CacheImpl< PageT, VictimPolicyT >::bufferAllocator, CacheImpl< PageT, VictimPolicyT >::calcDirtyThreshholds(), CacheAllocator::deallocate(), IntrusiveListMutator< T, DerivedListNode >::detach(), CacheImpl< PageT, VictimPolicyT >::findFreePage(), CacheImpl< PageT, VictimPolicyT >::freePage(), PageBucket< PageT >::mutex, PageBucket< PageT >::pageList, CacheImpl< PageT, VictimPolicyT >::pages, CacheImpl< PageT, VictimPolicyT >::unallocatedBucket, and CacheImpl< PageT, VictimPolicyT >::victimPolicy.

00277 { 00278 assert(nMemPagesDesired <= pages.size()); 00279
00280
00281 SXMutexExclusiveGuard unallocatedBucketGuard(unallocatedBucket.mutex); 00282 uint nMemPages = 00283 pages.size() - unallocatedBucket.pageList.size(); 00284 if (nMemPages < nMemPagesDesired) { 00285
00286 00287
00288
00289 int nMemPagesToAllocate = nMemPagesDesired - nMemPages; 00290 std::vector buffers(nMemPagesToAllocate); 00291 00292 for (int i = 0; i < nMemPagesToAllocate; ++i) { 00293 int errorCode; 00294 PBuffer pBuffer = static_cast( 00295 bufferAllocator.allocate(&errorCode)); 00296 00297 if (pBuffer == NULL) { 00298
00299 for (int i = 0; i < nMemPagesToAllocate; i++) { 00300 if (buffers[i] == NULL) { 00301 break; 00302 } 00303 00304
00305
00306
00307
00308 bufferAllocator.deallocate(buffers[i]); 00309 } 00310 buffers.clear(); 00311 std::vector(0).swap(buffers); 00312 00313 throw SysCallExcn("mmap failed", errorCode); 00314 } 00315 00316 buffers[i] = pBuffer; 00317 } 00318 00319 PageBucketMutator mutator(unallocatedBucket.pageList); 00320 for (int i = 0; i < nMemPagesToAllocate; i++) { 00321 PBuffer pBuffer = buffers[i]; 00322 PageT *page = mutator.detach(); 00323 assert(!page->pBuffer); 00324 page->pBuffer = pBuffer; 00325 victimPolicy.registerPage(*page); 00326
00327 freePage(*page); 00328 } 00329 } else { 00330
00331 for (; nMemPages > nMemPagesDesired; --nMemPages) { 00332 PageT *page; 00333 do { 00334 page = findFreePage(); 00335 } while (!page); 00336 00337 int errorCode; 00338 if (bufferAllocator.deallocate(page->pBuffer, &errorCode)) { 00339
00340
00341 freePage(*page); 00342 throw SysCallExcn("munmap failed", errorCode); 00343 } 00344 page->pBuffer = NULL; 00345 victimPolicy.unregisterPage(*page); 00346
00347 unallocatedBucket.pageList.push_back(*page); 00348 } 00349 } 00350 00351 calcDirtyThreshholds(nMemPagesDesired); 00352
00353 victimPolicy.setAllocatedPageCount(nMemPagesDesired); 00354 }

template<class PageT, class VictimPolicyT>

| uint CacheImpl< PageT, VictimPolicyT >::getAllocatedPageCount | ( | | ) | [virtual] | | ------------------------------------------------------------------------------------------------------------------------------------------------- | - | | - | ----------- |

template<class PageT, class VictimPolicyT>

| uint CacheImpl< PageT, VictimPolicyT >::getMaxAllocatedPageCount | ( | | ) | [virtual] | | ---------------------------------------------------------------------------------------------------------------------------------------------------- | - | | - | ----------- |

template<class PageT, class VictimPolicyT>

PageT * CacheImpl< PageT, VictimPolicyT >::lockPage ( BlockId blockId,
LockMode lockMode,
bool readIfUnmapped,
MappedPageListener * pMappedPageListener,
TxnId txnId
) [virtual]

Locks a page into memory with the specified concurrency mode.

When the page contents are no longer needed, the caller must invoke the unlockPage() method with the same concurrency mode to release it. If the desired page is already locked by another thread with an incompatible concurrency mode, blocks until the page becomes available (unless the lock mode is of the NoWait variety, in which case returns NULL immediately). Note that NoWait locking only applies to lock contention, not I/O, so if an unmapped page is locked in NoWait mode, blocks until the read completes.

The device referenced by the requested blockId must already be registered with the cache and must remain registered for the duration of the lock.

Notes on concurrency modes:

Parameters:

blockId the BlockId of the page to be locked
lockMode the desired concurrency mode
readIfUnmapped if true (the default) the page data is read as part of mapping; if false, the page data is left invalid until first write (used when allocating a new block with invalid contents)
pMappedPageListener optional listener to receive notifications when this page is written; if specified, it must match all prior and subsequent lock requests for the same page mapping
txnId optional TxnId to associate with lock; default is IMPLICIT_TXN_ID, which uses current thread ID as an implicit TxnId

Returns:

the locked CachePage, or NULL if a NoWait attempt failed

Implements CacheAccessor.

Definition at line 358 of file CacheMethodsImpl.h.

References CachePage::DATA_READ, CachePage::DATA_WRITE, ETERNITY, CompoundId::getDeviceId(), LOCKMODE_S_NOWAIT, LOCKMODE_X, LOCKMODE_X_NOWAIT, and NULL_BLOCK_ID.

00361 { 00362
00363 00364 assert(blockId != NULL_BLOCK_ID); 00365 assert(CompoundId::getDeviceId(blockId) != NULL_DEVICE_ID); 00366 PageBucketT &bucket = getHashBucket(blockId); 00367 PageT *page = lookupPage(bucket,blockId,true); 00368 if (page) { 00369 assert(page->pMappedPageListener == pMappedPageListener); 00370
00371
00372 incrementStatsCounter(nCacheHits); 00373 } else { 00374 do { 00375 page = findFreePage(); 00376 } while (!page); 00377 00378
00379
00380
00381 00382 PageT &mappedPage = mapPage( 00383 bucket,*page,blockId,pMappedPageListener,readIfUnmapped); 00384 if (&mappedPage == page) { 00385
00386
00387 if (readIfUnmapped) { 00388 readPageAsync(*page); 00389 } 00390 } else { 00391
00392
00393 page = &mappedPage; 00394 } 00395 if (readIfUnmapped) { 00396
00397
00398
00399 StrictMutexGuard pageGuard(page->mutex); 00400 while (page->dataStatus == CachePage::DATA_READ) { 00401 page->waitForPendingIO(pageGuard); 00402 } 00403 } 00404 } 00405 00406 incrementStatsCounter(nCacheRequests); 00407 00408
00409 00410 if (!page->lock.waitFor(lockMode,ETERNITY,txnId)) { 00411
00412 assert((lockMode == LOCKMODE_S_NOWAIT) || 00413 (lockMode == LOCKMODE_X_NOWAIT)); 00414 StrictMutexGuard pageGuard(page->mutex); 00415 page->nReferences--; 00416 if (!page->nReferences) { 00417 victimPolicy.notifyPageUnpin(*page); 00418 } 00419 return NULL; 00420 } 00421 if ((lockMode == LOCKMODE_X) || (lockMode == LOCKMODE_X_NOWAIT)) { 00422
00423
00424
00425 00426
00427 StrictMutexGuard pageGuard(page->mutex); 00428 while (page->dataStatus == CachePage::DATA_WRITE) { 00429 page->waitForPendingIO(pageGuard); 00430 } 00431 #ifdef DEBUG 00432 int errorCode; 00433 if (bufferAllocator.setProtection( 00434 page->pBuffer, cbPage, false, &errorCode)) 00435 { 00436 throw new SysCallExcn("memory protection failed", errorCode); 00437 } 00438 #endif 00439 } else { 00440
00441 #ifdef DEBUG 00442 StrictMutexGuard pageGuard(page->mutex); 00443 if (page->nReferences == 1) { 00444 int errorCode; 00445 if (bufferAllocator.setProtection( 00446 page->pBuffer, cbPage, true, &errorCode)) 00447 { 00448 throw new SysCallExcn("memory protection failed", errorCode); 00449 } 00450 } 00451 #endif 00452 } 00453 return page; 00454 }

template<class PageT, class VictimPolicyT>

PageT & CacheImpl< PageT, VictimPolicyT >::lockScratchPage ( BlockNum blockNum ) [virtual]

Allocates a free page buffer for scratch usage.

The returned page is considered to be locked in exclusive mode but not mapped to any device. To release the page, use unlock(LOCKMODE_X).

Although the page remains unmapped, its BlockId is set for the duration of the lock. The caller can supply a BlockNum, which need not be unique.

Parameters:

blockNum the block number to use when making up the Page's BlockId; the device ID will be NULL_DEVICE_ID

Returns:

the locked page

Implements Cache.

Definition at line 546 of file CacheMethodsImpl.h.

References CachePage::DATA_DIRTY, CompoundId::setBlockNum(), and CompoundId::setDeviceId().

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::discardPage ( BlockId blockId ) [virtual]

Flushes and/or unmaps selected pages.

Parameters:

pagePredicate caller-provided interface for deciding which pages should be checkpointed; the given PagePredicate will be called for each mapped page, and only those which satisfy the predicate will be affected by the checkpoint (note that the page mutex is held for the duration of the call, so implementations must take care to avoid deadlock)
checkpointType type of checkpoint to execute

Returns:

number of pages for which pagePredicate returned true

Implements Cache.

Definition at line 698 of file CacheMethodsImpl.h.

References CHECKPOINT_FLUSH_AND_UNMAP, and CachePage::DATA_WRITE.

00700 { 00701
00702
00703 00704 uint nPages = 0; 00705 bool countPages = true; 00706 00707 FlushPhase flushPhase; 00708 if (checkpointType >= CHECKPOINT_FLUSH_AND_UNMAP) { 00709 flushPhase = phaseInitiate; 00710 } else { 00711 flushPhase = phaseSkip; 00712 } 00713 for (;;) { 00714 for (uint i = 0; i < pages.size(); i++) { 00715 PageT &page = *(pages[i]); 00716 StrictMutexGuard pageGuard(page.mutex); 00717
00718 if (!page.hasBlockId()) { 00719 continue; 00720 } 00721 if (!pagePredicate(page)) { 00722 continue; 00723 } 00724 if (countPages) { 00725 ++nPages; 00726 } 00727 if (flushPhase == phaseInitiate) { 00728 if (page.isDirty()) { 00729
00730
00731 assert(!page.isExclusiveLockHeld()); 00732 incrementStatsCounter(nCheckpointWrites); 00733
00734 writePageAsync(page); 00735 } 00736 } else if (flushPhase == phaseWait) { 00737 BlockId origBlockId = page.getBlockId(); 00738 MappedPageListener *origListener = page.pMappedPageListener; 00739 while (page.dataStatus == CachePage::DATA_WRITE) { 00740 page.waitForPendingIO(pageGuard); 00741 } 00742 00743
00744
00745
00746
00747
00748
00749
00750
00751
00752
00753
00754 if (page.pMappedPageListener && 00755 page.pMappedPageListener == origListener && 00756 page.getBlockId() == origBlockId) 00757 { 00758 MappedPageListener *newListener = 00759 page.pMappedPageListener->notifyAfterPageCheckpointFlush( 00760 page); 00761 if (newListener != NULL) { 00762 page.pMappedPageListener = newListener; 00763 } 00764 } 00765 } else { 00766 if (checkpointType <= CHECKPOINT_FLUSH_AND_UNMAP) { 00767 unmapAndFreeDiscardedPage(page,pageGuard); 00768 } 00769 } 00770 } 00771 countPages = false; 00772 if (flushPhase == phaseInitiate) { 00773 flushPhase = phaseWait; 00774 continue; 00775 } 00776 if (flushPhase == phaseWait) { 00777 if (checkpointType <= CHECKPOINT_FLUSH_AND_UNMAP) { 00778 flushPhase = phaseSkip; 00779 continue; 00780 } 00781 } 00782 return nPages; 00783 } 00784 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::collectStats ( CacheStats & stats ) [virtual]

Gets a snapshot of cache activity; as a side-effect, clears cumulative performance counters.

Parameters:

stats receives the snapshot

Implements Cache.

Definition at line 1087 of file CacheMethodsImpl.h.

References CacheStats::nCheckpointWrites, CacheStats::nCheckpointWritesSinceInit, CacheStats::nDirtyPages, CacheStats::nHits, CacheStats::nHitsSinceInit, CacheStats::nIoRetries, CacheStats::nIoRetriesSinceInit, CacheStats::nLazyWriteCalls, CacheStats::nLazyWriteCallsSinceInit, CacheStats::nLazyWrites, CacheStats::nLazyWritesSinceInit, CacheStats::nMemPagesAllocated, CacheStats::nMemPagesMax, CacheStats::nMemPagesUnused, CacheStats::nPageReads, CacheStats::nPageReadsSinceInit, CacheStats::nPageWrites, CacheStats::nPageWritesSinceInit, CacheStats::nRejectedPrefetches, CacheStats::nRejectedPrefetchesSinceInit, CacheStats::nRequests, CacheStats::nRequestsSinceInit, CacheStats::nSuccessfulPrefetches, CacheStats::nSuccessfulPrefetchesSinceInit, CacheStats::nVictimizations, CacheStats::nVictimizationsSinceInit, CacheStats::nVictimizationWrites, and CacheStats::nVictimizationWritesSinceInit.

Registers the given device with the Cache; must be called exactly once before any other caching operations can be requested for pages of this device.

Parameters:

deviceId the ID of the device to be registered
pDevice the device to be registered

Implements Cache.

Definition at line 825 of file CacheMethodsImpl.h.

References opaqueToInt().

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::unregisterDevice ( DeviceId deviceId ) [virtual]

template<class PageT, class VictimPolicyT>

bool CacheImpl< PageT, VictimPolicyT >::prefetchPage ( BlockId blockId,
MappedPageListener * pMappedPageListener
) [virtual]

Hints that a page should be prefetched in preparation for a future lock request.

Parameters:

blockId the BlockId of the page to be prefetched
pMappedPageListener optional listener to receive notifications when this page is written; if specified, it must match all prior and subsequent lock requests for the same page mapping

Returns:

true if the pre-fetch request was successful

Implements CacheAccessor.

Definition at line 568 of file CacheMethodsImpl.h.

References NULL_BLOCK_ID.

00569 { 00570 assert(blockId != NULL_BLOCK_ID); 00571 if (isPageMapped(blockId)) { 00572
00573
00574 successfulPrefetch(); 00575 return true; 00576 } 00577 PageT *page = findFreePage(); 00578 if (!page) { 00579
00580 rejectedPrefetch(); 00581 return false; 00582 } 00583 00584 PageBucketT &bucket = getHashBucket(blockId); 00585 bool bPendingRead = true; 00586
00587
00588
00589 bool bIncRef = false; 00590 PageT &mappedPage = mapPage( 00591 bucket,*page,blockId,pMappedPageListener,bPendingRead,bIncRef); 00592 if (&mappedPage == page) { 00593 if (readPageAsync(*page)) { 00594 successfulPrefetch(); 00595 } else { 00596 rejectedPrefetch(); 00597 return false; 00598 } 00599 } else { 00600
00601
00602 page = &mappedPage; 00603 } 00604 return true; 00605 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::prefetchBatch ( BlockId blockId,
uint nPages,
MappedPageListener * pMappedPageListener
) [virtual]

Hints that a contiguous run of pages should be prefetched.

Parameters:

blockId the BlockId of the first page to be prefetched; more will be prefetched depending on the configured batch size
nPages number of pages in batch
pMappedPageListener optional listener to receive notifications when this page is written; if specified, it must match all prior and subsequent lock requests for the same page mapping

Implements CacheAccessor.

Definition at line 609 of file CacheMethodsImpl.h.

References RandomAccessRequest::bindingList, RandomAccessRequest::cbOffset, RandomAccessRequest::cbTransfer, CompoundId::getDeviceId(), CompoundId::incBlockNum(), NULL_BLOCK_ID, RandomAccessRequest::pDevice, RandomAccessRequest::READ, DeviceAccessScheduler::schedule(), and RandomAccessRequest::type.

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::flushPage ( CachePage & page,
bool async
) [virtual]

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::unlockPage ( CachePage & page,
LockMode lockMode,
TxnId txnId
) [virtual]

Releases lock held on page.

Parameters:

page the page to be unlocked
lockMode must correspond to value passed to Cache::lockPage; however, for pages locked with NOWAIT, the equivalent unlock type should be normal (e.g. LOCKMODE_S instead of LOCKMODE_S_NOWAIT)
txnId must correspond to value passed to Cache::lockPage

Implements CacheAccessor.

Definition at line 458 of file CacheMethodsImpl.h.

References CachePage::DATA_INVALID, CompoundId::getDeviceId(), LOCKMODE_S_NOWAIT, and NULL_BLOCK_ID.

00460 { 00461 assert(lockMode < LOCKMODE_S_NOWAIT); 00462 PageT &page = static_cast<PageT &>(vPage); 00463 StrictMutexGuard pageGuard(page.mutex); 00464 assert(page.nReferences); 00465 bool bFree = false; 00466 assert(page.hasBlockId()); 00467 if (CompoundId::getDeviceId(page.getBlockId()) == NULL_DEVICE_ID) { 00468
00469 bFree = true; 00470 } else { 00471 int errorCode; 00472 if (bufferAllocator.setProtection( 00473 page.pBuffer, cbPage, false, &errorCode)) 00474 { 00475 throw new SysCallExcn("memory protection failed", errorCode); 00476 } 00477 00478 page.lock.release(lockMode,txnId); 00479 } 00480 page.nReferences--; 00481 if (!page.nReferences) { 00482 if (bFree) { 00483
00484
00485
00486 page.dataStatus = CachePage::DATA_INVALID; 00487 page.blockId = NULL_BLOCK_ID; 00488 freePage(page); 00489 } else { 00490
00491
00492 victimPolicy.notifyPageUnpin(page); 00493 } 00494 00495
00496
00497 freePageCondition.notify_all(); 00498 } 00499 }

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::nicePage ( CachePage & page ) [virtual]

Marks a page as nice, indicating that it is very unlikely the page's mapping will be needed again any time soon, so it is a good candidate for victimization.

Parameters:

page the page to be marked

Implements CacheAccessor.

Definition at line 818 of file CacheMethodsImpl.h.

00819 { 00820 victimPolicy.notifyPageNice(static_cast<PageT &>(page)); 00821 }

template<class PageT, class VictimPolicyT>

bool CacheImpl< PageT, VictimPolicyT >::isPageMapped ( BlockId blockId ) [virtual]

template<class PageT, class VictimPolicyT>

| CacheAllocator & CacheImpl< PageT, VictimPolicyT >::getAllocator | ( | | ) | const [virtual] | | ------------------------------------------------------------------------------------------------------------------- | - | | - | ----------------- |

template<class PageT, class VictimPolicyT>

void CacheImpl< PageT, VictimPolicyT >::getPrefetchParams ( uint & prefetchPagesMax,
uint & prefetchThrottleRate
) [virtual]

Retrieves the current pre-fetch caching parameters that determine how many pages should be pre-fetched and how often the pre-fetches should occur.

Parameters:

[out] prefetchPagesMax max number of outstanding pre-fetch pages
[out] prefetchThrottleRate the number of successful pre-fetches that have to occur before the pre-fetch rate is throttled back up, in the event that it has been throttled down due to rejected requests

Implements CacheAccessor.

Definition at line 105 of file CacheMethodsImpl.h.

| uint Cache::getPageSize | ( | | ) | const [inline, inherited] | | ------------------------------------------------------------------------------------ | - | | - | --------------------------- |

| uint Cache::getMaxLockedPages | ( | | ) | [virtual, inherited] | | ------------------------------------------------------------------------------------------ | - | | - | ---------------------- |

void Cache::setMaxLockedPages ( uint nPages ) [virtual, inherited]

Sets the page lock quota on this accessor.

Ignored for accessor implementations that don't support quotas.

Parameters:

Implements CacheAccessor.

Definition at line 74 of file Cache.cpp.

void Cache::setTxnId ( TxnId txnId ) [virtual, inherited]

Sets a default TxnId to use for locking pages (to be used when IMPLICIT_TXN_ID is specified).

Not all CacheAccessor implementations support this behavior.

Parameters:

Implements CacheAccessor.

Definition at line 78 of file Cache.cpp.

| TxnId Cache::getTxnId | ( | | ) | const [virtual, inherited] | | --------------------- | - | | - | ---------------------------- |

void Cache::writeStats ( StatsTarget & target ) [virtual, inherited]

Writes a current stats snapshot to a StatsTarget.

Parameters:

target receives the stats

Implements StatsSource.

Definition at line 87 of file Cache.cpp.

References Cache::collectStats(), CacheStats::nCheckpointWrites, CacheStats::nCheckpointWritesSinceInit, CacheStats::nDirtyPages, CacheStats::nHits, CacheStats::nHitsSinceInit, CacheStats::nIoRetries, CacheStats::nIoRetriesSinceInit, CacheStats::nLazyWriteCalls, CacheStats::nLazyWriteCallsSinceInit, CacheStats::nLazyWrites, CacheStats::nLazyWritesSinceInit, CacheStats::nMemPagesAllocated, CacheStats::nMemPagesMax, CacheStats::nMemPagesUnused, CacheStats::nPageReads, CacheStats::nPageReadsSinceInit, CacheStats::nPageWrites, CacheStats::nPageWritesSinceInit, CacheStats::nRejectedPrefetches, CacheStats::nRejectedPrefetchesSinceInit, CacheStats::nRequests, CacheStats::nRequestsSinceInit, CacheStats::nSuccessfulPrefetches, CacheStats::nSuccessfulPrefetchesSinceInit, CacheStats::nVictimizations, CacheStats::nVictimizationsSinceInit, CacheStats::nVictimizationWrites, CacheStats::nVictimizationWritesSinceInit, and StatsTarget::writeCounter().

00088 { 00089 CacheStats stats; 00090 collectStats(stats); 00091 target.writeCounter( 00092 "CacheHits", stats.nHits); 00093 target.writeCounter( 00094 "CacheHitsSinceInit", stats.nHitsSinceInit); 00095 target.writeCounter( 00096 "CacheRequests", stats.nRequests); 00097 target.writeCounter( 00098 "CacheRequestsSinceInit", stats.nRequestsSinceInit); 00099 target.writeCounter( 00100 "CacheVictimizations", stats.nVictimizations); 00101 target.writeCounter( 00102 "CacheVictimizationsSinceInit", stats.nVictimizationsSinceInit); 00103 target.writeCounter( 00104 "CacheDirtyPages", stats.nDirtyPages); 00105 target.writeCounter( 00106 "CachePagesRead", stats.nPageReads); 00107 target.writeCounter( 00108 "CachePagesReadSinceInit", stats.nPageReadsSinceInit); 00109 target.writeCounter( 00110 "CachePagesWritten", stats.nPageWrites); 00111 target.writeCounter( 00112 "CachePagesWrittenSinceInit", stats.nPageWritesSinceInit); 00113 target.writeCounter( 00114 "CachePagePrefetchesRejected", stats.nRejectedPrefetches); 00115 target.writeCounter( 00116 "CachePagePrefetchesRejectedSinceInit", 00117 stats.nRejectedPrefetchesSinceInit); 00118 target.writeCounter( 00119 "CachePageIoRetries", stats.nIoRetries); 00120 target.writeCounter( 00121 "CachePageIoRetriesSinceInit", 00122 stats.nIoRetriesSinceInit); 00123 target.writeCounter( 00124 "CachePagesPrefetched", stats.nSuccessfulPrefetches); 00125 target.writeCounter( 00126 "CachePagesPrefetchedSinceInit", 00127 stats.nSuccessfulPrefetchesSinceInit); 00128 target.writeCounter("CacheLazyWrites", stats.nLazyWrites); 00129 target.writeCounter("CacheLazyWritesSinceInit", stats.nLazyWritesSinceInit); 00130 target.writeCounter("CacheLazyWriteCalls", stats.nLazyWriteCalls); 00131 target.writeCounter( 00132 "CacheLazyWriteCallsSinceInit", 00133 stats.nLazyWriteCallsSinceInit); 00134 target.writeCounter("CacheVictimizationWrites", stats.nVictimizationWrites); 00135 target.writeCounter( 00136 "CacheVictimizationWritesSinceInit", 00137 stats.nVictimizationWritesSinceInit); 00138 target.writeCounter("CacheCheckpointWrites", stats.nCheckpointWrites); 00139 target.writeCounter( 00140 "CacheCheckpointWritesSinceInit", 00141 stats.nCheckpointWritesSinceInit); 00142 target.writeCounter( 00143 "CachePagesAllocated", stats.nMemPagesAllocated); 00144 target.writeCounter( 00145 "CachePagesUnused", stats.nMemPagesUnused); 00146 target.writeCounter( 00147 "CachePagesAllocationLimit", stats.nMemPagesMax); 00148 }

| bool ClosableObject::isClosed | ( | | ) | const [inline, inherited] | | ----------------------------- | - | | - | --------------------------- |

Returns:

whether the object has been closed

Definition at line 58 of file ClosableObject.h.

| void ClosableObject::close | ( | | ) | [inherited] | | -------------------------- | - | | - | ------------- |

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

| void ThreadTracker::onThreadStart | ( | | ) | [virtual, inherited] | | --------------------------------- | - | | - | ---------------------- |

| void ThreadTracker::onThreadEnd | ( | | ) | [virtual, inherited] | | ------------------------------- | - | | - | ---------------------- |

FennelExcn * ThreadTracker::cloneExcn ( std::exception & ex ) [virtual, inherited]

Member Data Documentation

template<class PageT, class VictimPolicyT>

Collection of registered devices indexed by DeviceId; this array is of fixed size, with a NULL slot indicating that the given device ID has not been registered; this permits synchronization-free access to the collection.

Definition at line 128 of file CacheImpl.h.

Referenced by CacheImpl< PageT, VictimPolicyT >::closeImpl().

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

Used by the lazy writer thread to indicate that the high-water dirty threshhold has been reached and page flushes should continue until the low-water threshhold is reached.

Definition at line 175 of file CacheImpl.h.

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

See CacheStats::nDirtyPages.

This is actually used for more than just statistics; the idle flush thread uses this in determining its activity.

Definition at line 197 of file CacheImpl.h.

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

Mutex coupled with freePageCondition.

Definition at line 253 of file CacheImpl.h.

template<class PageT, class VictimPolicyT>

Condition variable used for notification of free page availability.

Definition at line 258 of file CacheImpl.h.

template<class PageT, class VictimPolicyT>

std::vector<PageT *> CacheImpl< PageT, VictimPolicyT >::pages [private]

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

Set only if bufferAllocator is owned by this cache.

Definition at line 280 of file CacheImpl.h.

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

template<class PageT, class VictimPolicyT>

Maximum number of outstanding pre-fetch requests.

Definition at line 302 of file CacheImpl.h.

template<class PageT, class VictimPolicyT>

The number of successful pre-fetches that have to occur before the pre-fetch rate is throttled back up, in the event that it has been throttled down due to rejected requests.

Definition at line 309 of file CacheImpl.h.


The documentation for this class was generated from the following files:


Generated on Mon Jun 22 04:00:27 2009 for Fennel by doxygen 1.5.1