Fennel: ExternalSortRunLoader Class Reference (original) (raw)

ExternalSortRunLoader manages the state of a run as it is being loaded and sorted in memory. More...

#include <[ExternalSortRunLoader.h](ExternalSortRunLoader%5F8h-source.html)>

Inheritance diagram for ExternalSortRunLoader:

List of all members.

Public Member Functions
ExternalSortRunLoader (ExternalSortInfo &info)
virtual ~ExternalSortRunLoader ()
void startRun ()
Prepares this loader to begin a new run.
bool isStarted ()
**Returns:**whether this loader has been started and not yet fetched
ExternalSortRC loadRun (ExecStreamBufAccessor &bufAccessor)
Loads data from buffer into a run.
void sort ()
Sorts loaded run.
uint getLoadedTupleCount ()
**Returns:**number of tuples loaded so far in current run
void releaseResources ()
Releases any resources acquired by this loader.
virtual ExternalSortFetchArray & bindFetchArray ()
Binds the fetch array which will be used implicitly by subsequent calls to fetch().
virtual ExternalSortRC fetch (uint nTuplesRequested)
Fetches tuples via the previously bound fetch array.
Public Attributes
bool runningParallelTask
Flag used only during parallel sort.
Private Member Functions
PBuffer allocateBuffer ()
Allocates one page buffer.
bool allocateDataBuffer ()
Allocates one data buffer.
bool allocateIndexBuffer ()
Allocates one index buffer.
PBuffer & getPointerArrayEntry (uint iTuple)
Dereferences a pointer array entry during quicksort.
void quickSortSwap (uint l, uint r)
uint quickSortPartition (uint i, uint j, PBuffer pivot)
PBuffer quickSortFindPivot (uint l, uint r)
void quickSort (uint l, uint r)
Private Attributes
ExternalSortInfo & sortInfo
Global information.
SegPageLock bufferLock
Cache page lock used to allocate pages of scratch memory.
uint nMemPagesMax
Maximum number of memory pages which can be allocated by this loader.
std::vector< PBuffer > freeBuffers
Array of allocated page buffers which have been recycled but not yet reused.
std::vector< PBuffer > indexBuffers
Array of page buffers which have been allocated as index buffers.
std::vector< PBuffer > dataBuffers
Array of page buffers which have been allocated as data buffers.
uint indexToPageShift
Precomputed bit rightshift count for converting a tuple index to a page number (0-based position within indexBuffers array).
uint indexPageMask
Precomputed bitmask for converting a tuple index to a 0-based position in pointer array on containing index page.
PBuffer pDataBuffer
Pointer to first free byte in current data buffer.
PBuffer pDataBufferEnd
Pointer to end of current data buffer.
PBuffer pIndexBuffer
Pointer to first free byte in current index buffer.
PBuffer pIndexBufferEnd
Pointer to end of current index buffer.
uint nTuplesLoaded
Number of tuples loaded so far.
uint nTuplesFetched
Number of tuples fetched so far.
ExternalSortFetchArray fetchArray
Array used to return fetch results.
TupleAccessor tupleAccessor
TupleAccessor tupleAccessor2
TupleProjectionAccessor keyAccessor
TupleProjectionAccessor keyAccessor2
TupleData keyData
TupleData keyData2

Detailed Description

ExternalSortRunLoader manages the state of a run as it is being loaded and sorted in memory.

As unsorted tuple data is loaded, it is saved in discontiguous memory pages (called data buffers). In addition, byte pointers to the loaded tuple images are recorded in separately allocated memory pages (called index buffers, and also discontiguous). When the loader's memory page quota is exhausted, the run is considered full. Before a run is stored or returned, it is sorted in memory using quicksort. The quicksort operates on the index buffers (moving pointers only, not data). Each pointer array access during the quicksort requires an indirection computation (first find the right page, and then find the tuple on that page).

Definition at line 52 of file ExternalSortRunLoader.h.

Constructor & Destructor Documentation

ExternalSortRunLoader::ExternalSortRunLoader ( ExternalSortInfo & info ) [explicit]

Definition at line 31 of file ExternalSortRunLoader.cpp.

References SegPageLock::accessSegment(), TupleProjectionAccessor::bind(), bufferLock, ExternalSortInfo::cbPage, TupleData::compute(), TupleAccessor::compute(), indexPageMask, indexToPageShift, keyAccessor, keyAccessor2, keyData, keyData2, ExternalSortInfo::keyDesc, ExternalSortInfo::keyProj, ExternalSortInfo::memSegmentAccessor, nMemPagesMax, nTuplesFetched, nTuplesLoaded, pDataBuffer, pIndexBuffer, runningParallelTask, sortInfo, tupleAccessor, tupleAccessor2, and ExternalSortInfo::tupleDesc.

| ExternalSortRunLoader::~ExternalSortRunLoader | ( | | ) | [virtual] | | ---------------------------------------------- | - | | - | ----------- |

Member Function Documentation

| PBuffer ExternalSortRunLoader::allocateBuffer | ( | | ) | [private] | | ------------------------------------------------------------------------------------------------- | - | | - | ----------- |

Allocates one page buffer.

Returns:

the allocated buffer, or NULL if quota has been reached

Definition at line 110 of file ExternalSortRunLoader.cpp.

References SegPageLock::allocatePage(), bufferLock, dataBuffers, freeBuffers, SegPageLock::getPage(), CachePage::getWritableData(), indexBuffers, nMemPagesMax, and SegPageLock::unlock().

Referenced by allocateDataBuffer(), and allocateIndexBuffer().

| bool ExternalSortRunLoader::allocateDataBuffer | ( | | ) | [private] | | ---------------------------------------------- | - | | - | ----------- |

| bool ExternalSortRunLoader::allocateIndexBuffer | ( | | ) | [private] | | ----------------------------------------------- | - | | - | ----------- |

PBuffer & ExternalSortRunLoader::getPointerArrayEntry ( uint iTuple ) [inline, private]
void ExternalSortRunLoader::quickSortSwap ( uint l,
uint r
) [inline, private]

Definition at line 303 of file ExternalSortRunLoader.cpp.

References ExternalSortInfo::compareKeys(), getPointerArrayEntry(), keyAccessor, keyAccessor2, keyData, keyData2, quickSortSwap(), TupleAccessor::setCurrentTupleBuf(), sortInfo, tupleAccessor, tupleAccessor2, and TupleProjectionAccessor::unmarshal().

Referenced by quickSort().

PBuffer ExternalSortRunLoader::quickSortFindPivot ( uint l,
uint r
) [private]

Definition at line 268 of file ExternalSortRunLoader.cpp.

References ExternalSortInfo::compareKeys(), getPointerArrayEntry(), keyAccessor, keyAccessor2, keyData, keyData2, TupleAccessor::setCurrentTupleBuf(), sortInfo, step_factor, tupleAccessor, tupleAccessor2, and TupleProjectionAccessor::unmarshal().

Referenced by quickSort().

void ExternalSortRunLoader::quickSort ( uint l,
uint r
) [private]

| void ExternalSortRunLoader::startRun | ( | | ) | | ------------------------------------ | - | | - |

Prepares this loader to begin a new run.

Definition at line 82 of file ExternalSortRunLoader.cpp.

References allocateDataBuffer(), allocateIndexBuffer(), dataBuffers, fetchArray, freeBuffers, indexBuffers, nMemPagesMax, ExternalSortInfo::nSortMemPagesPerRun, ExternalSortFetchArray::nTuples, nTuplesFetched, nTuplesLoaded, pDataBuffer, pIndexBuffer, and sortInfo.

Referenced by ExternalSortExecStreamImpl::computeFirstResult(), and ExternalSortExecStreamImpl::computeFirstResultParallel().

| bool ExternalSortRunLoader::isStarted | ( | | ) | | ------------------------------------- | - | | - |

Loads data from buffer into a run.

Parameters:

bufAccessor buffer from which to read run

Returns:

result of load

Definition at line 173 of file ExternalSortRunLoader.cpp.

References allocateDataBuffer(), allocateIndexBuffer(), ExecStreamBufAccessor::consumeData(), ExecStreamBufAccessor::demandData(), EXTSORT_OVERFLOW, EXTSORT_SUCCESS, TupleAccessor::getBufferByteCount(), ExecStreamBufAccessor::getConsumptionAvailable(), ExecStreamBufAccessor::getConsumptionStart(), TupleAccessor::getMaxByteCount(), nTuplesLoaded, pDataBuffer, pDataBufferEnd, pIndexBuffer, pIndexBufferEnd, and tupleAccessor.

Referenced by ExternalSortExecStreamImpl::computeFirstResult().

| void ExternalSortRunLoader::sort | ( | | ) | | -------------------------------- | - | | - |

| uint ExternalSortRunLoader::getLoadedTupleCount | ( | | ) | | ------------------------------------------------------------------------------------------------------------ | - | | - |

| void ExternalSortRunLoader::releaseResources | ( | | ) | | -------------------------------------------- | - | | - |

Member Data Documentation

The documentation for this class was generated from the following files:

Generated on Mon Jun 22 04:00:32 2009 for Fennel by doxygen 1.5.1