Fennel: LbmGeneratorExecStream Class Reference (original) (raw)

LbmGeneratorExecStream reads the column store clusters corresponding to the columns of an index, and creates bitmap entries for those rows. More...

#include <[LbmGeneratorExecStream.h](LbmGeneratorExecStream%5F8h-source.html)>

Inheritance diagram for LbmGeneratorExecStream:

List of all members.

Public Member Functions
virtual void prepare (LbmGeneratorExecStreamParams const &params)
virtual void open (bool restart)
Opens this stream, acquiring any resources needed in order to be able to fetch data.
virtual ExecStreamResult execute (ExecStreamQuantum const &quantum)
Executes this stream.
virtual void getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType)
Determines resource requirements for this stream.
virtual void setResourceAllocation (ExecStreamResourceQuantity &quantity)
Sets current resource allocation for this stream.
virtual void closeImpl ()
Implements ClosableObject.
virtual void prepare (BTreeExecStreamParams const &params)
virtual void prepare (SingleOutputExecStreamParams const &params)
virtual void prepare (ExecStreamParams const &params)
Prepares this stream for execution.
virtual void setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
Initializes the buffer accessors for inputs to this stream.
virtual void setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors)
Initializes the buffer accessors for outputs from this stream.
virtual ExecStreamBufProvision getOutputBufProvision () const
Queries the BufferProvision which this stream is capable of when producing tuples.
virtual bool canEarlyClose ()
**Returns:**true if the stream can be closed early
ExecStreamGraph & getGraph () const
**Returns:**reference to containing graph
ExecStreamId getStreamId () const
**Returns:**the identifier for this stream within containing graph
virtual void getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void setName (std::string const &)
Sets unique name of this stream.
virtual std::string const & getName () const
**Returns:**the name of this stream, as known by the optimizer
virtual bool mayBlock () const
Queries whether this stream's implementation may block when execute() is called.
virtual void checkAbort () const
Checks whether there is an abort request for this stream's scheduler.
virtual ExecStreamBufProvision getOutputBufConversion () const
Queries the BufferProvision to which this stream needs its output to be converted, if any.
virtual ExecStreamBufProvision getInputBufProvision () const
Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
bool isClosed () const
**Returns:**whether the object has been closed
void close ()
Closes this object, releasing any unallocated resources.
virtual void initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
For use when initialization has to be deferred until after construction.
void trace (TraceLevel level, std::string message) const
Records a trace message.
bool isTracing () const
**Returns:**true iff tracing is enabled for this source
bool isTracingLevel (TraceLevel level) const
Determines whether a particular level is being traced.
TraceTarget & getTraceTarget () const
**Returns:**the TraceTarget for this source
SharedTraceTarget getSharedTraceTarget () const
**Returns:**the SharedTraceTarget for this source
std::string getTraceSourceName () const
Gets the name of this source.
void setTraceSourceName (std::string const &n)
Sets the name of this source.
TraceLevel getMinimumTraceLevel () const
void disableTracing ()
virtual void initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name)
For use when initialization has to be deferred until after construction.
void postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index)
Posts an exception, such as a row exception.
void postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index)
Posts an exception, such as a row exception.
bool hasTarget () const
**Returns:**true iff an error target has been set
ErrorTarget & getErrorTarget () const
**Returns:**the ErrorTarget for this source
SharedErrorTarget getSharedErrorTarget () const
**Returns:**the SharedErrorTarget for this source
std::string getErrorSourceName () const
Gets the name of this source.
void setErrorSourceName (std::string const &n)
Sets the name of this source.
void disableTarget ()
Static Public Member Functions
static SharedBTreeWriter newWriter (BTreeExecStreamParams const &params)
static void copyParamsToDescriptor (BTreeDescriptor &, BTreeParams const &, SharedCacheAccessor const &)
Protected Member Functions
virtual SharedBTreeReader newReader ()
SharedBTreeWriter newWriter (bool monotonic=false)
virtual void endSearch ()
Forgets the current reader or writer's search, releasing any page locks.
Protected Attributes
BTreeDescriptor treeDescriptor
SegmentAccessor scratchAccessor
BTreeOwnerRootMap * pRootMap
SharedBTreeAccessBase pBTreeAccessBase
SharedBTreeReader pBTreeReader
DynamicParamId rootPageIdParamId
SharedExecStreamBufAccessor pOutAccessor
bool isOpen
Whether this stream is currently open.
ExecStreamGraph * pGraph
Dataflow graph containing this stream.
ExecStreamId id
Identifier for this stream; local to its containing graph.
std::string name
Name of stream, as known by optimizer.
SharedDynamicParamManager pDynamicParamManager
The dynamic parameter manager available to this stream.
SharedLogicalTxn pTxn
The transaction embracing the stream.
ExecStreamResourceQuantity resourceAllocation
Resource quantities currently allocated to this stream.
SharedCacheAccessor pQuotaAccessor
CacheAccessor used for quota tracking.
SharedCacheAccessor pScratchQuotaAccessor
CacheAccessor used for scratch page quota tracking.
bool needsClose
Private Member Functions
ExecStreamResult generateSingleKeyBitmaps (ExecStreamQuantum const &quantum)
Generates bitmaps for a single column index.
ExecStreamResult generateMultiKeyBitmaps (ExecStreamQuantum const &quantum)
Generates bitmaps for multi-column indexes.
void createSingletonBitmapEntry ()
Creates a singleton bitmap entry and resets state to indicate that the current row has been processed if successful.
bool generateBitmaps ()
Reads values from a single batch corresponding to a compressed batch and generates bitmap entries.
bool generateSingletons ()
Reads values from a single batch corresponding to a non-compressed batch and generates singleton bitmap entries.
bool advanceReader (SharedLcsClusterReader &pScan)
Advances a single cluster and its corresponding column readers one rid forward in the current batch.
bool initBitmapTable (uint nEntries)
Initializes bitmap table, increasing the size, as needed and assigns buffers to bitmap entries.
void initRidAndBitmap (TupleData &bitmapTuple, LcsRid *pCurrRid)
Sets rid in a bitmap tupledata.
bool addRidToBitmap (uint keycode, TupleData &keyvalue, LcsRid rid)
Adds rid to a bitmap entry being constructing for a specified key.
PBuffer flushBuffer (LcsRid addRid)
Flushes out an existing buffer currently in use by another LbmEntry.
bool flushTable (uint start)
Flushes out entire table of bitmap entries.
bool flushEntry (uint keycode)
Flushes a single entry in the bitmap table and resets the LbmEntry associated with the table entry.
void syncColumns (SharedLcsClusterReader &pScan)
Positions column readers based on new cluster reader position.
bool readColVals (SharedLcsClusterReader &pScan, TupleDataWithBuffer &tupleData, uint colStart)
Reads column values based on current position of cluster reader.
virtual void buildOutputProj (TupleProjection &outputProj, LcsRowScanBaseExecStreamParams const &params)
Builds outputProj from params.
virtual void prepare (LcsRowScanBaseExecStreamParams const &params)
virtual void prepare (ConfluenceExecStreamParams const &params)
virtual void getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
Initializes the buffer accessors for inputs to this stream.
virtual ExecStreamBufProvision getInputBufProvision () const
Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
Private Attributes
uint numMiscScratchPages
Number of misc scratch pages needed; excludes pages for generating bitmap entries.
uint maxNumScratchPages
Number of scratch pages to allocate for constructing bitmaps as determined by the resource governor.
DynamicParamId insertRowCountParamId
Dynamic parameter id used to pass along number of rows loaded.
bool createIndex
True if index is being loaded as part of a create index statement.
std::vector< DynamicParamId > parameterIds
Ordered list of dynamic parameter ids for the clusters that provide source data for this stream.
uint scratchPageSize
Size of a scratch page.
uint entrySize
Size of a bitmap entry buffer.
uint maxBitmapSize
Max size of a bitmap entry.
uint minBitmapSize
Min size of a bitmap entry.
ClusterPageLock scratchLock
Lock for scratch accessor.
TupleData inputTuple
Input tuple data.
RecordNum numRowsToLoad
Number of rows to load.
RecordNum rowCount
Running count of number of rows read.
LcsRid startRid
Starting rid;.
LcsRid currRid
Current rid being loaded.
TupleDataWithBuffer bitmapTuple
Tuple data with buffer for the bitmap tuple.
TupleDescriptor bitmapTupleDesc
Tuple descriptor representing bitmap tuple.
TupleData outputTuple
Pointer to generated tuple data.
uint nIdxKeys
Number of keys in the bitmap index, excluding the starting rid.
bool batchRead
True if current batch has been read.
uint currBatch
Current batch entry being processed.
std::vector< uint16_t > keyCodes
Keycodes read from a batch.
std::vector< LbmEntryInfo > bitmapTable
Table of bitmap entries under construction.
uint nBitmapEntries
Number of entries in the bitmap table.
uint flushIdx
Index of buffer entry to flush.
uint nBitmapBuffers
Number of entries in the bitmap buffer table.
uint nScratchPagesAllocated
Number of scratch pages allocated.
std::vector< PBuffer > scratchPages
Vector of pointers to scratch pages allocated.
LbmPendingProduceType producePending
Produce of one or more output tuples pending: LBM_TABLEFLUSH_PENDING, LBM_ENTRYFLUSH_PENDING, LBM_FINALFLUSH_PENDING, LBM_NOFLUSH_PENDING.
uint flushStart
Index into bitmap table from which to start a pending table flush or the single entry currently being flushed.
bool skipRead
If true, skip the initial read the next time generator is called since we haven't finished processing the current rowid.
bool doneReading
If true, all rows from the column store table have been read.
bool revertToSingletons
If true, even though the current batch is compressed and the index has a single key, generate singleton bitmaps for the batch due to lack of buffer space.
VectorOfUint projMap
Projection map that maps columns read from cluster to their position in the output projection.
uint nClusters
Number of clusters to be scanned.
boost::scoped_array< SharedLcsClusterReader > pClusters
Array containing cluster readers.
TupleDescriptor projDescriptor
Tuple descriptor representing columns to be projected from scans.
std::vector< int > nonClusterCols
List of the non-cluster columns that need to be projected.
bool allSpecial
True in the special case where we are only reading special columns.
CircularBuffer< LcsRidRun > ridRuns
Circular buffer of rid runs.
std::vector< UnalignedAttributeAccessor > attrAccessors
Accessors used for loading actual column values.
std::vector< SharedExecStreamBufAccessor > inAccessors

Detailed Description

LbmGeneratorExecStream reads the column store clusters corresponding to the columns of an index, and creates bitmap entries for those rows.

Its input is a rowcount, indicating the number of rows it needs to create entries for, and the startRid corresponding to the first row. It's assumed that the rows it needs to create entries for are all sequential, starting at that start value.

If possible, LbmGeneratorExecStream will try to take advantage of compressed column store batches to create bitmap entries spanning multiple rows.

As output, LbmGeneratorExecStream writes out the bitmap entries it has created. It also passes along the rowcount passed in as its input, using a dynamic parameter that will be read downstream.

Version:

Id

//open/dev/fennel/lucidera/bitmap/LbmGeneratorExecStream.h#15

Definition at line 92 of file LbmGeneratorExecStream.h.


Member Function Documentation

Generates bitmaps for multi-column indexes.

Parameters:

quantum quantum for stream

Returns:

either EXECRC_BUF_OVERFLOW or EXECRC_EOS

Definition at line 290 of file LbmGeneratorExecStream.cpp.

References bitmapTuple, createSingletonBitmapEntry(), currRid, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, flushEntry(), LcsRowScanBaseExecStream::nClusters, ExecStreamQuantum::nTuplesMax, opaqueToInt(), LcsRowScanBaseExecStream::pClusters, LcsRowScanBaseExecStream::readColVals(), TupleDataWithBuffer::resetBuffer(), skipRead, and LcsRowScanBaseExecStream::syncColumns().

Referenced by execute().

| void LbmGeneratorExecStream::createSingletonBitmapEntry | ( | | ) | [private] | | ------------------------------------------------------- | - | | - | ----------- |

| bool LbmGeneratorExecStream::generateBitmaps | ( | | ) | [private] | | -------------------------------------------- | - | | - | ----------- |

Reads values from a single batch corresponding to a compressed batch and generates bitmap entries.

Returns:

true if successfully processed batch; false if overflow occurred while writing out tuples

Definition at line 365 of file LbmGeneratorExecStream.cpp.

References addRidToBitmap(), LcsRowScanBaseExecStream::attrAccessors, batchRead, bitmapTuple, currBatch, currRid, flushTable(), generateSingletons(), LcsColumnReader::getBatchValCount(), LcsColumnReader::getBatchValue(), initBitmapTable(), initRidAndBitmap(), keyCodes, LcsRowScanBaseExecStream::pClusters, LcsColumnReader::readCompressedBatch(), TupleDataWithBuffer::resetBuffer(), revertToSingletons, rowCount, and skipRead.

Referenced by generateSingleKeyBitmaps().

| bool LbmGeneratorExecStream::generateSingletons | ( | | ) | [private] | | ----------------------------------------------- | - | | - | ----------- |

Reads values from a single batch corresponding to a non-compressed batch and generates singleton bitmap entries.

Returns:

true if successfully processed batch; false if overflow occurred while writing out tuples

Definition at line 428 of file LbmGeneratorExecStream.cpp.

References advanceReader(), LcsRowScanBaseExecStream::attrAccessors, batchRead, bitmapTuple, createSingletonBitmapEntry(), flushEntry(), LcsRowScanBaseExecStream::pClusters, LcsRowScanBaseExecStream::projMap, TupleDataWithBuffer::resetBuffer(), and skipRead.

Referenced by generateBitmaps(), and generateSingleKeyBitmaps().

bool LbmGeneratorExecStream::initBitmapTable ( uint nEntries ) [private]

Initializes bitmap table, increasing the size, as needed and assigns buffers to bitmap entries.

Parameters:

nEntries desired size of the table

Returns:

true if the table was successfully initialized, indicating that there was enough space to accomodate the number of keys in the current batch

Definition at line 482 of file LbmGeneratorExecStream.cpp.

References SegNodeLock< Node >::allocatePage(), bitmapTable, entrySize, flushIdx, SegPageLock::getPage(), CachePage::getWritableData(), maxBitmapSize, maxNumScratchPages, minBitmapSize, nBitmapEntries, nScratchPagesAllocated, scratchLock, scratchPages, and scratchPageSize.

Referenced by execute(), and generateBitmaps().

void LbmGeneratorExecStream::initRidAndBitmap ( TupleData & bitmapTuple,
LcsRid * pCurrRid
) [private]
bool LbmGeneratorExecStream::addRidToBitmap ( uint keycode,
TupleData & keyvalue,
LcsRid rid
) [private]

Adds rid to a bitmap entry being constructing for a specified key.

Creates a new bitmap entry, as needed, and flushes out entries as buffer space fills out.

Parameters:

keycode keycode corresponding to the value of the rid being added; always 0 in the case of a batch that is either non-compressed or has more than 1 key
keyvalue tupledata containing the initial bitmap entry value; i.e., only the key value and rid are set
rid rid to be added

Returns:

false if an error was encountered flushing out a bitmap entry

Definition at line 571 of file LbmGeneratorExecStream.cpp.

References bitmapTable, bitmapTupleDesc, entrySize, flushBuffer(), flushEntry(), and nBitmapEntries.

Referenced by createSingletonBitmapEntry(), and generateBitmaps().

PBuffer LbmGeneratorExecStream::flushBuffer ( LcsRid addRid ) [private]
bool LbmGeneratorExecStream::flushTable ( uint start ) [private]
bool LbmGeneratorExecStream::flushEntry ( uint keycode ) [private]

Flushes a single entry in the bitmap table and resets the LbmEntry associated with the table entry.

Parameters:

keycode index of the bitmap entry to be flushed

Returns:

true if entries was successfully written to output stream

Definition at line 667 of file LbmGeneratorExecStream.cpp.

References bitmapTable, flushStart, LBM_ENTRYFLUSH_PENDING, outputTuple, SingleOutputExecStream::pOutAccessor, producePending, LbmEntry::toString(), and TRACE_FINE.

Referenced by addRidToBitmap(), flushBuffer(), flushTable(), generateMultiKeyBitmaps(), and generateSingletons().

Definition at line 29 of file LbmGeneratorExecStream.cpp.

References SegPageLock::accessSegment(), LcsRowScanBaseExecStream::attrAccessors, bitmapTuple, bitmapTupleDesc, TupleData::compute(), TupleDataWithBuffer::computeAndAllocate(), LbmGeneratorExecStreamParams::createIndex, createIndex, LbmEntry::getSizeBounds(), ConfluenceExecStream::inAccessors, inputTuple, LbmGeneratorExecStreamParams::insertRowCountParamId, insertRowCountParamId, BTreeDescriptor::keyProjection, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, maxBitmapSize, minBitmapSize, LcsRowScanBaseExecStream::nClusters, nIdxKeys, opaqueToInt(), parameterIds, SingleOutputExecStream::pOutAccessor, LcsRowScanBaseExecStream::prepare(), BTreeExecStream::prepare(), SegmentAccessor::pSegment, CircularBuffer< T >::resize(), LcsRowScanBaseExecStream::ridRuns, BTreeExecStream::scratchAccessor, scratchLock, scratchPageSize, BTreeDescriptor::segmentAccessor, BTreeExecStream::treeDescriptor, and BTreeDescriptor::tupleDescriptor.

void LbmGeneratorExecStream::open ( bool restart ) [virtual]

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:

restart if true, the stream must be already open, and should reset itself to start from the beginning of its result set

Reimplemented from BTreeExecStream.

Definition at line 75 of file LbmGeneratorExecStream.cpp.

References batchRead, CircularBuffer< T >::clear(), doneReading, flushIdx, ConfluenceExecStream::inAccessors, insertRowCountParamId, LBM_NOFLUSH_PENDING, nBitmapBuffers, nBitmapEntries, LcsRowScanBaseExecStream::nClusters, nScratchPagesAllocated, opaqueToInt(), LcsRowScanBaseExecStream::open(), BTreeExecStream::open(), parameterIds, LcsRowScanBaseExecStream::pClusters, ExecStream::pDynamicParamManager, producePending, revertToSingletons, LcsRowScanBaseExecStream::ridRuns, rowCount, and skipRead.

Executes this stream.

Parameters:

quantum governs the maximum amount of execution to perform

Returns:

code indicating reason execution ceased

Implements ExecStream.

Definition at line 145 of file LbmGeneratorExecStream.cpp.

References bitmapTable, createIndex, currRid, doneReading, EXECBUF_EOS, EXECRC_BUF_OVERFLOW, EXECRC_BUF_UNDERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, flushStart, flushTable(), generateMultiKeyBitmaps(), generateSingleKeyBitmaps(), ConfluenceExecStream::inAccessors, initBitmapTable(), inputTuple, insertRowCountParamId, LBM_ENTRYFLUSH_PENDING, LBM_NOFLUSH_PENDING, LBM_TABLEFLUSH_PENDING, MAXU, LcsRowScanBaseExecStream::nClusters, nIdxKeys, LcsRidRun::nRids, numRowsToLoad, outputTuple, LcsRowScanBaseExecStream::pClusters, ExecStream::pDynamicParamManager, SingleOutputExecStream::pOutAccessor, producePending, CircularBuffer< T >::push_back(), LcsRowScanBaseExecStream::ridRuns, rowCount, LcsRidRun::startRid, startRid, and LcsRowScanBaseExecStream::syncColumns().

| void LbmGeneratorExecStream::closeImpl | ( | | ) | [virtual] | | -------------------------------------- | - | | - | ----------- |

SharedBTreeWriter BTreeExecStream::newWriter ( bool monotonic = false ) [protected, inherited]

| void BTreeExecStream::endSearch | ( | | ) | [protected, virtual, inherited] | | ------------------------------- | - | | - | --------------------------------- |

Definition at line 30 of file BTreeExecStream.cpp.

References BTreeExecStream::copyParamsToDescriptor(), ExecStreamParams::pCacheAccessor, SingleOutputExecStream::prepare(), BTreeParams::pRootMap, BTreeExecStream::pRootMap, BTreeParams::rootPageIdParamId, BTreeExecStream::rootPageIdParamId, ExecStreamParams::scratchAccessor, BTreeExecStream::scratchAccessor, and BTreeExecStream::treeDescriptor.

Referenced by LcsClusterAppendExecStream::prepare(), prepare(), BTreeReadExecStream::prepare(), and BTreeInsertExecStream::prepare().

Definition at line 48 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::getOutputBufProvision(), SingleOutputExecStreamParams::outputTupleDesc, SingleOutputExecStreamParams::outputTupleFormat, SingleOutputExecStream::pOutAccessor, and ExecStream::prepare().

Referenced by BTreeExecStream::prepare(), FlatFileExecStreamImpl::prepare(), ValuesExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), ConfluenceExecStream::prepare(), and ConduitExecStream::prepare().

void ExecStream::prepare ( ExecStreamParams const & params ) [virtual, inherited]

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:

params instance of stream parameterization class which should be used to prepare this stream

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

Definition at line 93 of file BTreeExecStream.cpp.

References BTreeParams::keyProj, BTreeDescriptor::keyProjection, BTreeParams::pageOwnerId, BTreeDescriptor::pageOwnerId, SegmentAccessor::pCacheAccessor, BTreeParams::pSegment, SegmentAccessor::pSegment, BTreeParams::rootPageId, BTreeDescriptor::rootPageId, BTreeDescriptor::segmentAccessor, BTreeParams::segmentId, BTreeDescriptor::segmentId, BTreeExecStream::treeDescriptor, BTreeParams::tupleDesc, and BTreeDescriptor::tupleDescriptor.

Referenced by BTreeExecStream::newWriter(), LbmSplicerExecStream::prepare(), and BTreeExecStream::prepare().

void SingleOutputExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & inAccessors ) [virtual, inherited]

Initializes the buffer accessors for inputs to this stream.

This method is only ever called once, before prepare.

Parameters:

inAccessors buffer accessors ordered by input stream

Implements ExecStream.

Reimplemented in ConduitExecStream, and ConfluenceExecStream.

Definition at line 35 of file SingleOutputExecStream.cpp.

00037 { 00038 assert(inAccessors.size() == 0); 00039 }

void SingleOutputExecStream::setOutputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & outAccessors ) [virtual, inherited]

| bool ExecStream::canEarlyClose | ( | | ) | [virtual, inherited] | | ------------------------------ | - | | - | ---------------------- |

| ExecStreamId ExecStream::getStreamId | ( | | ) | const [inline, inherited] | | -------------------------------------------------------------------------------------------------- | - | | - | --------------------------- |

Returns:

the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

00289 { 00290 return id; 00291 }

Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, FlatFileExecStreamImpl, BTreeInsertExecStream, BTreeReadExecStream, FtrsTableWriterExecStream, LbmChopperExecStream, LbmSplicerExecStream, LcsClusterAppendExecStream, LcsClusterReplaceExecStream, LcsRowScanBaseExecStream, and LcsRowScanExecStream.

Definition at line 102 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, and ExecStreamResourceQuantity::nThreads.

00105 { 00106 minQuantity.nThreads = 0; 00107 minQuantity.nCachePages = 0; 00108 optQuantity = minQuantity; 00109 }

| void ExecStream::setName | ( | std::string const & | | ) | [virtual, inherited] | | ------------------------ | - | ------------------- | | - | ---------------------- |

| std::string const & ExecStream::getName | ( | | ) | const [virtual, inherited] | | --------------------------------------- | - | | - | ---------------------------- |

| bool ExecStream::mayBlock | ( | | ) | const [virtual, inherited] | | ------------------------- | - | | - | ---------------------------- |

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:

whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

00168 { 00169 return false; 00170 }

| void ExecStream::checkAbort | ( | | ) | const [virtual, inherited] | | --------------------------- | - | | - | ---------------------------- |

| bool ClosableObject::isClosed | ( | | ) | const [inline, inherited] | | ----------------------------- | - | | - | --------------------------- |

Returns:

whether the object has been closed

Definition at line 58 of file ClosableObject.h.

| void ClosableObject::close | ( | | ) | [inherited] | | -------------------------- | - | | - | ------------- |

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

void TraceSource::initTraceSource ( SharedTraceTarget pTraceTarget,
std::string name
) [virtual, inherited]
void TraceSource::trace ( TraceLevel level,
std::string message
) const [inherited]

| bool TraceSource::isTracing | ( | | ) | const [inline, inherited] | | --------------------------- | - | | - | --------------------------- |

bool TraceSource::isTracingLevel ( TraceLevel level ) const [inline, inherited]

| TraceTarget& TraceSource::getTraceTarget | ( | | ) | const [inline, inherited] | | ----------------------------------------------------------------- | - | | - | --------------------------- |

| std::string TraceSource::getTraceSourceName | ( | | ) | const [inline, inherited] | | ------------------------------------------- | - | | - | --------------------------- |

void TraceSource::setTraceSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137 { 00138 name = n; 00139 }

| TraceLevel TraceSource::getMinimumTraceLevel | ( | | ) | const [inline, inherited] | | ------------------------------------------------------------------------------------------------------ | - | | - | --------------------------- |

| void TraceSource::disableTracing | ( | | ) | [inherited] | | -------------------------------- | - | | - | ------------- |

void ErrorSource::initErrorSource ( SharedErrorTarget pErrorTarget,
const std::string & name
) [virtual, inherited]
void ErrorSource::postError ( ErrorLevel level,
const std::string & message,
void * address,
long capacity,
int index
) [inherited]

| bool ErrorSource::hasTarget | ( | | ) | const [inline, inherited] | | --------------------------- | - | | - | --------------------------- |

| ErrorTarget& ErrorSource::getErrorTarget | ( | | ) | const [inline, inherited] | | ----------------------------------------------------------------- | - | | - | --------------------------- |

Returns:

the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

| std::string ErrorSource::getErrorSourceName | ( | | ) | const [inline, inherited] | | ------------------------------------------- | - | | - | --------------------------- |

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:

the name

Definition at line 139 of file ErrorSource.h.

00140 { 00141 return name; 00142 }

void ErrorSource::setErrorSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

00149 { 00150 name = n; 00151 }

| void ErrorSource::disableTarget | ( | | ) | [inherited] | | ------------------------------- | - | | - | ------------- |

Definition at line 33 of file LcsRowScanBaseExecStream.cpp.

References LcsRowScanBaseExecStream::allSpecial, LcsRowScanBaseExecStream::buildOutputProj(), BTreeParams::keyProj, BTreeDescriptor::keyProjection, LCS_RID_COLUMN_ID, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, LcsRowScanBaseExecStream::nClusters, LcsRowScanBaseExecStream::nonClusterCols, LcsRowScanBaseExecStreamParams::outputProj, BTreeParams::pageOwnerId, BTreeDescriptor::pageOwnerId, ExecStreamParams::pCacheAccessor, SegmentAccessor::pCacheAccessor, LcsRowScanBaseExecStream::pClusters, SingleOutputExecStream::pOutAccessor, ConfluenceExecStream::prepare(), LcsRowScanBaseExecStream::projDescriptor, LcsRowScanBaseExecStream::projMap, BTreeParams::pSegment, SegmentAccessor::pSegment, LcsRowScanBaseExecStream::ridRuns, BTreeParams::rootPageId, BTreeDescriptor::rootPageId, BTreeDescriptor::segmentAccessor, BTreeParams::segmentId, BTreeDescriptor::segmentId, BTreeParams::tupleDesc, and BTreeDescriptor::tupleDescriptor.

Referenced by LcsRowScanExecStream::prepare(), and prepare().

Definition at line 37 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::getInputBufProvision(), ConfluenceExecStream::inAccessors, and SingleOutputExecStream::prepare().

Referenced by LcsRowScanBaseExecStream::prepare(), LbmUnionExecStream::prepare(), LbmChopperExecStream::prepare(), LbmBitOpExecStream::prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), CartesianJoinExecStream::prepare(), and BarrierExecStream::prepare().

void ConfluenceExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & inAccessors ) [virtual, inherited]

Member Data Documentation

Definition at line 113 of file BTreeExecStream.h.

Referenced by BTreeInsertExecStream::buildTree(), BTreeExecStream::closeImpl(), BTreeExecStream::copyParamsToDescriptor(), LcsClusterReplaceExecStream::getTupleForLoad(), LcsClusterAppendExecStream::initLoad(), LcsClusterReplaceExecStream::initTupleLoadParams(), BTreePrefetchSearchExecStream::innerSearchLoop(), BTreePrefetchSearchExecStream::newReader(), BTreeExecStream::newReader(), BTreeExecStream::newWriter(), LcsClusterReplaceExecStream::open(), BTreeSearchExecStream::open(), BTreePrefetchSearchExecStream::open(), BTreeInsertExecStream::open(), BTreeExecStream::open(), LcsClusterReplaceExecStream::prepare(), LcsClusterAppendExecStream::prepare(), LbmSearchExecStream::prepare(), prepare(), BTreeSearchExecStream::prepare(), BTreeInsertExecStream::prepare(), BTreeExecStream::prepare(), and BTreeInsertExecStream::truncateTree().

Definition at line 56 of file SingleOutputExecStream.h.

Referenced by LcsClusterAppendExecStream::compress(), ExternalSortExecStreamImpl::execute(), LcsRowScanExecStream::execute(), LbmUnionExecStream::execute(), LbmNormalizerExecStream::execute(), execute(), LbmChopperExecStream::execute(), LhxJoinExecStream::execute(), LhxAggExecStream::execute(), FtrsTableWriterExecStream::execute(), BTreeSortExecStream::execute(), BTreeSearchUniqueExecStream::execute(), BTreeScanExecStream::execute(), BTreePrefetchSearchExecStream::execute(), BTreeInsertExecStream::execute(), FlatFileExecStreamImpl::execute(), ValuesExecStream::execute(), UncollectExecStream::execute(), SortedAggExecStream::execute(), SegBufferReaderExecStream::execute(), ScratchBufferExecStream::execute(), ReshapeExecStream::execute(), MockResourceExecStream::execute(), MockProducerExecStream::execute(), MergeExecStream::execute(), DoubleBufferExecStream::execute(), CorrelationJoinExecStream::execute(), CopyExecStream::execute(), CollectExecStream::execute(), CartesianJoinExecStream::execute(), BernoulliSamplingExecStream::execute(), BarrierExecStream::execute(), CalcExecStream::execute(), flushEntry(), MockProducerExecStream::getProducedRowCount(), ExternalSortExecStreamImpl::getResourceRequirements(), BTreeSearchExecStream::innerFetchLoop(), LbmUnionExecStream::open(), LbmChopperExecStream::open(), LbmBitOpExecStream::open(), SingleOutputExecStream::open(), SegBufferReaderExecStream::open(), SegBufferExecStream::open(), ScratchBufferExecStream::open(), DoubleBufferExecStream::open(), CollectExecStream::open(), SegBufferExecStream::openBufferForRead(), ConduitExecStream::precheckConduitBuffers(), LcsRowScanExecStream::prepare(), LcsRowScanBaseExecStream::prepare(), LcsClusterAppendExecStream::prepare(), LbmUnionExecStream::prepare(), prepare(), LhxJoinExecStream::prepare(), LhxAggExecStream::prepare(), FtrsTableWriterExecStream::prepare(), FlatFileExecStreamImpl::prepare(), UncollectExecStream::prepare(), SortedAggExecStream::prepare(), SingleOutputExecStream::prepare(), ReshapeExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), ConduitExecStream::prepare(), CollectExecStream::prepare(), CartesianJoinExecStream::prepare(), BernoulliSamplingExecStream::prepare(), BarrierExecStream::prepare(), CalcExecStream::prepare(), SortedAggExecStream::produce(), LbmBitOpExecStream::producePendingOutput(), LbmUnionExecStream::produceTuple(), LbmNormalizerExecStream::produceTuple(), LbmMinusExecStream::produceTuple(), LbmChopperExecStream::produceTuple(), LbmBitOpExecStream::produceTuple(), LbmBitOpExecStream::readInput(), LbmMinusExecStream::readMinuendInput(), and SingleOutputExecStream::setOutputBufAccessors().

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

Referenced by CorrelationJoinExecStream::close(), SegBufferReaderExecStream::closeImpl(), LbmUnionExecStream::execute(), LbmSplicerExecStream::execute(), execute(), LbmChopperExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), LcsClusterReplaceExecStream::open(), LbmUnionExecStream::open(), LbmSplicerExecStream::open(), open(), LbmBitOpExecStream::open(), BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), SegBufferWriterExecStream::open(), SegBufferReaderExecStream::open(), NestedLoopJoinExecStream::open(), CorrelationJoinExecStream::open(), ExecStream::prepare(), CalcExecStream::prepare(), NestedLoopJoinExecStream::processLeftInput(), LbmSearchExecStream::reachedTupleLimit(), ReshapeExecStream::readDynamicParams(), SegBufferWriterExecStream::readReaderRefCount(), BTreeSearchExecStream::readSearchKey(), BTreeSearchExecStream::readUpperBoundKey(), LbmSearchExecStream::setAdditionalKeys(), and LbmBitOpExecStream::writeStartRidParamValue().

Number of clusters to be scanned.

Definition at line 82 of file LcsRowScanBaseExecStream.h.

Referenced by LcsRowScanBaseExecStream::closeImpl(), LcsRowScanExecStream::execute(), execute(), generateMultiKeyBitmaps(), LcsRowScanBaseExecStream::getResourceRequirements(), LcsRowScanBaseExecStream::LcsRowScanBaseExecStream(), LcsRowScanBaseExecStream::open(), open(), LcsRowScanBaseExecStream::prepare(), prepare(), and LcsRowScanExecStream::prepareResidualFilters().

Array containing cluster readers.

Definition at line 87 of file LcsRowScanBaseExecStream.h.

Referenced by LcsRowScanBaseExecStream::closeImpl(), LcsRowScanExecStream::execute(), execute(), generateBitmaps(), generateMultiKeyBitmaps(), generateSingleKeyBitmaps(), generateSingletons(), LcsRowScanBaseExecStream::open(), open(), LcsRowScanBaseExecStream::prepare(), and LcsRowScanExecStream::prepareResidualFilters().

Definition at line 50 of file ConfluenceExecStream.h.

Referenced by NestedLoopJoinExecStream::checkNumInputs(), CartesianJoinExecStream::checkNumInputs(), LbmMinusExecStream::comparePrefixes(), execute(), MergeExecStream::execute(), BarrierExecStream::execute(), LbmMinusExecStream::findMinInput(), LcsRowScanExecStream::initializeFiltersIfNeeded(), LcsRowScanExecStream::open(), LbmUnionExecStream::open(), LbmMinusExecStream::open(), open(), LbmChopperExecStream::open(), LbmBitOpExecStream::open(), ConfluenceExecStream::open(), LcsRowScanExecStream::prepare(), LbmUnionExecStream::prepare(), LbmMinusExecStream::prepare(), prepare(), LbmChopperExecStream::prepare(), LbmBitOpExecStream::prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), ConfluenceExecStream::prepare(), CartesianJoinExecStream::prepare(), BarrierExecStream::prepare(), NestedLoopJoinExecStream::preProcessRightInput(), BarrierExecStream::processInputTuple(), LbmBitOpExecStream::producePendingOutput(), LbmMinusExecStream::restartSubtrahends(), LhxJoinExecStream::setHashInfo(), and ConfluenceExecStream::setInputBufAccessors().


The documentation for this class was generated from the following files:


Generated on Mon Jun 22 04:00:34 2009 for Fennel by doxygen 1.5.1