Fennel: LbmGeneratorExecStream Class Reference (original) (raw)
LbmGeneratorExecStream reads the column store clusters corresponding to the columns of an index, and creates bitmap entries for those rows. More...
#include <[LbmGeneratorExecStream.h](LbmGeneratorExecStream%5F8h-source.html)>
Inheritance diagram for LbmGeneratorExecStream:

| Public Member Functions | |
|---|---|
| virtual void | prepare (LbmGeneratorExecStreamParams const ¶ms) |
| virtual void | open (bool restart) |
| Opens this stream, acquiring any resources needed in order to be able to fetch data. | |
| virtual ExecStreamResult | execute (ExecStreamQuantum const &quantum) |
| Executes this stream. | |
| virtual void | getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType) |
| Determines resource requirements for this stream. | |
| virtual void | setResourceAllocation (ExecStreamResourceQuantity &quantity) |
| Sets current resource allocation for this stream. | |
| virtual void | closeImpl () |
| Implements ClosableObject. | |
| virtual void | prepare (BTreeExecStreamParams const ¶ms) |
| virtual void | prepare (SingleOutputExecStreamParams const ¶ms) |
| virtual void | prepare (ExecStreamParams const ¶ms) |
| Prepares this stream for execution. | |
| virtual void | setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors) |
| Initializes the buffer accessors for inputs to this stream. | |
| virtual void | setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors) |
| Initializes the buffer accessors for outputs from this stream. | |
| virtual ExecStreamBufProvision | getOutputBufProvision () const |
| Queries the BufferProvision which this stream is capable of when producing tuples. | |
| virtual bool | canEarlyClose () |
| **Returns:**true if the stream can be closed early | |
| ExecStreamGraph & | getGraph () const |
| **Returns:**reference to containing graph | |
| ExecStreamId | getStreamId () const |
| **Returns:**the identifier for this stream within containing graph | |
| virtual void | getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity) |
| virtual void | setName (std::string const &) |
| Sets unique name of this stream. | |
| virtual std::string const & | getName () const |
| **Returns:**the name of this stream, as known by the optimizer | |
| virtual bool | mayBlock () const |
| Queries whether this stream's implementation may block when execute() is called. | |
| virtual void | checkAbort () const |
| Checks whether there is an abort request for this stream's scheduler. | |
| virtual ExecStreamBufProvision | getOutputBufConversion () const |
| Queries the BufferProvision to which this stream needs its output to be converted, if any. | |
| virtual ExecStreamBufProvision | getInputBufProvision () const |
| Queries the BufferProvision which this stream requires of its inputs when consuming their tuples. | |
| bool | isClosed () const |
| **Returns:**whether the object has been closed | |
| void | close () |
| Closes this object, releasing any unallocated resources. | |
| virtual void | initTraceSource (SharedTraceTarget pTraceTarget, std::string name) |
| For use when initialization has to be deferred until after construction. | |
| void | trace (TraceLevel level, std::string message) const |
| Records a trace message. | |
| bool | isTracing () const |
| **Returns:**true iff tracing is enabled for this source | |
| bool | isTracingLevel (TraceLevel level) const |
| Determines whether a particular level is being traced. | |
| TraceTarget & | getTraceTarget () const |
| **Returns:**the TraceTarget for this source | |
| SharedTraceTarget | getSharedTraceTarget () const |
| **Returns:**the SharedTraceTarget for this source | |
| std::string | getTraceSourceName () const |
| Gets the name of this source. | |
| void | setTraceSourceName (std::string const &n) |
| Sets the name of this source. | |
| TraceLevel | getMinimumTraceLevel () const |
| void | disableTracing () |
| virtual void | initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name) |
| For use when initialization has to be deferred until after construction. | |
| void | postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index) |
| Posts an exception, such as a row exception. | |
| void | postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index) |
| Posts an exception, such as a row exception. | |
| bool | hasTarget () const |
| **Returns:**true iff an error target has been set | |
| ErrorTarget & | getErrorTarget () const |
| **Returns:**the ErrorTarget for this source | |
| SharedErrorTarget | getSharedErrorTarget () const |
| **Returns:**the SharedErrorTarget for this source | |
| std::string | getErrorSourceName () const |
| Gets the name of this source. | |
| void | setErrorSourceName (std::string const &n) |
| Sets the name of this source. | |
| void | disableTarget () |
| Static Public Member Functions | |
| static SharedBTreeWriter | newWriter (BTreeExecStreamParams const ¶ms) |
| static void | copyParamsToDescriptor (BTreeDescriptor &, BTreeParams const &, SharedCacheAccessor const &) |
| Protected Member Functions | |
| virtual SharedBTreeReader | newReader () |
| SharedBTreeWriter | newWriter (bool monotonic=false) |
| virtual void | endSearch () |
| Forgets the current reader or writer's search, releasing any page locks. | |
| Protected Attributes | |
| BTreeDescriptor | treeDescriptor |
| SegmentAccessor | scratchAccessor |
| BTreeOwnerRootMap * | pRootMap |
| SharedBTreeAccessBase | pBTreeAccessBase |
| SharedBTreeReader | pBTreeReader |
| DynamicParamId | rootPageIdParamId |
| SharedExecStreamBufAccessor | pOutAccessor |
| bool | isOpen |
| Whether this stream is currently open. | |
| ExecStreamGraph * | pGraph |
| Dataflow graph containing this stream. | |
| ExecStreamId | id |
| Identifier for this stream; local to its containing graph. | |
| std::string | name |
| Name of stream, as known by optimizer. | |
| SharedDynamicParamManager | pDynamicParamManager |
| The dynamic parameter manager available to this stream. | |
| SharedLogicalTxn | pTxn |
| The transaction embracing the stream. | |
| ExecStreamResourceQuantity | resourceAllocation |
| Resource quantities currently allocated to this stream. | |
| SharedCacheAccessor | pQuotaAccessor |
| CacheAccessor used for quota tracking. | |
| SharedCacheAccessor | pScratchQuotaAccessor |
| CacheAccessor used for scratch page quota tracking. | |
| bool | needsClose |
| Private Member Functions | |
| ExecStreamResult | generateSingleKeyBitmaps (ExecStreamQuantum const &quantum) |
| Generates bitmaps for a single column index. | |
| ExecStreamResult | generateMultiKeyBitmaps (ExecStreamQuantum const &quantum) |
| Generates bitmaps for multi-column indexes. | |
| void | createSingletonBitmapEntry () |
| Creates a singleton bitmap entry and resets state to indicate that the current row has been processed if successful. | |
| bool | generateBitmaps () |
| Reads values from a single batch corresponding to a compressed batch and generates bitmap entries. | |
| bool | generateSingletons () |
| Reads values from a single batch corresponding to a non-compressed batch and generates singleton bitmap entries. | |
| bool | advanceReader (SharedLcsClusterReader &pScan) |
| Advances a single cluster and its corresponding column readers one rid forward in the current batch. | |
| bool | initBitmapTable (uint nEntries) |
| Initializes bitmap table, increasing the size, as needed and assigns buffers to bitmap entries. | |
| void | initRidAndBitmap (TupleData &bitmapTuple, LcsRid *pCurrRid) |
| Sets rid in a bitmap tupledata. | |
| bool | addRidToBitmap (uint keycode, TupleData &keyvalue, LcsRid rid) |
| Adds rid to a bitmap entry being constructing for a specified key. | |
| PBuffer | flushBuffer (LcsRid addRid) |
| Flushes out an existing buffer currently in use by another LbmEntry. | |
| bool | flushTable (uint start) |
| Flushes out entire table of bitmap entries. | |
| bool | flushEntry (uint keycode) |
| Flushes a single entry in the bitmap table and resets the LbmEntry associated with the table entry. | |
| void | syncColumns (SharedLcsClusterReader &pScan) |
| Positions column readers based on new cluster reader position. | |
| bool | readColVals (SharedLcsClusterReader &pScan, TupleDataWithBuffer &tupleData, uint colStart) |
| Reads column values based on current position of cluster reader. | |
| virtual void | buildOutputProj (TupleProjection &outputProj, LcsRowScanBaseExecStreamParams const ¶ms) |
| Builds outputProj from params. | |
| virtual void | prepare (LcsRowScanBaseExecStreamParams const ¶ms) |
| virtual void | prepare (ConfluenceExecStreamParams const ¶ms) |
| virtual void | getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity) |
| virtual void | setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors) |
| Initializes the buffer accessors for inputs to this stream. | |
| virtual ExecStreamBufProvision | getInputBufProvision () const |
| Queries the BufferProvision which this stream requires of its inputs when consuming their tuples. | |
| Private Attributes | |
| uint | numMiscScratchPages |
| Number of misc scratch pages needed; excludes pages for generating bitmap entries. | |
| uint | maxNumScratchPages |
| Number of scratch pages to allocate for constructing bitmaps as determined by the resource governor. | |
| DynamicParamId | insertRowCountParamId |
| Dynamic parameter id used to pass along number of rows loaded. | |
| bool | createIndex |
| True if index is being loaded as part of a create index statement. | |
| std::vector< DynamicParamId > | parameterIds |
| Ordered list of dynamic parameter ids for the clusters that provide source data for this stream. | |
| uint | scratchPageSize |
| Size of a scratch page. | |
| uint | entrySize |
| Size of a bitmap entry buffer. | |
| uint | maxBitmapSize |
| Max size of a bitmap entry. | |
| uint | minBitmapSize |
| Min size of a bitmap entry. | |
| ClusterPageLock | scratchLock |
| Lock for scratch accessor. | |
| TupleData | inputTuple |
| Input tuple data. | |
| RecordNum | numRowsToLoad |
| Number of rows to load. | |
| RecordNum | rowCount |
| Running count of number of rows read. | |
| LcsRid | startRid |
| Starting rid;. | |
| LcsRid | currRid |
| Current rid being loaded. | |
| TupleDataWithBuffer | bitmapTuple |
| Tuple data with buffer for the bitmap tuple. | |
| TupleDescriptor | bitmapTupleDesc |
| Tuple descriptor representing bitmap tuple. | |
| TupleData | outputTuple |
| Pointer to generated tuple data. | |
| uint | nIdxKeys |
| Number of keys in the bitmap index, excluding the starting rid. | |
| bool | batchRead |
| True if current batch has been read. | |
| uint | currBatch |
| Current batch entry being processed. | |
| std::vector< uint16_t > | keyCodes |
| Keycodes read from a batch. | |
| std::vector< LbmEntryInfo > | bitmapTable |
| Table of bitmap entries under construction. | |
| uint | nBitmapEntries |
| Number of entries in the bitmap table. | |
| uint | flushIdx |
| Index of buffer entry to flush. | |
| uint | nBitmapBuffers |
| Number of entries in the bitmap buffer table. | |
| uint | nScratchPagesAllocated |
| Number of scratch pages allocated. | |
| std::vector< PBuffer > | scratchPages |
| Vector of pointers to scratch pages allocated. | |
| LbmPendingProduceType | producePending |
| Produce of one or more output tuples pending: LBM_TABLEFLUSH_PENDING, LBM_ENTRYFLUSH_PENDING, LBM_FINALFLUSH_PENDING, LBM_NOFLUSH_PENDING. | |
| uint | flushStart |
| Index into bitmap table from which to start a pending table flush or the single entry currently being flushed. | |
| bool | skipRead |
| If true, skip the initial read the next time generator is called since we haven't finished processing the current rowid. | |
| bool | doneReading |
| If true, all rows from the column store table have been read. | |
| bool | revertToSingletons |
| If true, even though the current batch is compressed and the index has a single key, generate singleton bitmaps for the batch due to lack of buffer space. | |
| VectorOfUint | projMap |
| Projection map that maps columns read from cluster to their position in the output projection. | |
| uint | nClusters |
| Number of clusters to be scanned. | |
| boost::scoped_array< SharedLcsClusterReader > | pClusters |
| Array containing cluster readers. | |
| TupleDescriptor | projDescriptor |
| Tuple descriptor representing columns to be projected from scans. | |
| std::vector< int > | nonClusterCols |
| List of the non-cluster columns that need to be projected. | |
| bool | allSpecial |
| True in the special case where we are only reading special columns. | |
| CircularBuffer< LcsRidRun > | ridRuns |
| Circular buffer of rid runs. | |
| std::vector< UnalignedAttributeAccessor > | attrAccessors |
| Accessors used for loading actual column values. | |
| std::vector< SharedExecStreamBufAccessor > | inAccessors |
Detailed Description
LbmGeneratorExecStream reads the column store clusters corresponding to the columns of an index, and creates bitmap entries for those rows.
Its input is a rowcount, indicating the number of rows it needs to create entries for, and the startRid corresponding to the first row. It's assumed that the rows it needs to create entries for are all sequential, starting at that start value.
If possible, LbmGeneratorExecStream will try to take advantage of compressed column store batches to create bitmap entries spanning multiple rows.
As output, LbmGeneratorExecStream writes out the bitmap entries it has created. It also passes along the rowcount passed in as its input, using a dynamic parameter that will be read downstream.
Version:
Id
//open/dev/fennel/lucidera/bitmap/LbmGeneratorExecStream.h#15
Definition at line 92 of file LbmGeneratorExecStream.h.
Member Function Documentation
Generates bitmaps for multi-column indexes.
Parameters:
| quantum | quantum for stream |
|---|
Returns:
either EXECRC_BUF_OVERFLOW or EXECRC_EOS
Definition at line 290 of file LbmGeneratorExecStream.cpp.
References bitmapTuple, createSingletonBitmapEntry(), currRid, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, flushEntry(), LcsRowScanBaseExecStream::nClusters, ExecStreamQuantum::nTuplesMax, opaqueToInt(), LcsRowScanBaseExecStream::pClusters, LcsRowScanBaseExecStream::readColVals(), TupleDataWithBuffer::resetBuffer(), skipRead, and LcsRowScanBaseExecStream::syncColumns().
Referenced by execute().
| void LbmGeneratorExecStream::createSingletonBitmapEntry | ( | | ) | [private] | | ------------------------------------------------------- | - | | - | ----------- |
| bool LbmGeneratorExecStream::generateBitmaps | ( | | ) | [private] | | -------------------------------------------- | - | | - | ----------- |
Reads values from a single batch corresponding to a compressed batch and generates bitmap entries.
Returns:
true if successfully processed batch; false if overflow occurred while writing out tuples
Definition at line 365 of file LbmGeneratorExecStream.cpp.
References addRidToBitmap(), LcsRowScanBaseExecStream::attrAccessors, batchRead, bitmapTuple, currBatch, currRid, flushTable(), generateSingletons(), LcsColumnReader::getBatchValCount(), LcsColumnReader::getBatchValue(), initBitmapTable(), initRidAndBitmap(), keyCodes, LcsRowScanBaseExecStream::pClusters, LcsColumnReader::readCompressedBatch(), TupleDataWithBuffer::resetBuffer(), revertToSingletons, rowCount, and skipRead.
Referenced by generateSingleKeyBitmaps().
| bool LbmGeneratorExecStream::generateSingletons | ( | | ) | [private] | | ----------------------------------------------- | - | | - | ----------- |
Reads values from a single batch corresponding to a non-compressed batch and generates singleton bitmap entries.
Returns:
true if successfully processed batch; false if overflow occurred while writing out tuples
Definition at line 428 of file LbmGeneratorExecStream.cpp.
References advanceReader(), LcsRowScanBaseExecStream::attrAccessors, batchRead, bitmapTuple, createSingletonBitmapEntry(), flushEntry(), LcsRowScanBaseExecStream::pClusters, LcsRowScanBaseExecStream::projMap, TupleDataWithBuffer::resetBuffer(), and skipRead.
Referenced by generateBitmaps(), and generateSingleKeyBitmaps().
| bool LbmGeneratorExecStream::initBitmapTable | ( | uint | nEntries | ) | [private] |
|---|
Initializes bitmap table, increasing the size, as needed and assigns buffers to bitmap entries.
Parameters:
| nEntries | desired size of the table |
|---|
Returns:
true if the table was successfully initialized, indicating that there was enough space to accomodate the number of keys in the current batch
Definition at line 482 of file LbmGeneratorExecStream.cpp.
References SegNodeLock< Node >::allocatePage(), bitmapTable, entrySize, flushIdx, SegPageLock::getPage(), CachePage::getWritableData(), maxBitmapSize, maxNumScratchPages, minBitmapSize, nBitmapEntries, nScratchPagesAllocated, scratchLock, scratchPages, and scratchPageSize.
Referenced by execute(), and generateBitmaps().
| void LbmGeneratorExecStream::initRidAndBitmap | ( | TupleData & | bitmapTuple, |
|---|---|---|---|
| LcsRid * | pCurrRid | ||
| ) | [private] |
| bool LbmGeneratorExecStream::addRidToBitmap | ( | uint | keycode, |
|---|---|---|---|
| TupleData & | keyvalue, | ||
| LcsRid | rid | ||
| ) | [private] |
Adds rid to a bitmap entry being constructing for a specified key.
Creates a new bitmap entry, as needed, and flushes out entries as buffer space fills out.
Parameters:
| keycode | keycode corresponding to the value of the rid being added; always 0 in the case of a batch that is either non-compressed or has more than 1 key |
|---|---|
| keyvalue | tupledata containing the initial bitmap entry value; i.e., only the key value and rid are set |
| rid | rid to be added |
Returns:
false if an error was encountered flushing out a bitmap entry
Definition at line 571 of file LbmGeneratorExecStream.cpp.
References bitmapTable, bitmapTupleDesc, entrySize, flushBuffer(), flushEntry(), and nBitmapEntries.
Referenced by createSingletonBitmapEntry(), and generateBitmaps().
| PBuffer LbmGeneratorExecStream::flushBuffer | ( | LcsRid | addRid | ) | [private] |
|---|
| bool LbmGeneratorExecStream::flushTable | ( | uint | start | ) | [private] |
|---|
| bool LbmGeneratorExecStream::flushEntry | ( | uint | keycode | ) | [private] |
|---|
Flushes a single entry in the bitmap table and resets the LbmEntry associated with the table entry.
Parameters:
| keycode | index of the bitmap entry to be flushed |
|---|
Returns:
true if entries was successfully written to output stream
Definition at line 667 of file LbmGeneratorExecStream.cpp.
References bitmapTable, flushStart, LBM_ENTRYFLUSH_PENDING, outputTuple, SingleOutputExecStream::pOutAccessor, producePending, LbmEntry::toString(), and TRACE_FINE.
Referenced by addRidToBitmap(), flushBuffer(), flushTable(), generateMultiKeyBitmaps(), and generateSingletons().
Definition at line 29 of file LbmGeneratorExecStream.cpp.
References SegPageLock::accessSegment(), LcsRowScanBaseExecStream::attrAccessors, bitmapTuple, bitmapTupleDesc, TupleData::compute(), TupleDataWithBuffer::computeAndAllocate(), LbmGeneratorExecStreamParams::createIndex, createIndex, LbmEntry::getSizeBounds(), ConfluenceExecStream::inAccessors, inputTuple, LbmGeneratorExecStreamParams::insertRowCountParamId, insertRowCountParamId, BTreeDescriptor::keyProjection, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, maxBitmapSize, minBitmapSize, LcsRowScanBaseExecStream::nClusters, nIdxKeys, opaqueToInt(), parameterIds, SingleOutputExecStream::pOutAccessor, LcsRowScanBaseExecStream::prepare(), BTreeExecStream::prepare(), SegmentAccessor::pSegment, CircularBuffer< T >::resize(), LcsRowScanBaseExecStream::ridRuns, BTreeExecStream::scratchAccessor, scratchLock, scratchPageSize, BTreeDescriptor::segmentAccessor, BTreeExecStream::treeDescriptor, and BTreeDescriptor::tupleDescriptor.
| void LbmGeneratorExecStream::open | ( | bool | restart | ) | [virtual] |
|---|
Opens this stream, acquiring any resources needed in order to be able to fetch data.
A precondition is that input streams must already be opened. A stream can be closed and reopened.
Parameters:
| restart | if true, the stream must be already open, and should reset itself to start from the beginning of its result set |
|---|
Reimplemented from BTreeExecStream.
Definition at line 75 of file LbmGeneratorExecStream.cpp.
References batchRead, CircularBuffer< T >::clear(), doneReading, flushIdx, ConfluenceExecStream::inAccessors, insertRowCountParamId, LBM_NOFLUSH_PENDING, nBitmapBuffers, nBitmapEntries, LcsRowScanBaseExecStream::nClusters, nScratchPagesAllocated, opaqueToInt(), LcsRowScanBaseExecStream::open(), BTreeExecStream::open(), parameterIds, LcsRowScanBaseExecStream::pClusters, ExecStream::pDynamicParamManager, producePending, revertToSingletons, LcsRowScanBaseExecStream::ridRuns, rowCount, and skipRead.
Executes this stream.
Parameters:
| quantum | governs the maximum amount of execution to perform |
|---|
Returns:
code indicating reason execution ceased
Implements ExecStream.
Definition at line 145 of file LbmGeneratorExecStream.cpp.
References bitmapTable, createIndex, currRid, doneReading, EXECBUF_EOS, EXECRC_BUF_OVERFLOW, EXECRC_BUF_UNDERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, flushStart, flushTable(), generateMultiKeyBitmaps(), generateSingleKeyBitmaps(), ConfluenceExecStream::inAccessors, initBitmapTable(), inputTuple, insertRowCountParamId, LBM_ENTRYFLUSH_PENDING, LBM_NOFLUSH_PENDING, LBM_TABLEFLUSH_PENDING, MAXU, LcsRowScanBaseExecStream::nClusters, nIdxKeys, LcsRidRun::nRids, numRowsToLoad, outputTuple, LcsRowScanBaseExecStream::pClusters, ExecStream::pDynamicParamManager, SingleOutputExecStream::pOutAccessor, producePending, CircularBuffer< T >::push_back(), LcsRowScanBaseExecStream::ridRuns, rowCount, LcsRidRun::startRid, startRid, and LcsRowScanBaseExecStream::syncColumns().
| void LbmGeneratorExecStream::closeImpl | ( | | ) | [virtual] | | -------------------------------------- | - | | - | ----------- |
| SharedBTreeWriter BTreeExecStream::newWriter | ( | bool | monotonic = false | ) | [protected, inherited] |
|---|
| void BTreeExecStream::endSearch | ( | | ) | [protected, virtual, inherited] | | ------------------------------- | - | | - | --------------------------------- |
Definition at line 30 of file BTreeExecStream.cpp.
References BTreeExecStream::copyParamsToDescriptor(), ExecStreamParams::pCacheAccessor, SingleOutputExecStream::prepare(), BTreeParams::pRootMap, BTreeExecStream::pRootMap, BTreeParams::rootPageIdParamId, BTreeExecStream::rootPageIdParamId, ExecStreamParams::scratchAccessor, BTreeExecStream::scratchAccessor, and BTreeExecStream::treeDescriptor.
Referenced by LcsClusterAppendExecStream::prepare(), prepare(), BTreeReadExecStream::prepare(), and BTreeInsertExecStream::prepare().
Definition at line 48 of file SingleOutputExecStream.cpp.
References SingleOutputExecStream::getOutputBufProvision(), SingleOutputExecStreamParams::outputTupleDesc, SingleOutputExecStreamParams::outputTupleFormat, SingleOutputExecStream::pOutAccessor, and ExecStream::prepare().
Referenced by BTreeExecStream::prepare(), FlatFileExecStreamImpl::prepare(), ValuesExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), ConfluenceExecStream::prepare(), and ConduitExecStream::prepare().
| void ExecStream::prepare | ( | ExecStreamParams const & | params | ) | [virtual, inherited] |
|---|
Prepares this stream for execution.
A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.
Parameters:
| params | instance of stream parameterization class which should be used to prepare this stream |
|---|
Definition at line 84 of file ExecStream.cpp.
References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.
Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().
Definition at line 93 of file BTreeExecStream.cpp.
References BTreeParams::keyProj, BTreeDescriptor::keyProjection, BTreeParams::pageOwnerId, BTreeDescriptor::pageOwnerId, SegmentAccessor::pCacheAccessor, BTreeParams::pSegment, SegmentAccessor::pSegment, BTreeParams::rootPageId, BTreeDescriptor::rootPageId, BTreeDescriptor::segmentAccessor, BTreeParams::segmentId, BTreeDescriptor::segmentId, BTreeExecStream::treeDescriptor, BTreeParams::tupleDesc, and BTreeDescriptor::tupleDescriptor.
Referenced by BTreeExecStream::newWriter(), LbmSplicerExecStream::prepare(), and BTreeExecStream::prepare().
| void SingleOutputExecStream::setInputBufAccessors | ( | std::vector< SharedExecStreamBufAccessor > const & | inAccessors | ) | [virtual, inherited] |
|---|
Initializes the buffer accessors for inputs to this stream.
This method is only ever called once, before prepare.
Parameters:
| inAccessors | buffer accessors ordered by input stream |
|---|
Implements ExecStream.
Reimplemented in ConduitExecStream, and ConfluenceExecStream.
Definition at line 35 of file SingleOutputExecStream.cpp.
00037 { 00038 assert(inAccessors.size() == 0); 00039 }
| void SingleOutputExecStream::setOutputBufAccessors | ( | std::vector< SharedExecStreamBufAccessor > const & | outAccessors | ) | [virtual, inherited] |
|---|
| bool ExecStream::canEarlyClose | ( | | ) | [virtual, inherited] | | ------------------------------ | - | | - | ---------------------- |
| ExecStreamId ExecStream::getStreamId | ( | | ) | const [inline, inherited] | | -------------------------------------------------------------------------------------------------- | - | | - | --------------------------- |
Returns:
the identifier for this stream within containing graph
Definition at line 288 of file ExecStream.h.
References ExecStream::id.
Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().
00289 { 00290 return id; 00291 }
Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, FlatFileExecStreamImpl, BTreeInsertExecStream, BTreeReadExecStream, FtrsTableWriterExecStream, LbmChopperExecStream, LbmSplicerExecStream, LcsClusterAppendExecStream, LcsClusterReplaceExecStream, LcsRowScanBaseExecStream, and LcsRowScanExecStream.
Definition at line 102 of file ExecStream.cpp.
References ExecStreamResourceQuantity::nCachePages, and ExecStreamResourceQuantity::nThreads.
00105 { 00106 minQuantity.nThreads = 0; 00107 minQuantity.nCachePages = 0; 00108 optQuantity = minQuantity; 00109 }
| void ExecStream::setName | ( | std::string const & | | ) | [virtual, inherited] | | ------------------------ | - | ------------------- | | - | ---------------------- |
| std::string const & ExecStream::getName | ( | | ) | const [virtual, inherited] | | --------------------------------------- | - | | - | ---------------------------- |
| bool ExecStream::mayBlock | ( | | ) | const [virtual, inherited] | | ------------------------- | - | | - | ---------------------------- |
Queries whether this stream's implementation may block when execute() is called.
For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.
Returns:
whether stream may block; default is false
Definition at line 167 of file ExecStream.cpp.
00168 { 00169 return false; 00170 }
| void ExecStream::checkAbort | ( | | ) | const [virtual, inherited] | | --------------------------- | - | | - | ---------------------------- |
| bool ClosableObject::isClosed | ( | | ) | const [inline, inherited] | | ----------------------------- | - | | - | --------------------------- |
Returns:
whether the object has been closed
Definition at line 58 of file ClosableObject.h.
| void ClosableObject::close | ( | | ) | [inherited] | | -------------------------- | - | | - | ------------- |
Closes this object, releasing any unallocated resources.
Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.
Definition at line 39 of file ClosableObject.cpp.
References ClosableObject::closeImpl(), and ClosableObject::needsClose.
Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().
| void TraceSource::initTraceSource | ( | SharedTraceTarget | pTraceTarget, |
|---|---|---|---|
| std::string | name | ||
| ) | [virtual, inherited] |
| void TraceSource::trace | ( | TraceLevel | level, |
|---|---|---|---|
| std::string | message | ||
| ) | const [inherited] |
| bool TraceSource::isTracing | ( | | ) | const [inline, inherited] | | --------------------------- | - | | - | --------------------------- |
| bool TraceSource::isTracingLevel | ( | TraceLevel | level | ) | const [inline, inherited] |
|---|
| TraceTarget& TraceSource::getTraceTarget | ( | | ) | const [inline, inherited] | | ----------------------------------------------------------------- | - | | - | --------------------------- |
| std::string TraceSource::getTraceSourceName | ( | | ) | const [inline, inherited] | | ------------------------------------------- | - | | - | --------------------------- |
| void TraceSource::setTraceSourceName | ( | std::string const & | n | ) | [inline, inherited] |
|---|
Sets the name of this source.
Useful to construct dynamic names for fine-grained filtering.
Definition at line 136 of file TraceSource.h.
00137 { 00138 name = n; 00139 }
| TraceLevel TraceSource::getMinimumTraceLevel | ( | | ) | const [inline, inherited] | | ------------------------------------------------------------------------------------------------------ | - | | - | --------------------------- |
| void TraceSource::disableTracing | ( | | ) | [inherited] | | -------------------------------- | - | | - | ------------- |
| void ErrorSource::initErrorSource | ( | SharedErrorTarget | pErrorTarget, |
|---|---|---|---|
| const std::string & | name | ||
| ) | [virtual, inherited] |
| void ErrorSource::postError | ( | ErrorLevel | level, |
|---|---|---|---|
| const std::string & | message, | ||
| void * | address, | ||
| long | capacity, | ||
| int | index | ||
| ) | [inherited] |
| bool ErrorSource::hasTarget | ( | | ) | const [inline, inherited] | | --------------------------- | - | | - | --------------------------- |
| ErrorTarget& ErrorSource::getErrorTarget | ( | | ) | const [inline, inherited] | | ----------------------------------------------------------------- | - | | - | --------------------------- |
Returns:
the SharedErrorTarget for this source
Definition at line 129 of file ErrorSource.h.
| std::string ErrorSource::getErrorSourceName | ( | | ) | const [inline, inherited] | | ------------------------------------------- | - | | - | --------------------------- |
Gets the name of this source.
Useful to construct nested names for subcomponents that are also ErrorSources.
Returns:
the name
Definition at line 139 of file ErrorSource.h.
00140 { 00141 return name; 00142 }
| void ErrorSource::setErrorSourceName | ( | std::string const & | n | ) | [inline, inherited] |
|---|
Sets the name of this source.
Useful to construct dynamic names for fine-grained filtering.
Definition at line 148 of file ErrorSource.h.
00149 { 00150 name = n; 00151 }
| void ErrorSource::disableTarget | ( | | ) | [inherited] | | ------------------------------- | - | | - | ------------- |
Definition at line 33 of file LcsRowScanBaseExecStream.cpp.
References LcsRowScanBaseExecStream::allSpecial, LcsRowScanBaseExecStream::buildOutputProj(), BTreeParams::keyProj, BTreeDescriptor::keyProjection, LCS_RID_COLUMN_ID, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, LcsRowScanBaseExecStream::nClusters, LcsRowScanBaseExecStream::nonClusterCols, LcsRowScanBaseExecStreamParams::outputProj, BTreeParams::pageOwnerId, BTreeDescriptor::pageOwnerId, ExecStreamParams::pCacheAccessor, SegmentAccessor::pCacheAccessor, LcsRowScanBaseExecStream::pClusters, SingleOutputExecStream::pOutAccessor, ConfluenceExecStream::prepare(), LcsRowScanBaseExecStream::projDescriptor, LcsRowScanBaseExecStream::projMap, BTreeParams::pSegment, SegmentAccessor::pSegment, LcsRowScanBaseExecStream::ridRuns, BTreeParams::rootPageId, BTreeDescriptor::rootPageId, BTreeDescriptor::segmentAccessor, BTreeParams::segmentId, BTreeDescriptor::segmentId, BTreeParams::tupleDesc, and BTreeDescriptor::tupleDescriptor.
Referenced by LcsRowScanExecStream::prepare(), and prepare().
Definition at line 37 of file ConfluenceExecStream.cpp.
References ConfluenceExecStream::getInputBufProvision(), ConfluenceExecStream::inAccessors, and SingleOutputExecStream::prepare().
Referenced by LcsRowScanBaseExecStream::prepare(), LbmUnionExecStream::prepare(), LbmChopperExecStream::prepare(), LbmBitOpExecStream::prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), CartesianJoinExecStream::prepare(), and BarrierExecStream::prepare().
| void ConfluenceExecStream::setInputBufAccessors | ( | std::vector< SharedExecStreamBufAccessor > const & | inAccessors | ) | [virtual, inherited] |
|---|
Member Data Documentation
Definition at line 113 of file BTreeExecStream.h.
Referenced by BTreeInsertExecStream::buildTree(), BTreeExecStream::closeImpl(), BTreeExecStream::copyParamsToDescriptor(), LcsClusterReplaceExecStream::getTupleForLoad(), LcsClusterAppendExecStream::initLoad(), LcsClusterReplaceExecStream::initTupleLoadParams(), BTreePrefetchSearchExecStream::innerSearchLoop(), BTreePrefetchSearchExecStream::newReader(), BTreeExecStream::newReader(), BTreeExecStream::newWriter(), LcsClusterReplaceExecStream::open(), BTreeSearchExecStream::open(), BTreePrefetchSearchExecStream::open(), BTreeInsertExecStream::open(), BTreeExecStream::open(), LcsClusterReplaceExecStream::prepare(), LcsClusterAppendExecStream::prepare(), LbmSearchExecStream::prepare(), prepare(), BTreeSearchExecStream::prepare(), BTreeInsertExecStream::prepare(), BTreeExecStream::prepare(), and BTreeInsertExecStream::truncateTree().
Definition at line 56 of file SingleOutputExecStream.h.
Referenced by LcsClusterAppendExecStream::compress(), ExternalSortExecStreamImpl::execute(), LcsRowScanExecStream::execute(), LbmUnionExecStream::execute(), LbmNormalizerExecStream::execute(), execute(), LbmChopperExecStream::execute(), LhxJoinExecStream::execute(), LhxAggExecStream::execute(), FtrsTableWriterExecStream::execute(), BTreeSortExecStream::execute(), BTreeSearchUniqueExecStream::execute(), BTreeScanExecStream::execute(), BTreePrefetchSearchExecStream::execute(), BTreeInsertExecStream::execute(), FlatFileExecStreamImpl::execute(), ValuesExecStream::execute(), UncollectExecStream::execute(), SortedAggExecStream::execute(), SegBufferReaderExecStream::execute(), ScratchBufferExecStream::execute(), ReshapeExecStream::execute(), MockResourceExecStream::execute(), MockProducerExecStream::execute(), MergeExecStream::execute(), DoubleBufferExecStream::execute(), CorrelationJoinExecStream::execute(), CopyExecStream::execute(), CollectExecStream::execute(), CartesianJoinExecStream::execute(), BernoulliSamplingExecStream::execute(), BarrierExecStream::execute(), CalcExecStream::execute(), flushEntry(), MockProducerExecStream::getProducedRowCount(), ExternalSortExecStreamImpl::getResourceRequirements(), BTreeSearchExecStream::innerFetchLoop(), LbmUnionExecStream::open(), LbmChopperExecStream::open(), LbmBitOpExecStream::open(), SingleOutputExecStream::open(), SegBufferReaderExecStream::open(), SegBufferExecStream::open(), ScratchBufferExecStream::open(), DoubleBufferExecStream::open(), CollectExecStream::open(), SegBufferExecStream::openBufferForRead(), ConduitExecStream::precheckConduitBuffers(), LcsRowScanExecStream::prepare(), LcsRowScanBaseExecStream::prepare(), LcsClusterAppendExecStream::prepare(), LbmUnionExecStream::prepare(), prepare(), LhxJoinExecStream::prepare(), LhxAggExecStream::prepare(), FtrsTableWriterExecStream::prepare(), FlatFileExecStreamImpl::prepare(), UncollectExecStream::prepare(), SortedAggExecStream::prepare(), SingleOutputExecStream::prepare(), ReshapeExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), ConduitExecStream::prepare(), CollectExecStream::prepare(), CartesianJoinExecStream::prepare(), BernoulliSamplingExecStream::prepare(), BarrierExecStream::prepare(), CalcExecStream::prepare(), SortedAggExecStream::produce(), LbmBitOpExecStream::producePendingOutput(), LbmUnionExecStream::produceTuple(), LbmNormalizerExecStream::produceTuple(), LbmMinusExecStream::produceTuple(), LbmChopperExecStream::produceTuple(), LbmBitOpExecStream::produceTuple(), LbmBitOpExecStream::readInput(), LbmMinusExecStream::readMinuendInput(), and SingleOutputExecStream::setOutputBufAccessors().
Dataflow graph containing this stream.
Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.
Definition at line 68 of file ExecStream.h.
Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().
The dynamic parameter manager available to this stream.
(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())
Definition at line 85 of file ExecStream.h.
Referenced by CorrelationJoinExecStream::close(), SegBufferReaderExecStream::closeImpl(), LbmUnionExecStream::execute(), LbmSplicerExecStream::execute(), execute(), LbmChopperExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), LcsClusterReplaceExecStream::open(), LbmUnionExecStream::open(), LbmSplicerExecStream::open(), open(), LbmBitOpExecStream::open(), BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), SegBufferWriterExecStream::open(), SegBufferReaderExecStream::open(), NestedLoopJoinExecStream::open(), CorrelationJoinExecStream::open(), ExecStream::prepare(), CalcExecStream::prepare(), NestedLoopJoinExecStream::processLeftInput(), LbmSearchExecStream::reachedTupleLimit(), ReshapeExecStream::readDynamicParams(), SegBufferWriterExecStream::readReaderRefCount(), BTreeSearchExecStream::readSearchKey(), BTreeSearchExecStream::readUpperBoundKey(), LbmSearchExecStream::setAdditionalKeys(), and LbmBitOpExecStream::writeStartRidParamValue().
Number of clusters to be scanned.
Definition at line 82 of file LcsRowScanBaseExecStream.h.
Referenced by LcsRowScanBaseExecStream::closeImpl(), LcsRowScanExecStream::execute(), execute(), generateMultiKeyBitmaps(), LcsRowScanBaseExecStream::getResourceRequirements(), LcsRowScanBaseExecStream::LcsRowScanBaseExecStream(), LcsRowScanBaseExecStream::open(), open(), LcsRowScanBaseExecStream::prepare(), prepare(), and LcsRowScanExecStream::prepareResidualFilters().
Array containing cluster readers.
Definition at line 87 of file LcsRowScanBaseExecStream.h.
Referenced by LcsRowScanBaseExecStream::closeImpl(), LcsRowScanExecStream::execute(), execute(), generateBitmaps(), generateMultiKeyBitmaps(), generateSingleKeyBitmaps(), generateSingletons(), LcsRowScanBaseExecStream::open(), open(), LcsRowScanBaseExecStream::prepare(), and LcsRowScanExecStream::prepareResidualFilters().
Definition at line 50 of file ConfluenceExecStream.h.
Referenced by NestedLoopJoinExecStream::checkNumInputs(), CartesianJoinExecStream::checkNumInputs(), LbmMinusExecStream::comparePrefixes(), execute(), MergeExecStream::execute(), BarrierExecStream::execute(), LbmMinusExecStream::findMinInput(), LcsRowScanExecStream::initializeFiltersIfNeeded(), LcsRowScanExecStream::open(), LbmUnionExecStream::open(), LbmMinusExecStream::open(), open(), LbmChopperExecStream::open(), LbmBitOpExecStream::open(), ConfluenceExecStream::open(), LcsRowScanExecStream::prepare(), LbmUnionExecStream::prepare(), LbmMinusExecStream::prepare(), prepare(), LbmChopperExecStream::prepare(), LbmBitOpExecStream::prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), ConfluenceExecStream::prepare(), CartesianJoinExecStream::prepare(), BarrierExecStream::prepare(), NestedLoopJoinExecStream::preProcessRightInput(), BarrierExecStream::processInputTuple(), LbmBitOpExecStream::producePendingOutput(), LbmMinusExecStream::restartSubtrahends(), LhxJoinExecStream::setHashInfo(), and ConfluenceExecStream::setInputBufAccessors().
The documentation for this class was generated from the following files:
- /home/pub/open/dev/fennel/lucidera/bitmap/LbmGeneratorExecStream.h
- /home/pub/open/dev/fennel/lucidera/bitmap/LbmGeneratorExecStream.cpp
