Fennel: LbmSplicerExecStream Class Reference (original) (raw)

LbmSplicerExecStream takes as input a stream of bitmap entries. More...

#include <[LbmSplicerExecStream.h](LbmSplicerExecStream%5F8h-source.html)>

Inheritance diagram for LbmSplicerExecStream:

List of all members.

Public Member Functions
virtual void prepare (LbmSplicerExecStreamParams const &params)
virtual void open (bool restart)
Opens this stream, acquiring any resources needed in order to be able to fetch data.
virtual ExecStreamResult execute (ExecStreamQuantum const &quantum)
Executes this stream.
virtual void getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void closeImpl ()
Implements ClosableObject.
virtual void prepare (DiffluenceExecStreamParams const &params)
virtual void prepare (SingleInputExecStreamParams const &params)
virtual void prepare (ExecStreamParams const &params)
Prepares this stream for execution.
virtual void setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors)
Initializes the buffer accessors for outputs from this stream.
virtual ExecStreamBufProvision getOutputBufProvision () const
Indicate to the consumer if the buffer is provided by this exec stream which is the producer.
virtual void setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
Initializes the buffer accessors for inputs to this stream.
virtual ExecStreamBufProvision getInputBufProvision () const
Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
virtual bool canEarlyClose ()
**Returns:**true if the stream can be closed early
ExecStreamGraph & getGraph () const
**Returns:**reference to containing graph
ExecStreamId getStreamId () const
**Returns:**the identifier for this stream within containing graph
virtual void getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType)
Determines resource requirements for this stream.
virtual void setResourceAllocation (ExecStreamResourceQuantity &quantity)
Sets current resource allocation for this stream.
virtual void setName (std::string const &)
Sets unique name of this stream.
virtual std::string const & getName () const
**Returns:**the name of this stream, as known by the optimizer
virtual bool mayBlock () const
Queries whether this stream's implementation may block when execute() is called.
virtual void checkAbort () const
Checks whether there is an abort request for this stream's scheduler.
virtual ExecStreamBufProvision getOutputBufConversion () const
Queries the BufferProvision to which this stream needs its output to be converted, if any.
bool isClosed () const
**Returns:**whether the object has been closed
void close ()
Closes this object, releasing any unallocated resources.
virtual void initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
For use when initialization has to be deferred until after construction.
void trace (TraceLevel level, std::string message) const
Records a trace message.
bool isTracing () const
**Returns:**true iff tracing is enabled for this source
bool isTracingLevel (TraceLevel level) const
Determines whether a particular level is being traced.
TraceTarget & getTraceTarget () const
**Returns:**the TraceTarget for this source
SharedTraceTarget getSharedTraceTarget () const
**Returns:**the SharedTraceTarget for this source
std::string getTraceSourceName () const
Gets the name of this source.
void setTraceSourceName (std::string const &n)
Sets the name of this source.
TraceLevel getMinimumTraceLevel () const
void disableTracing ()
virtual void initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name)
For use when initialization has to be deferred until after construction.
void postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index)
Posts an exception, such as a row exception.
void postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index)
Posts an exception, such as a row exception.
bool hasTarget () const
**Returns:**true iff an error target has been set
ErrorTarget & getErrorTarget () const
**Returns:**the ErrorTarget for this source
SharedErrorTarget getSharedErrorTarget () const
**Returns:**the SharedErrorTarget for this source
std::string getErrorSourceName () const
Gets the name of this source.
void setErrorSourceName (std::string const &n)
Sets the name of this source.
void disableTarget ()
Protected Attributes
std::vector< SharedExecStreamBufAccessor > outAccessors
List of output buffer accessors.
TupleDescriptor outputTupleDesc
Output tuple descriptor.
SharedExecStreamBufAccessor pInAccessor
bool isOpen
Whether this stream is currently open.
ExecStreamGraph * pGraph
Dataflow graph containing this stream.
ExecStreamId id
Identifier for this stream; local to its containing graph.
std::string name
Name of stream, as known by optimizer.
SharedDynamicParamManager pDynamicParamManager
The dynamic parameter manager available to this stream.
SharedLogicalTxn pTxn
The transaction embracing the stream.
ExecStreamResourceQuantity resourceAllocation
Resource quantities currently allocated to this stream.
SharedCacheAccessor pQuotaAccessor
CacheAccessor used for quota tracking.
SharedCacheAccessor pScratchQuotaAccessor
CacheAccessor used for scratch page quota tracking.
bool needsClose
Private Member Functions
bool isEmpty ()
Determines, and remembers whether the bitmap being updated is empty.
bool existingEntry (TupleData const &bitmapEntry)
Determines whether a bitmap entry already exists in the bitmap index, based on the index key values.
bool findMatchingBTreeEntry (TupleData const &bitmapEntry, TupleData &bTreeTupleData, bool leastUpper)
Searches the btree, looking for the btree record matching the index keys and startRid of a specified bitmap entry.
bool findBTreeEntry (TupleData const &bitmapEntry, TupleData &bTreeTupleData)
Searches the btree, looking for the first btree record which overlaps with a given bitmap entry (or the insertion point for a new one if no existing match exists).
bool ridOverlaps (LcsRid rid, TupleData &bitmapTupleData, bool firstByte)
Determines whether a rid intersects the rid range spanned by a bitmap entry.
void findBetterEntry (TupleData const &bitmapEntry)
Determines if there exists a better entry in the btree, corresponding to the bitmap entry passed in, as compared to whatever is the current bitmap entry.
void spliceEntry (TupleData &bitmapEntry)
Splices a bitmap entry to the current entry under construction.
void insertBitmapEntry ()
Inserts current bitmap entry under construction into the bitmap index.
void createNewBitmapEntry (TupleData const &bitmapEntry)
Creates a new current bitmap entry for construction.
void upsertSingleton (TupleData const &bitmapEntry)
Efficient update or insert of tuples into unique indexes.
ExecStreamResult getValidatedTuple ()
Reads input tuples and filters out Rids that cause key violations.
bool uniqueRequired (const TupleData &tuple)
Determines whether a tuple key must be unique.
uint countKeyRows (const TupleData &tuple)
Counts the number of rows in the index with a particular key value, prior to modification.
void setUpsertRid (LcsRid rid)
For key values that can only have one valid rid, remembers the value of a single rid that was chosen to be for insert or update.
const LcsRid * getUpsertRidPtr () const
For key values that can only have one valid rid, gets a pointer to a rid to be inserted into the index.
void postViolation (const TupleData &input, const TupleData &violation)
Generates an error record and posts it to an ErrorTarget.
Private Attributes
SegmentAccessor scratchAccessor
Scratch accessor.
BTreeDescriptor writeBTreeDesc
Descriptor corresponding to the btree that splicer writes to.
BTreeDescriptor deletionBTreeDesc
Descriptor corresponding to the deletion index btree.
DynamicParamId insertRowCountParamId
Parameter id of dynamic parameter containing final insert row count.
bool createNewIndex
If true, create a new index that the splicer will be writing.
bool newIndexCreated
True if a new index was created.
DynamicParamId writeRowCountParamId
Parameter id of the dynamic parameter used to write the row count affected by this stream that will be read downstream.
uint maxEntrySize
Maximum size of an LbmEntry.
boost::scoped_array< FixedBuffer > bitmapBuffer
Buffer for LbmEntry.
boost::scoped_array< FixedBuffer > mergeBuffer
Buffer for merges on LbmEntry.
SharedLbmEntry pCurrentEntry
Current bitmap entry under construction.
bool currEntry
true if current entry is under construction
bool currExistingEntry
True if current bitmap entry under construction refers to an already existing entry in the bitmap index.
LcsRid currBTreeStartRid
Start rid of the current existing entry in the bitmap index.
bool emptyTableUnknown
True if we need to determine emptyTable.
bool emptyTable
True if table that the bitmap is being constructed on was empty to begin with.
SharedBTreeWriter bTreeWriter
Writes btree index corresponding to bitmap.
bool bTreeWriterMoved
Whether btree writer position was moved for unique constraint validation.
bool isDone
True if output has been produced.
TupleData inputTuple
Input tuple.
TupleData singletonTuple
Input tuple corresponding to singleton entries.
TupleData outputTuple
Output tuple containing rowcount.
RecordNum numRowsLoaded
Number of rows loaded into bitmap index.
TupleData bTreeTupleData
Tuple data for reading bitmaps from btree index.
TupleData tempBTreeTupleData
TupleDescriptor bitmapTupleDesc
Tuple descriptor representing bitmap tuple.
uint nIdxKeys
Number of keys in the bitmap index, excluding the starting rid.
bool computeRowCount
True if splicer needs to compute the row count result rather than reading it from a dynamic parameter.
bool uniqueKey
Whether the index being updated is a unique key.
bool currValidation
Whether an input tuple is currently being validated.
bool firstValidation
Whether any tuple has been validated yet.
TupleDataWithBuffer currUniqueKey
The current unique key value being validated.
uint nKeyRows
Number of rows for the current key value (after taking into account the deletion index).
LbmDeletionIndexReader deletionReader
Reads rids from the deletion index.
TupleData deletionTuple
Current tuple for deletion index rid reader.
LbmTupleRidReader inputRidReader
Reads rids from an input tuple.
bool nullUpsertRid
True if no RID has been accepted as the update/insert for the current key value.
LcsRid upsertRid
If a RID has been accepted as the update/insert for the current key value, this contains the value of the accepted RID.
SharedExecStreamBufAccessor violationAccessor
Accessor for the violation output stream.
TupleData violationTuple
Violation data tuple.
std::string errorMsg
Error message for constraint violations.
TupleDescriptor errorDesc
TupleDescriptor for error records.
TupleData errorTuple
TupleData used to build error records.
SnapshotRandomAllocationSegment * pSnapshotSegment
Underlying snapshot segment corresponding to the index that will be written.
PageId origRootPageId
The original root pageId of the index that will be written.

Detailed Description

LbmSplicerExecStream takes as input a stream of bitmap entries.

If possible, it splices the entries into larger entries before writing them into a btree. The bitmap entries from the input can correspond to new entries, or they may need to be spliced into existing entries in the btree.

As output, LbmSplicerExecStream writes out a count corresponding to the number of new row values inserted into the btree. This value is either directly computed by LbmSplicerExecStream (provided the bitmap entry inputs all correspond to singleton rids), or it is passed into LbmSplicerExecStream via a dynamic parameter.

Optionally, LbmSplicerExecStream may also write out the row count into a dynamic parameter that is read downstream by another exec stream.

If the index being updated is a primary key or a unique key, then multiple entries mapping to the same key value are rejected. When a new key value is encountered, undeleted entries are counted for that key value. If an entry already exists, then all inputs with the key value are rejected. Otherwise, if no entry exists, a single input RID is accepted as the valid insert (or update). (We deviate from the SQL standard.) In order to count existing entries, we read from the deletion index. Violations are written to a second output. Unique keys differ from primary keys in that multiple entries are allowed for null keys.

Version:

Id

//open/dev/fennel/lucidera/bitmap/LbmSplicerExecStream.h#16

Definition at line 78 of file LbmSplicerExecStream.h.

Member Function Documentation

| bool LbmSplicerExecStream::isEmpty | ( | | ) | [private] | | ---------------------------------- | - | | - | ----------- |

bool LbmSplicerExecStream::existingEntry ( TupleData const & bitmapEntry ) [private]

Determines whether a bitmap entry already exists in the bitmap index, based on the index key values.

If multiple entries have the same key value, locates the last entry with that key value, i.e., the one with the largest starting rid value. Avoids index lookup if table was empty at the start of the load. Sets current bitmap entry either to the existing btree entry (if it exists). Otherwise, sets it to the bitmap entry passed in.

Parameters:

bitmapEntry tupledata corresponding to bitmap entry being checked

Returns:

true if entry already exists in bitmap index; always return false if table was empty at the start of the load

Definition at line 302 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, bTreeWriterMoved, createNewBitmapEntry(), currExistingEntry, findBTreeEntry(), and isEmpty().

Referenced by execute().

bool LbmSplicerExecStream::findMatchingBTreeEntry ( TupleData const & bitmapEntry,
TupleData & bTreeTupleData,
bool leastUpper
) [private]

Searches the btree, looking for the btree record matching the index keys and startRid of a specified bitmap entry.

Parameters:

bitmapEntry entry for which we are trying to match
bTreeTupleData tuple data where the btree record will be returned if a matching entry is found; if a non-match is found, the tuple data contains the greatest lower bound btree entry found
leastUpper if true, use a least upper bound search to locate the btree record

Returns:

true if a matching entry is found in the btree; false otherwise; (in this case, bTreeWriter is positioned at the location of the greatest lower bound btree entry corresponding to the bitmap entry)

Definition at line 321 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, bTreeWriter, and DUP_SEEK_BEGIN.

Referenced by findBTreeEntry(), and insertBitmapEntry().

bool LbmSplicerExecStream::findBTreeEntry ( TupleData const & bitmapEntry,
TupleData & bTreeTupleData
) [private]

Searches the btree, looking for the first btree record which overlaps with a given bitmap entry (or the insertion point for a new one if no existing match exists).

Parameters:

bitmapEntry entry for which to find an overlap match
bTreeTupleData tuple data where the btree record will be returned if a matching entry is found (otherwise invalid data references are returned here after search)

Returns:

true if entry found in btree; false if no entry found (in this case, bTreeWriter is positioned to correct location for inserting new entry)

Definition at line 335 of file LbmSplicerExecStream.cpp.

References bitmapTupleDesc, bTreeTupleData, bTreeWriter, TupleDescriptor::compareTuplesKey(), DUP_SEEK_BEGIN, findMatchingBTreeEntry(), nIdxKeys, ridOverlaps(), and LbmSegment::roundToByteBoundary().

Referenced by countKeyRows(), existingEntry(), findBetterEntry(), and upsertSingleton().

00337 { 00338
00339
00340 bool match = 00341 findMatchingBTreeEntry(bitmapEntry, bTreeTupleData, (nIdxKeys > 0)); 00342 00343 if (match == false) { 00344 if (nIdxKeys == 0) { 00345
00346
00347
00348
00349
00350
00351
00352 LcsRid newRid = *reinterpret_cast<LcsRid const *> 00353 (bitmapEntry[0].pData); 00354 if (ridOverlaps(newRid, bTreeTupleData, false)) { 00355 match = bTreeWriter->searchNext(); 00356 if (match) { 00357 bTreeWriter->getTupleAccessorForRead().unmarshal( 00358 bTreeTupleData); 00359 if (ridOverlaps(newRid, bTreeTupleData, true)) { 00360 match = bTreeWriter->searchForKey( 00361 bitmapEntry, DUP_SEEK_BEGIN, false); 00362 assert(match == false); 00363 bTreeWriter->getTupleAccessorForRead().unmarshal( 00364 bTreeTupleData); 00365 } 00366 } 00367 } 00368 match = true; 00369 00370 } else { 00371
00372
00373
00374
00375
00376
00377
00378
00379
00380
00381 if (bTreeWriter->isSingular()) { 00382 int keyComp = 00383 bitmapTupleDesc.compareTuplesKey( 00384 bTreeTupleData, 00385 bitmapEntry, 00386 nIdxKeys); 00387 if (keyComp == 0) { 00388 assert( 00389 LbmSegment::roundToByteBoundary( 00390 *reinterpret_cast<LcsRid const *>( 00391 bTreeTupleData[nIdxKeys].pData)) == 00392 LbmSegment::roundToByteBoundary( 00393 *reinterpret_cast<LcsRid const *>( 00394 bitmapEntry[nIdxKeys].pData))); 00395 return true; 00396 } 00397 } 00398 00399
00400 match = 00401 bTreeWriter->searchForKey(bitmapEntry, DUP_SEEK_BEGIN, false); 00402 assert(match == false); 00403 bTreeWriter->getTupleAccessorForRead().unmarshal(bTreeTupleData); 00404 int keyComp = 00405 bitmapTupleDesc.compareTuplesKey( 00406 bTreeTupleData, 00407 bitmapEntry, 00408 nIdxKeys); 00409 if (keyComp == 0) { 00410 match = true; 00411 } 00412 } 00413 } 00414 return match; 00415 }

bool LbmSplicerExecStream::ridOverlaps ( LcsRid rid,
TupleData & bitmapTupleData,
bool firstByte
) [private]
void LbmSplicerExecStream::findBetterEntry ( TupleData const & bitmapEntry ) [private]

Determines if there exists a better entry in the btree, corresponding to the bitmap entry passed in, as compared to whatever is the current bitmap entry.

If there is, the current entry is written to the btree, and the current entry is set to the btree entry found.

Parameters:

bitmapEntry tupleData corresponding to the entry to be spliced

Definition at line 443 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, computeRowCount, createNewBitmapEntry(), currExistingEntry, findBTreeEntry(), insertBitmapEntry(), isEmpty(), LbmSegment::LbmOneByteSize, pCurrentEntry, and LbmSegment::roundToByteBoundary().

Referenced by execute().

void LbmSplicerExecStream::spliceEntry ( TupleData & bitmapEntry ) [private]

| void LbmSplicerExecStream::insertBitmapEntry | ( | | ) | [private] | | -------------------------------------------- | - | | - | ----------- |

Inserts current bitmap entry under construction into the bitmap index.

Definition at line 509 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, bTreeWriter, bTreeWriterMoved, computeRowCount, currBTreeStartRid, currExistingEntry, DUP_FAIL, findMatchingBTreeEntry(), nIdxKeys, pCurrentEntry, tempBTreeTupleData, LbmEntry::toString(), and TRACE_FINE.

Referenced by execute(), findBetterEntry(), and spliceEntry().

void LbmSplicerExecStream::createNewBitmapEntry ( TupleData const & bitmapEntry ) [private]
void LbmSplicerExecStream::upsertSingleton ( TupleData const & bitmapEntry ) [private]

Reads input tuples and filters out Rids that cause key violations.

Key violations are posted to an ErrorTarget for logging while Rids are output to a violation stream for deletion. This method completes when a valid tuple has been generated. Sets the inputTuple field.

Returns:

EXECRC_YIELD on success, or other status codes for input stream underflow, end of input stream, or violation stream overflow

Definition at line 566 of file LbmSplicerExecStream.cpp.

References LbmIterableRidReader::advance(), bitmapTupleDesc, bTreeWriter, TupleDescriptor::compareTuplesKey(), computeRowCount, countKeyRows(), BTreeBuilder::createEmptyRoot(), createNewIndex, currUniqueKey, currValidation, deletionReader, emptyTable, emptyTableUnknown, EXECRC_BUF_OVERFLOW, EXECRC_BUF_UNDERFLOW, EXECRC_YIELD, firstValidation, LbmIterableRidReader::getNext(), BTreeAccessBase::getRootPageId(), getUpsertRidPtr(), LbmIterableRidReader::hasNext(), LbmTupleRidReader::init(), inputRidReader, inputTuple, newIndexCreated, nIdxKeys, nKeyRows, NULL_PAGE_ID, nullUpsertRid, numRowsLoaded, LbmIterableRidReader::peek(), SingleInputExecStream::pInAccessor, postViolation(), SegmentAccessor::pSegment, TupleDataWithBuffer::resetBuffer(), BTreeDescriptor::rootPageId, scratchAccessor, LbmDeletionIndexReader::searchForRid(), BTreeDescriptor::segmentAccessor, setUpsertRid(), singletonTuple, LbmEntry::toString(), TRACE_FINE, uniqueRequired(), violationAccessor, violationTuple, and writeBTreeDesc.

Referenced by execute().

bool LbmSplicerExecStream::uniqueRequired ( const TupleData & tuple ) [private]

Determines whether a tuple key must be unique.

A tuple key must be unique if the index has a uniqueness constraint and the tuple key does not contain any nulls.

Parameters:

tuple input tuple to be checked for uniqueness requirments

Returns:

true if the tuple key must be unique

Definition at line 694 of file LbmSplicerExecStream.cpp.

References nIdxKeys, and uniqueKey.

Referenced by execute(), and getValidatedTuple().

00695 { 00696 if (uniqueKey) { 00697 for (uint i = 0; i < nIdxKeys; i++) { 00698 if (tuple[i].isNull()) { 00699 return false; 00700 } 00701 } 00702 return true; 00703 } 00704 return false; 00705 }

uint LbmSplicerExecStream::countKeyRows ( const TupleData & tuple ) [private]

Counts the number of rows in the index with a particular key value, prior to modification.

This count factors in the deletion index.

Parameters:

tuple tupledata containing the key value to search the index for

Returns:

number of rows with a key value, less the deleted rows

Definition at line 707 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, deletionReader, findBTreeEntry(), LbmEntry::getStartRid(), inputTuple, isEmpty(), LbmEntry::isSingleton(), LbmDeletionIndexReader::searchForRid(), and uniqueKey.

Referenced by getValidatedTuple().

void LbmSplicerExecStream::setUpsertRid ( LcsRid rid ) [inline, private]

| const LcsRid * LbmSplicerExecStream::getUpsertRidPtr | ( | | ) | const [inline, private] | | ----------------------------------------------------- | - | | - | ------------------------- |

void LbmSplicerExecStream::postViolation ( const TupleData & input,
const TupleData & violation
) [private]

Definition at line 32 of file LbmSplicerExecStream.cpp.

References bitmapTupleDesc, LbmSplicerExecStreamParams::bTreeParams, bTreeTupleData, TupleData::compute(), TupleDataWithBuffer::computeAndAllocate(), computeRowCount, BTreeExecStream::copyParamsToDescriptor(), LbmSplicerExecStreamParams::createNewIndex, createNewIndex, currUniqueKey, deletionBTreeDesc, deletionTuple, LbmEntry::getSizeBounds(), inputTuple, LbmSplicerExecStreamParams::insertRowCountParamId, insertRowCountParamId, BTreeDescriptor::keyProjection, maxEntrySize, nIdxKeys, numRowsLoaded, opaqueToInt(), origRootPageId, DiffluenceExecStream::outAccessors, outputTuple, SingleInputExecStream::pInAccessor, DiffluenceExecStream::prepare(), SegmentAccessor::pSegment, BTreeDescriptor::rootPageId, scratchAccessor, BTreeDescriptor::segmentAccessor, singletonTuple, tempBTreeTupleData, BTreeDescriptor::tupleDescriptor, uniqueKey, violationAccessor, violationTuple, writeBTreeDesc, LbmSplicerExecStreamParams::writeRowCountParamId, and writeRowCountParamId.

void LbmSplicerExecStream::open ( bool restart ) [virtual]

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:

restart if true, the stream must be already open, and should reset itself to start from the beginning of its result set

Reimplemented from DiffluenceExecStream.

Definition at line 110 of file LbmSplicerExecStream.cpp.

References bitmapBuffer, bitmapTupleDesc, bTreeWriter, bTreeWriterMoved, createNewIndex, currEntry, currExistingEntry, currValidation, deletionBTreeDesc, deletionReader, deletionTuple, emptyTable, emptyTableUnknown, firstValidation, FixedBuffer, SegmentFactory::getSnapshotSegment(), LbmDeletionIndexReader::init(), isDone, maxEntrySize, mergeBuffer, newIndexCreated, numRowsLoaded, opaqueToInt(), DiffluenceExecStream::open(), DiffluenceExecStream::outAccessors, pCurrentEntry, ExecStream::pDynamicParamManager, pSnapshotSegment, scratchAccessor, uniqueKey, writeBTreeDesc, and writeRowCountParamId.

Executes this stream.

Parameters:

quantum governs the maximum amount of execution to perform

Returns:

code indicating reason execution ceased

Implements ExecStream.

Definition at line 195 of file LbmSplicerExecStream.cpp.

References bitmapTupleDesc, TupleDatum::cbData, computeRowCount, currEntry, EXECBUF_EOS, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, EXECRC_YIELD, existingEntry(), findBetterEntry(), getValidatedTuple(), inputTuple, insertBitmapEntry(), insertRowCountParamId, isDone, newIndexCreated, ExecStreamQuantum::nTuplesMax, numRowsLoaded, opaqueToInt(), origRootPageId, DiffluenceExecStream::outAccessors, outputTuple, pCurrentEntry, TupleDatum::pData, ExecStream::pDynamicParamManager, SingleInputExecStream::pInAccessor, pSnapshotSegment, BTreeDescriptor::rootPageId, spliceEntry(), uniqueRequired(), upsertSingleton(), SnapshotRandomAllocationSegment::versionPage(), writeBTreeDesc, and writeRowCountParamId.

| void LbmSplicerExecStream::closeImpl | ( | | ) | [virtual] | | ------------------------------------ | - | | - | ----------- |

void ExecStream::prepare ( ExecStreamParams const & params ) [virtual, inherited]

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:

params instance of stream parameterization class which should be used to prepare this stream

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

void DiffluenceExecStream::setOutputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & outAccessors ) [virtual, inherited]
void SingleInputExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & inAccessors ) [virtual, inherited]

| bool ExecStream::canEarlyClose | ( | | ) | [virtual, inherited] | | ------------------------------ | - | | - | ---------------------- |

| ExecStreamId ExecStream::getStreamId | ( | | ) | const [inline, inherited] | | -------------------------------------------------------------------------------------------------- | - | | - | --------------------------- |

Returns:

the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

00289 { 00290 return id; 00291 }

Determines resource requirements for this stream.

Default implementation declares zero resource requirements.

Parameters:

minQuantity receives the minimum resource quantity needed by this stream in order to execute
optQuantity receives the resource quantity needed by this stream in order to execute optimally
optType Receives the value indicating the accuracy of the optQuantity parameter. This parameter is optional and defaults to EXEC_RESOURCE_ACCURATE if omitted. If the optimum setting is an estimate or no value can be specified (e.g., due to lack of statistics), then this parameter needs to be used to indicate a non-accurate optimum resource setting.

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 93 of file ExecStream.cpp.

References EXEC_RESOURCE_ACCURATE.

Referenced by ExternalSortExecStreamImpl::getResourceRequirements(), LcsRowScanBaseExecStream::getResourceRequirements(), LcsClusterAppendExecStream::getResourceRequirements(), LbmUnionExecStream::getResourceRequirements(), getResourceRequirements(), LbmGeneratorExecStream::getResourceRequirements(), LbmChopperExecStream::getResourceRequirements(), LhxJoinExecStream::getResourceRequirements(), LhxAggExecStream::getResourceRequirements(), FtrsTableWriterExecStream::getResourceRequirements(), BTreeReadExecStream::getResourceRequirements(), BTreeInsertExecStream::getResourceRequirements(), FlatFileExecStreamImpl::getResourceRequirements(), SegBufferWriterExecStream::getResourceRequirements(), SegBufferReaderExecStream::getResourceRequirements(), SegBufferExecStream::getResourceRequirements(), ScratchBufferExecStream::getResourceRequirements(), and DoubleBufferExecStream::getResourceRequirements().

Sets current resource allocation for this stream.

If called while the stream is open, this indicates a request for the stream to dynamically adjust its memory usage. If the stream is incapable of honoring the request, it should update quantity with the actual amounts still in use.

Parameters:

quantity allocated resource quantity

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 111 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStream::resourceAllocation.

Referenced by ExternalSortExecStreamImpl::setResourceAllocation(), LbmUnionExecStream::setResourceAllocation(), LbmGeneratorExecStream::setResourceAllocation(), LhxJoinExecStream::setResourceAllocation(), LhxAggExecStream::setResourceAllocation(), and BTreePrefetchSearchExecStream::setResourceAllocation().

| void ExecStream::setName | ( | std::string const & | | ) | [virtual, inherited] | | ------------------------ | - | ------------------- | | - | ---------------------- |

| std::string const & ExecStream::getName | ( | | ) | const [virtual, inherited] | | --------------------------------------- | - | | - | ---------------------------- |

| bool ExecStream::mayBlock | ( | | ) | const [virtual, inherited] | | ------------------------- | - | | - | ---------------------------- |

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:

whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

00168 { 00169 return false; 00170 }

| void ExecStream::checkAbort | ( | | ) | const [virtual, inherited] | | --------------------------- | - | | - | ---------------------------- |

| bool ClosableObject::isClosed | ( | | ) | const [inline, inherited] | | ----------------------------- | - | | - | --------------------------- |

Returns:

whether the object has been closed

Definition at line 58 of file ClosableObject.h.

| void ClosableObject::close | ( | | ) | [inherited] | | -------------------------- | - | | - | ------------- |

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

void TraceSource::initTraceSource ( SharedTraceTarget pTraceTarget,
std::string name
) [virtual, inherited]
void TraceSource::trace ( TraceLevel level,
std::string message
) const [inherited]

| bool TraceSource::isTracing | ( | | ) | const [inline, inherited] | | --------------------------- | - | | - | --------------------------- |

bool TraceSource::isTracingLevel ( TraceLevel level ) const [inline, inherited]

| TraceTarget& TraceSource::getTraceTarget | ( | | ) | const [inline, inherited] | | ----------------------------------------------------------------- | - | | - | --------------------------- |

| std::string TraceSource::getTraceSourceName | ( | | ) | const [inline, inherited] | | ------------------------------------------- | - | | - | --------------------------- |

void TraceSource::setTraceSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137 { 00138 name = n; 00139 }

| TraceLevel TraceSource::getMinimumTraceLevel | ( | | ) | const [inline, inherited] | | ------------------------------------------------------------------------------------------------------ | - | | - | --------------------------- |

| void TraceSource::disableTracing | ( | | ) | [inherited] | | -------------------------------- | - | | - | ------------- |

void ErrorSource::initErrorSource ( SharedErrorTarget pErrorTarget,
const std::string & name
) [virtual, inherited]
void ErrorSource::postError ( ErrorLevel level,
const std::string & message,
void * address,
long capacity,
int index
) [inherited]

| bool ErrorSource::hasTarget | ( | | ) | const [inline, inherited] | | --------------------------- | - | | - | --------------------------- |

| ErrorTarget& ErrorSource::getErrorTarget | ( | | ) | const [inline, inherited] | | ----------------------------------------------------------------- | - | | - | --------------------------- |

Returns:

the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

| std::string ErrorSource::getErrorSourceName | ( | | ) | const [inline, inherited] | | ------------------------------------------- | - | | - | --------------------------- |

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:

the name

Definition at line 139 of file ErrorSource.h.

00140 { 00141 return name; 00142 }

void ErrorSource::setErrorSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

00149 { 00150 name = n; 00151 }

| void ErrorSource::disableTarget | ( | | ) | [inherited] | | ------------------------------- | - | | - | ------------- |

Member Data Documentation

True if a new index was created.

This will only be true if createNewIndex is true and there is at least one input tuple, indicating that the existing index needs to be recreated and the new one versioned off of the original.

Definition at line 112 of file LbmSplicerExecStream.h.

Referenced by execute(), getValidatedTuple(), and open().

True if we need to determine emptyTable.

We can't do this in init() because upstream ExecStream's may insert into the index being updated. An example of early update is when the deletion phase of a merge statement appends the deletion index. The same index may be later is appended for merge violations.

Definition at line 163 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), isEmpty(), and open().

Whether btree writer position was moved for unique constraint validation.

If currExistingEntry is true, then the btree writer is expected to stay positioned at the existing entry. However, constraint validation may run side searches and can reposition the btree writer.

Definition at line 182 of file LbmSplicerExecStream.h.

Referenced by existingEntry(), insertBitmapEntry(), and open().

Underlying snapshot segment corresponding to the index that will be written.

Only used when a new index is dynamically created.

Definition at line 314 of file LbmSplicerExecStream.h.

Referenced by execute(), and open().

Output tuple descriptor.

Currently, all outputs must have the same descriptor.

Definition at line 69 of file DiffluenceExecStream.h.

Definition at line 51 of file SingleInputExecStream.h.

Referenced by SortedAggExecStream::compareGroupByKeys(), ExternalSortExecStreamImpl::computeFirstResult(), ExternalSortExecStreamImpl::execute(), execute(), FtrsTableWriterExecStream::execute(), BTreeSortExecStream::execute(), BTreeSearchUniqueExecStream::execute(), BTreeSearchExecStream::execute(), BTreePrefetchSearchExecStream::execute(), BTreeInsertExecStream::execute(), JavaSinkExecStream::execute(), UncollectExecStream::execute(), SplitterExecStream::execute(), SortedAggExecStream::execute(), SegBufferWriterExecStream::execute(), SegBufferReaderExecStream::execute(), SegBufferExecStream::execute(), ScratchBufferExecStream::execute(), ReshapeExecStream::execute(), MockConsumerExecStream::execute(), DoubleBufferExecStream::execute(), CopyExecStream::execute(), CollectExecStream::execute(), BernoulliSamplingExecStream::execute(), CalcExecStream::execute(), BTreePrefetchSearchExecStream::getNextPageForPrefetch(), LcsClusterReplaceExecStream::getTupleForLoad(), LcsClusterAppendExecStream::getTupleForLoad(), getValidatedTuple(), LcsClusterReplaceExecStream::initTupleLoadParams(), BTreeSearchExecStream::innerSearchLoop(), BTreePrefetchSearchExecStream::innerSearchLoop(), LcsClusterReplaceExecStream::open(), LbmNormalizerExecStream::open(), SingleInputExecStream::open(), SegBufferWriterExecStream::open(), SegBufferExecStream::open(), ScratchBufferExecStream::open(), MockConsumerExecStream::open(), DoubleBufferExecStream::open(), CollectExecStream::open(), LcsClusterAppendExecStream::postProcessTuple(), ConduitExecStream::precheckConduitBuffers(), ExternalSortExecStreamImpl::prepare(), LcsClusterAppendExecStream::prepare(), prepare(), LbmNormalizerExecStream::prepare(), BTreeSearchExecStream::prepare(), BTreeInsertExecStream::prepare(), UncollectExecStream::prepare(), SortedAggExecStream::prepare(), SingleInputExecStream::prepare(), SegBufferReaderExecStream::prepare(), ReshapeExecStream::prepare(), DiffluenceExecStream::prepare(), ConduitExecStream::prepare(), CollectExecStream::prepare(), BernoulliSamplingExecStream::prepare(), CalcExecStream::prepare(), SortedAggExecStream::produce(), BTreeSearchExecStream::readSearchKey(), LhxAggExecStream::setAggComputers(), LhxAggExecStream::setHashInfo(), SingleInputExecStream::setInputBufAccessors(), and JavaSinkExecStream::stuffByteBuffer().

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

Referenced by CorrelationJoinExecStream::close(), SegBufferReaderExecStream::closeImpl(), LbmUnionExecStream::execute(), execute(), LbmGeneratorExecStream::execute(), LbmChopperExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), LcsClusterReplaceExecStream::open(), LbmUnionExecStream::open(), open(), LbmGeneratorExecStream::open(), LbmBitOpExecStream::open(), BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), SegBufferWriterExecStream::open(), SegBufferReaderExecStream::open(), NestedLoopJoinExecStream::open(), CorrelationJoinExecStream::open(), ExecStream::prepare(), CalcExecStream::prepare(), NestedLoopJoinExecStream::processLeftInput(), LbmSearchExecStream::reachedTupleLimit(), ReshapeExecStream::readDynamicParams(), SegBufferWriterExecStream::readReaderRefCount(), BTreeSearchExecStream::readSearchKey(), BTreeSearchExecStream::readUpperBoundKey(), LbmSearchExecStream::setAdditionalKeys(), and LbmBitOpExecStream::writeStartRidParamValue().

The documentation for this class was generated from the following files:

Generated on Mon Jun 22 04:00:36 2009 for Fennel by doxygen 1.5.1