Fennel: SegBufferWriterExecStream Class Reference (original) (raw)

• Main Page

• Namespaces

• Classes

• Files

• Directories

• Related Pages

• Alphabetical List

• Class List

• Class Hierarchy

• Class Members

SegBufferWriterExecStream reads its input stream and writes the data to a buffer so it can be read by one or more SegBufferReaderExecStreams. More...

#include <[SegBufferWriterExecStream.h](SegBufferWriterExecStream%5F8h-source.html)>

Inheritance diagram for SegBufferWriterExecStream:

List of all members.

Detailed Description

SegBufferWriterExecStream reads its input stream and writes the data to a buffer so it can be read by one or more SegBufferReaderExecStreams.

It does not write the buffered data to its output stream. When this stream has completed buffering its input, it will pass the first pageId of the buffered data through its output stream to any reader streams that are ready to start processing data.

The writer and reader stream instances all share a dynamic parameter. The dynamic parameter is created by this stream and read by the reader streams. The parameter is a reference counter used to keep track of the number of active readers, which this stream uses to determine if it can destroy the buffered data.

Version:

Id

//open/dev/fennel/exec/SegBufferWriterExecStream.h#3

Definition at line 65 of file SegBufferWriterExecStream.h.

Member Function Documentation

| int64_t SegBufferWriterExecStream::readReaderRefCount | ( | | ) | [private] | | ---------------------------------------------------------------------------------------------------------- | - | | - | ----------- |

| bool SegBufferWriterExecStream::canEarlyClose | ( | | ) | [virtual] | | --------------------------------------------- | - | | - | ----------- |

Definition at line 34 of file SegBufferWriterExecStream.cpp.

References bufferSegmentAccessor, TupleData::compute(), firstBufferPageId, DiffluenceExecStream::outAccessors, outputBufSize, outputTuple, outputWritten, paramCreated, DiffluenceExecStream::prepare(), SegBufferWriterExecStreamParams::readerRefCountParamId, and readerRefCountParamId.

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:

Reimplemented from DiffluenceExecStream.

Definition at line 63 of file SegBufferWriterExecStream.cpp.

References BUFPROV_CONSUMER, firstBufferPageId, FixedBuffer, StandardTypeDescriptorFactory::newDataType(), nOutputsWritten, NULL_PAGE_ID, DiffluenceExecStream::open(), outputBufSize, outputTupleBuffer, outputWritten, paramCreated, ExecStream::pDynamicParamManager, SingleInputExecStream::pInAccessor, readerRefCountParamId, and STANDARD_TYPE_UINT_64.

Executes this stream.

Parameters:

Returns:

code indicating reason execution ceased

Implements ExecStream.

Definition at line 114 of file SegBufferWriterExecStream.cpp.

References bufferSegmentAccessor, ExecStreamGraphImpl::closeProducers(), EXECBUF_EMPTY, EXECBUF_EOS, EXECBUF_NONEMPTY, EXECBUF_OVERFLOW, EXECBUF_UNDERFLOW, EXECRC_BUF_OVERFLOW, EXECRC_EOS, firstBufferPageId, ExecStream::getGraph(), ExecStream::getStreamId(), TupleAccessor::marshal(), SegBufferWriter::newSegBufferWriter(), nOutputsWritten, NULL_PAGE_ID, DiffluenceExecStream::outAccessors, outputBufSize, outputTuple, outputTupleBuffer, outputWritten, SingleInputExecStream::pInAccessor, and pSegBufferWriter.

| void SegBufferWriterExecStream::closeImpl | ( | | ) | [virtual] | | ----------------------------------------- | - | | - | ----------- |

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

| ExecStreamId ExecStream::getStreamId | ( | | ) | const [inline, inherited] | | -------------------------------------------------------------------------------------------------- | - | | - | --------------------------- |

Returns:

the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

00289 { 00290 return id; 00291 }

Determines resource requirements for this stream.

Default implementation declares zero resource requirements.

Parameters:

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 93 of file ExecStream.cpp.

References EXEC_RESOURCE_ACCURATE.

Referenced by ExternalSortExecStreamImpl::getResourceRequirements(), LcsRowScanBaseExecStream::getResourceRequirements(), LcsClusterAppendExecStream::getResourceRequirements(), LbmUnionExecStream::getResourceRequirements(), LbmSplicerExecStream::getResourceRequirements(), LbmGeneratorExecStream::getResourceRequirements(), LbmChopperExecStream::getResourceRequirements(), LhxJoinExecStream::getResourceRequirements(), LhxAggExecStream::getResourceRequirements(), FtrsTableWriterExecStream::getResourceRequirements(), BTreeReadExecStream::getResourceRequirements(), BTreeInsertExecStream::getResourceRequirements(), FlatFileExecStreamImpl::getResourceRequirements(), getResourceRequirements(), SegBufferReaderExecStream::getResourceRequirements(), SegBufferExecStream::getResourceRequirements(), ScratchBufferExecStream::getResourceRequirements(), and DoubleBufferExecStream::getResourceRequirements().

Sets current resource allocation for this stream.

If called while the stream is open, this indicates a request for the stream to dynamically adjust its memory usage. If the stream is incapable of honoring the request, it should update quantity with the actual amounts still in use.

Parameters:

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 111 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStream::resourceAllocation.

Referenced by ExternalSortExecStreamImpl::setResourceAllocation(), LbmUnionExecStream::setResourceAllocation(), LbmGeneratorExecStream::setResourceAllocation(), LhxJoinExecStream::setResourceAllocation(), LhxAggExecStream::setResourceAllocation(), and BTreePrefetchSearchExecStream::setResourceAllocation().

| void ExecStream::setName | ( | std::string const & | | ) | [virtual, inherited] | | ------------------------ | - | ------------------- | | - | ---------------------- |

| std::string const & ExecStream::getName | ( | | ) | const [virtual, inherited] | | --------------------------------------- | - | | - | ---------------------------- |

| bool ExecStream::mayBlock | ( | | ) | const [virtual, inherited] | | ------------------------- | - | | - | ---------------------------- |

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:

whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

00168 { 00169 return false; 00170 }

| void ExecStream::checkAbort | ( | | ) | const [virtual, inherited] | | --------------------------- | - | | - | ---------------------------- |

| bool ClosableObject::isClosed | ( | | ) | const [inline, inherited] | | ----------------------------- | - | | - | --------------------------- |

Returns:

whether the object has been closed

Definition at line 58 of file ClosableObject.h.

| void ClosableObject::close | ( | | ) | [inherited] | | -------------------------- | - | | - | ------------- |

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

| bool TraceSource::isTracing | ( | | ) | const [inline, inherited] | | --------------------------- | - | | - | --------------------------- |

| TraceTarget& TraceSource::getTraceTarget | ( | | ) | const [inline, inherited] | | ----------------------------------------------------------------- | - | | - | --------------------------- |

| std::string TraceSource::getTraceSourceName | ( | | ) | const [inline, inherited] | | ------------------------------------------- | - | | - | --------------------------- |

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137 { 00138 name = n; 00139 }

| TraceLevel TraceSource::getMinimumTraceLevel | ( | | ) | const [inline, inherited] | | ------------------------------------------------------------------------------------------------------ | - | | - | --------------------------- |

| void TraceSource::disableTracing | ( | | ) | [inherited] | | -------------------------------- | - | | - | ------------- |

| bool ErrorSource::hasTarget | ( | | ) | const [inline, inherited] | | --------------------------- | - | | - | --------------------------- |

| ErrorTarget& ErrorSource::getErrorTarget | ( | | ) | const [inline, inherited] | | ----------------------------------------------------------------- | - | | - | --------------------------- |

Returns:

the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

| std::string ErrorSource::getErrorSourceName | ( | | ) | const [inline, inherited] | | ------------------------------------------- | - | | - | --------------------------- |

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:

the name

Definition at line 139 of file ErrorSource.h.

00140 { 00141 return name; 00142 }

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

00149 { 00150 name = n; 00151 }

| void ErrorSource::disableTarget | ( | | ) | [inherited] | | ------------------------------- | - | | - | ------------- |

Member Data Documentation

Output tuple descriptor.

Currently, all outputs must have the same descriptor.

Definition at line 69 of file DiffluenceExecStream.h.

Definition at line 51 of file SingleInputExecStream.h.

Referenced by SortedAggExecStream::compareGroupByKeys(), ExternalSortExecStreamImpl::computeFirstResult(), ExternalSortExecStreamImpl::execute(), LbmSplicerExecStream::execute(), FtrsTableWriterExecStream::execute(), BTreeSortExecStream::execute(), BTreeSearchUniqueExecStream::execute(), BTreeSearchExecStream::execute(), BTreePrefetchSearchExecStream::execute(), BTreeInsertExecStream::execute(), JavaSinkExecStream::execute(), UncollectExecStream::execute(), SplitterExecStream::execute(), SortedAggExecStream::execute(), execute(), SegBufferReaderExecStream::execute(), SegBufferExecStream::execute(), ScratchBufferExecStream::execute(), ReshapeExecStream::execute(), MockConsumerExecStream::execute(), DoubleBufferExecStream::execute(), CopyExecStream::execute(), CollectExecStream::execute(), BernoulliSamplingExecStream::execute(), CalcExecStream::execute(), BTreePrefetchSearchExecStream::getNextPageForPrefetch(), LcsClusterReplaceExecStream::getTupleForLoad(), LcsClusterAppendExecStream::getTupleForLoad(), LbmSplicerExecStream::getValidatedTuple(), LcsClusterReplaceExecStream::initTupleLoadParams(), BTreeSearchExecStream::innerSearchLoop(), BTreePrefetchSearchExecStream::innerSearchLoop(), LcsClusterReplaceExecStream::open(), LbmNormalizerExecStream::open(), SingleInputExecStream::open(), open(), SegBufferExecStream::open(), ScratchBufferExecStream::open(), MockConsumerExecStream::open(), DoubleBufferExecStream::open(), CollectExecStream::open(), LcsClusterAppendExecStream::postProcessTuple(), ConduitExecStream::precheckConduitBuffers(), ExternalSortExecStreamImpl::prepare(), LcsClusterAppendExecStream::prepare(), LbmSplicerExecStream::prepare(), LbmNormalizerExecStream::prepare(), BTreeSearchExecStream::prepare(), BTreeInsertExecStream::prepare(), UncollectExecStream::prepare(), SortedAggExecStream::prepare(), SingleInputExecStream::prepare(), SegBufferReaderExecStream::prepare(), ReshapeExecStream::prepare(), DiffluenceExecStream::prepare(), ConduitExecStream::prepare(), CollectExecStream::prepare(), BernoulliSamplingExecStream::prepare(), CalcExecStream::prepare(), SortedAggExecStream::produce(), BTreeSearchExecStream::readSearchKey(), LhxAggExecStream::setAggComputers(), LhxAggExecStream::setHashInfo(), SingleInputExecStream::setInputBufAccessors(), and JavaSinkExecStream::stuffByteBuffer().

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

Referenced by CorrelationJoinExecStream::close(), SegBufferReaderExecStream::closeImpl(), LbmUnionExecStream::execute(), LbmSplicerExecStream::execute(), LbmGeneratorExecStream::execute(), LbmChopperExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), LcsClusterReplaceExecStream::open(), LbmUnionExecStream::open(), LbmSplicerExecStream::open(), LbmGeneratorExecStream::open(), LbmBitOpExecStream::open(), BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), open(), SegBufferReaderExecStream::open(), NestedLoopJoinExecStream::open(), CorrelationJoinExecStream::open(), ExecStream::prepare(), CalcExecStream::prepare(), NestedLoopJoinExecStream::processLeftInput(), LbmSearchExecStream::reachedTupleLimit(), ReshapeExecStream::readDynamicParams(), readReaderRefCount(), BTreeSearchExecStream::readSearchKey(), BTreeSearchExecStream::readUpperBoundKey(), LbmSearchExecStream::setAdditionalKeys(), and LbmBitOpExecStream::writeStartRidParamValue().

The documentation for this class was generated from the following files:

• /home/pub/open/dev/fennel/exec/SegBufferWriterExecStream.h
• /home/pub/open/dev/fennel/exec/SegBufferWriterExecStream.cpp

Generated on Mon Jun 22 04:00:45 2009 for Fennel by 1.5.1